Aren/NetBSD
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
72507061e4
NetBSD/sys/ufs/lfs/lfs_segment.c

2848 lines
76 KiB
C
Raw Normal View History

Install a new sysctl, vfs.lfs.ignore_lazy_sync, which causes LFS to ignore the "smooth" syncer, as if vfs.sync.*delay = 0, but only for LFS. The default is "on", i.e., ignore lazy sync. Reduce the amount of polling/busy-waiting done by lfs_putpages(). To accomplish this, copied genfs_putpages() and modified it to indicate which page it was that caused it to return with EDEADLK. fsync()/fdatasync() should no longer ever fail with EAGAIN, and should not consume huge quantities of cpu. Also, try to make dirops less likely to be written as the result of a VOP_PUTPAGES(), while ensuring that they are written regularly.
2007-04-17 05:16:46 +04:00
/* $NetBSD: lfs_segment.c,v 1.199 2007/04/17 01:16:46 perseant Exp $ */
New RCS ID's, take two. they're more aesthecially pleasant, and use 'NetBSD'
1994-06-29 10:39:25 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*-
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* Copyright (c) 1999, 2000, 2001, 2002, 2003 The NetBSD Foundation, Inc.
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
* All rights reserved.
*
* This code is derived from software contributed to The NetBSD Foundation
* by Konrad E. Schroder <perseant@hhhh.org>.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* This product includes software developed by the NetBSD
* Foundation, Inc. and its contributors.
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
* 4. Neither the name of The NetBSD Foundation nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Copyright (c) 1991, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
Move UCB-licensed code from 4-clause to 3-clause licence. Patches provided by Joel Baker in PR 22364, verified by myself.
2003-08-07 20:26:28 +04:00
* 3. Neither the name of the University nor the names of its contributors
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
* @(#)lfs_segment.c 8.10 (Berkeley) 6/10/95
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
add RCSID
2001-11-08 05:39:06 +03:00
#include <sys/cdefs.h>
Install a new sysctl, vfs.lfs.ignore_lazy_sync, which causes LFS to ignore the "smooth" syncer, as if vfs.sync.*delay = 0, but only for LFS. The default is "on", i.e., ignore lazy sync. Reduce the amount of polling/busy-waiting done by lfs_putpages(). To accomplish this, copied genfs_putpages() and modified it to indicate which page it was that caused it to return with EDEADLK. fsync()/fdatasync() should no longer ever fail with EAGAIN, and should not consume huge quantities of cpu. Also, try to make dirops less likely to be written as the result of a VOP_PUTPAGES(), while ensuring that they are written regularly.
2007-04-17 05:16:46 +04:00
__KERNEL_RCSID(0, "$NetBSD: lfs_segment.c,v 1.199 2007/04/17 01:16:46 perseant Exp $");
add RCSID
2001-11-08 05:39:06 +03:00
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
#ifdef DEBUG
# define vndebug(vp, str) do { \
if (VTOI(vp)->i_flag & IN_CLEANING) \
DLOG((DLOG_WVNODE, "not writing ino %d because %s (op %d)\n", \
VTOI(vp)->i_number, (str), op)); \
} while(0)
#else
# define vndebug(vp, str)
#endif
#define ivndebug(vp, str) \
DLOG((DLOG_WVNODE, "ino %d: %s\n", VTOI(vp)->i_number, (str)))
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
use _KERNEL_OPT
2001-05-30 15:57:16 +04:00
#if defined(_KERNEL_OPT)
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
#include "opt_ddb.h"
only include opt_ddb.h for !LKM
2000-11-30 18:59:47 +03:00
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#include <sys/param.h>
#include <sys/systm.h>
#include <sys/namei.h>
#include <sys/kernel.h>
#include <sys/resourcevar.h>
#include <sys/file.h>
#include <sys/stat.h>
#include <sys/buf.h>
#include <sys/proc.h>
#include <sys/vnode.h>
#include <sys/mount.h>
integrate kauth.
2006-05-15 01:31:52 +04:00
#include <sys/kauth.h>
Change LFCNWRAP{STOP,GO} to make them more suitable for snapshotting; in particular, the caller can now choose whether to wait for the condition to be met, and if the caller of LFCNWRAPSTOP dies or otherwise closes the descriptor, the filesystem is started again. Updated the ckckp regression test to use the new semantics. dump_lfs(8) now uses the fcntls to implement LFS-style snapshotting through the -X flag, addressing PR#33457 albeit not using fss(4). Fixed a couple other problems with dump_lfs that manifested themselves during testing.
2006-06-24 09:28:54 +04:00
#include <sys/syslog.h>
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#include <miscfs/specfs/specdev.h>
#include <miscfs/fifofs/fifo.h>
#include <ufs/ufs/inode.h>
#include <ufs/ufs/dir.h>
#include <ufs/ufs/ufsmount.h>
#include <ufs/ufs/ufs_extern.h>
#include <ufs/lfs/lfs.h>
#include <ufs/lfs/lfs_extern.h>
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
#include <uvm/uvm.h>
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#include <uvm/uvm_extern.h>
Add extensible malloc types, adapted from FreeBSD. This turns malloc types into a structure, a pointer to which is passed around, instead of an int constant. Allow the limit to be adjusted when the malloc type is defined, or with a function call, as suggested by Jonathan Stone.
2003-02-01 09:23:35 +03:00
MALLOC_DEFINE(M_SEGMENT, "LFS segment", "Segment for LFS");
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
extern int count_lock_queue(void);
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
extern struct simplelock vnode_free_list_slock; /* XXX */
- buffer cache MP locks. - avoid changing buffer state on the free queue.
2003-09-07 15:47:07 +04:00
extern struct simplelock bqueue_slock; /* XXX */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
static void lfs_generic_callback(struct buf *, void (*)(struct buf *));
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
static void lfs_free_aiodone(struct buf *);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
static void lfs_super_aiodone(struct buf *);
static void lfs_cluster_aiodone(struct buf *);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
static void lfs_cluster_callback(struct buf *);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Determine if it's OK to start a partial in this segment, or if we need
* to go on to a new segment.
*/
#define LFS_PARTIAL_FITS(fs) \
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
((fs)->lfs_fsbpseg - ((fs)->lfs_offset - (fs)->lfs_curseg) > \
fragstofsb((fs), (fs)->lfs_frag))
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
/*
* Figure out whether we should do a checkpoint write or go ahead with
* an ordinary write.
*/
#define LFS_SHOULD_CHECKPOINT(fs, flags) \
Don't try to write all the vnodes, when the cleaner needs a vnode to be recycled.
2006-07-21 03:12:26 +04:00
((flags & SEGM_CLEAN) == 0 && \
((fs->lfs_nactive > LFS_MAX_ACTIVE || \
(flags & SEGM_CKP) || \
fs->lfs_nclean < LFS_MAX_ACTIVE)))
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
int lfs_match_fake(struct lfs *, struct buf *);
void lfs_newseg(struct lfs *);
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
/* XXX ondisk32 */
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
void lfs_shellsort(struct buf **, int32_t *, int, int);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
void lfs_supercallback(struct buf *);
void lfs_updatemeta(struct segment *);
void lfs_writesuper(struct lfs *, daddr_t);
int lfs_writevnodes(struct lfs *fs, struct mount *mp,
struct segment *sp, int dirops);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int lfs_allclean_wakeup; /* Cleaner wakeup address. */
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
int lfs_writeindir = 1; /* whether to flush indir on non-ckp */
Better checking for held inode locks in lfs_fastvget, for a number of error conditions. Also change the default setting of lfs_clean_vnhead to 0, which seems to make the locking problems go away (although this is difficult to test as I can't reliably reproduce them).
1999-04-12 04:11:01 +04:00
int lfs_clean_vnhead = 0; /* Allow freeing to head of vn list */
Address ufs_hashlock/ufs_ihashins protocol bug, discovered while doing a post-mortem of a production machine. Also, take the active dirop count off of the fs and make it global (since it is measuring a global resource) and tie the threshold value LFS_MAXDIROP to desiredvnodes.
1999-11-06 23:33:05 +03:00
int lfs_dirvcount = 0; /* # active dirops */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Statistics Counters */
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int lfs_dostats = 1;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
struct lfs_stats lfs_stats;
/* op values to lfs_writevnodes */
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
#define VN_REG 0
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#define VN_DIROP 1
#define VN_EMPTY 2
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
#define VN_CLEAN 3
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
* XXX KS - Set modification time on the Ifile, so the cleaner can
* read the fs mod time off of it. We don't set IN_UPDATE here,
* since we don't really need this to be flushed to disk (and in any
* case that wouldn't happen to the Ifile until we checkpoint).
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_imtime(struct lfs *fs)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
{
struct timespec ts;
struct inode *ip;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_MAYBE_SEGLOCK(fs);
fix a simonb-timecounters regression. the precision of getnanotime() is not suitable for file timestamps. esp. when it's nfs-exported. - introduce vfs_timestamp(). (the name is from freebsd. currently merely a wrapper of nanotime()) - for ufs-like filesystems, use it rather than getnanotime(). XXX check other filesystems.
2006-06-23 18:13:02 +04:00
vfs_timestamp(&ts);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
ip = VTOI(fs->lfs_ivnode);
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
ip->i_ffs1_mtime = ts.tv_sec;
ip->i_ffs1_mtimensec = ts.tv_nsec;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Ifile and meta data blocks are not marked busy, so segment writes MUST be
* single threaded. Currently, there are two paths into lfs_segwrite, sync()
* and getnewbuf(). They both mark the file system busy. Lfs_vflush()
* explicitly marks the file system busy. So lfs_segwrite is safe. I think.
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#define IS_FLUSHING(fs,vp) ((fs)->lfs_flushvp == (vp))
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_vflush(struct vnode *vp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct inode *ip;
struct lfs *fs;
struct segment *sp;
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
struct buf *bp, *nbp, *tbp, *tnbp;
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
int error, s;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
int flushed;
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
int relock;
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
int loopcount;
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
Move variable initialization to the top of lfs_vflush
1999-03-30 20:03:16 +04:00
ip = VTOI(vp);
fs = VFSTOUFS(vp->v_mount)->um_lfs;
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
relock = 0;
Move variable initialization to the top of lfs_vflush
1999-03-30 20:03:16 +04:00
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
top:
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_NO_SEGLOCK(fs);
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ip->i_flag & IN_CLEANING) {
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
ivndebug(vp,"vflush/in_cleaning");
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
LFS_CLR_UINO(ip, IN_CLEANING);
LFS_SET_UINO(ip, IN_MODIFIED);
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
/*
* Toss any cleaning buffers that have real counterparts
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* to avoid losing new data.
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
*/
s = splbio();
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
for (bp = LIST_FIRST(&vp->v_dirtyblkhd); bp; bp = nbp) {
nbp = LIST_NEXT(bp, b_vnbufs);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (!LFS_IS_MALLOC_BUF(bp))
continue;
/*
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
* Look for pages matching the range covered
* by cleaning blocks. It's okay if more dirty
* pages appear, so long as none disappear out
* from under us.
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
*/
if (bp->b_lblkno > 0 && vp->v_type == VREG &&
vp != fs->lfs_ivnode) {
struct vm_page *pg;
voff_t off;
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
simple_lock(&vp->v_interlock);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
for (off = lblktosize(fs, bp->b_lblkno);
off < lblktosize(fs, bp->b_lblkno + 1);
off += PAGE_SIZE) {
pg = uvm_pagelookup(&vp->v_uobj, off);
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
if (pg == NULL)
continue;
if ((pg->flags & PG_CLEAN) == 0 ||
pmap_is_modified(pg)) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
fs->lfs_avail += btofsb(fs,
bp->b_bcount);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
wakeup(&fs->lfs_avail);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_freebuf(fs, bp);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
bp = NULL;
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
simple_unlock(&vp->v_interlock);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
goto nextbp;
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
}
}
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
simple_unlock(&vp->v_interlock);
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
for (tbp = LIST_FIRST(&vp->v_dirtyblkhd); tbp;
tbp = tnbp)
{
tnbp = LIST_NEXT(tbp, b_vnbufs);
if (tbp->b_vp == bp->b_vp
&& tbp->b_lblkno == bp->b_lblkno
&& tbp != bp)
{
fs->lfs_avail += btofsb(fs,
bp->b_bcount);
wakeup(&fs->lfs_avail);
lfs_freebuf(fs, bp);
bp = NULL;
break;
}
}
nextbp:
SO C requires a statement after a label.
2003-03-15 05:27:18 +03:00
;
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
}
splx(s);
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
}
/* If the node is being written, wait until that is done */
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&vp->v_interlock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
s = splbio();
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (WRITEINPROG(vp)) {
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
ivndebug(vp,"vflush/writeinprog");
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ltsleep(vp, (PRIBIO+1), "lfs_vw", 0, &vp->v_interlock);
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
splx(s);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&vp->v_interlock);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/* Protect against VXLOCK deadlock in vinvalbuf() */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_seglock(fs, SEGM_SYNC);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
/* If we're supposed to flush a freed inode, just toss it */
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when many inodes are cleaned at once. Make sure that we write all the pages on vnodes that are being flushed, even if we don't think there's room; drain v_numoutput before lfs_vflush() completes. Also, don't allow a vnode that is in the process of being cleaned to be chosen by getnewvnode(); this avoids a segment accounting panic in the case that a large number of inodes are fed to lfs_markv() all at once.
2006-05-13 03:36:11 +04:00
if (ip->i_lfs_iflags & LFSI_DELETED) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_VNODE, "lfs_vflush: ino %d freed, not flushing\n",
ip->i_number));
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
s = splbio();
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when many inodes are cleaned at once. Make sure that we write all the pages on vnodes that are being flushed, even if we don't think there's room; drain v_numoutput before lfs_vflush() completes. Also, don't allow a vnode that is in the process of being cleaned to be chosen by getnewvnode(); this avoids a segment accounting panic in the case that a large number of inodes are fed to lfs_markv() all at once.
2006-05-13 03:36:11 +04:00
/* Drain v_numoutput */
simple_lock(&global_v_numoutput_slock);
while (vp->v_numoutput > 0) {
vp->v_flag |= VBWAIT;
ltsleep(&vp->v_numoutput, PRIBIO + 1, "lfs_vf4", 0,
&global_v_numoutput_slock);
}
simple_unlock(&global_v_numoutput_slock);
KASSERT(vp->v_numoutput == 0);
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
for (bp = LIST_FIRST(&vp->v_dirtyblkhd); bp; bp = nbp) {
nbp = LIST_NEXT(bp, b_vnbufs);
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when many inodes are cleaned at once. Make sure that we write all the pages on vnodes that are being flushed, even if we don't think there's room; drain v_numoutput before lfs_vflush() completes. Also, don't allow a vnode that is in the process of being cleaned to be chosen by getnewvnode(); this avoids a segment accounting panic in the case that a large number of inodes are fed to lfs_markv() all at once.
2006-05-13 03:36:11 +04:00
KASSERT((bp->b_flags & B_GATHERED) == 0);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
if (bp->b_flags & B_DELWRI) { /* XXX always true? */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_avail += btofsb(fs, bp->b_bcount);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
wakeup(&fs->lfs_avail);
}
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
/* Copied from lfs_writeseg */
if (bp->b_flags & B_CALL) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
biodone(bp);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
} else {
bremfree(bp);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
LFS_UNLOCK_BUF(bp);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
bp->b_flags &= ~(B_ERROR | B_READ | B_DELWRI |
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
B_GATHERED);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
bp->b_flags |= B_DONE;
reassignbuf(bp, vp);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
brelse(bp);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
}
}
splx(s);
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
LFS_CLR_UINO(ip, IN_CLEANING);
LFS_CLR_UINO(ip, IN_MODIFIED | IN_ACCESSED);
update for IN_ACCESSED changes
2000-05-31 05:40:01 +04:00
ip->i_flag &= ~IN_ALLMOD;
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_VNODE, "lfs_vflush: done not flushing ino %d\n",
ip->i_number));
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
lfs_segunlock(fs);
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when many inodes are cleaned at once. Make sure that we write all the pages on vnodes that are being flushed, even if we don't think there's room; drain v_numoutput before lfs_vflush() completes. Also, don't allow a vnode that is in the process of being cleaned to be chosen by getnewvnode(); this avoids a segment accounting panic in the case that a large number of inodes are fed to lfs_markv() all at once.
2006-05-13 03:36:11 +04:00
KASSERT(LIST_FIRST(&vp->v_dirtyblkhd) == NULL);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
return 0;
}
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
fs->lfs_flushvp = vp;
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
if (LFS_SHOULD_CHECKPOINT(fs, fs->lfs_sp->seg_flags)) {
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
error = lfs_segwrite(vp->v_mount, SEGM_CKP | SEGM_SYNC);
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
fs->lfs_flushvp = NULL;
KASSERT(fs->lfs_flushvp_fakevref == 0);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
lfs_segunlock(fs);
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when many inodes are cleaned at once. Make sure that we write all the pages on vnodes that are being flushed, even if we don't think there's room; drain v_numoutput before lfs_vflush() completes. Also, don't allow a vnode that is in the process of being cleaned to be chosen by getnewvnode(); this avoids a segment accounting panic in the case that a large number of inodes are fed to lfs_markv() all at once.
2006-05-13 03:36:11 +04:00
/* Make sure that any pending buffers get written */
s = splbio();
simple_lock(&global_v_numoutput_slock);
while (vp->v_numoutput > 0) {
vp->v_flag |= VBWAIT;
ltsleep(&vp->v_numoutput, PRIBIO + 1, "lfs_vf3", 0,
&global_v_numoutput_slock);
}
simple_unlock(&global_v_numoutput_slock);
splx(s);
KASSERT(LIST_FIRST(&vp->v_dirtyblkhd) == NULL);
KASSERT(vp->v_numoutput == 0);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
return error;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp = fs->lfs_sp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
flushed = 0;
if (VPISEMPTY(vp)) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_writevnodes(fs, vp->v_mount, sp, VN_EMPTY);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
++flushed;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
} else if ((ip->i_flag & IN_CLEANING) &&
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
(fs->lfs_sp->seg_flags & SEGM_CLEAN)) {
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
ivndebug(vp,"vflush/clean");
lfs_writevnodes(fs, vp->v_mount, sp, VN_CLEAN);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
++flushed;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
} else if (lfs_dostats) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (!VPISEMPTY(vp) || (VTOI(vp)->i_flag & IN_ALLMOD))
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.vflush_invoked;
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
ivndebug(vp,"vflush");
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
#ifdef DIAGNOSTIC
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_flag & VDIROP) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_VNODE, "lfs_vflush: flushing VDIROP\n"));
/* panic("lfs_vflush: VDIROP being flushed...this can\'t happen"); */
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_usecount < 0) {
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
printf("usecount=%ld\n", (long)vp->v_usecount);
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
panic("lfs_vflush: usecount<0");
}
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
do {
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
loopcount = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
do {
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
if (LIST_FIRST(&vp->v_dirtyblkhd) != NULL) {
relock = lfs_writefile(fs, sp, vp);
if (relock) {
/*
* Might have to wait for the
* cleaner to run; but we're
* still not done with this vnode.
*/
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
KDASSERT(ip->i_number != LFS_IFILE_INUM);
Postpone the segment accounting changes coming from truncation until the inode that makes those changes valid is either written to disk by lfs_writeinode() or discarded by lfs_vfree(). A couple of locking fixes are also included as well.
2006-05-01 01:19:42 +04:00
lfs_writeinode(fs, sp, ip);
LFS_SET_UINO(ip, IN_MODIFIED);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
lfs_writeseg(fs, sp);
lfs_segunlock(fs);
lfs_segunlock_relock(fs);
goto top;
}
}
Introduce two fcntl calls that freeze the filesystem right at the point where segment 0 is being considered for writing. This allows for automated checkpoint vailidity scanning, and could be used (in conjunction with the existing LFCNREWIND) for e.g. snapshot dumps as well. Include a regression test that does such scanning. When writing the Ifile, loop through the dirty block list three times to make sure that the checkpoint is always consistent (the first and second times the Ifile blocks can cross a segment boundary; not so the third time unless the segments are very small). Discovered by using the aforementioned regression test.
2006-04-18 00:02:34 +04:00
/*
* If we begin a new segment in the middle of writing
* the Ifile, it creates an inconsistent checkpoint,
* since the Ifile information for the new segment
* is not up-to-date. Take care of this here by
* sending the Ifile through again in case there
* are newly dirtied blocks. But wait, there's more!
* This second Ifile write could *also* cross a segment
* boundary, if the first one was large. The second
* one is guaranteed to be no more than 8 blocks,
* though (two segment blocks and supporting indirects)
* so the third write *will not* cross the boundary.
*/
if (vp == fs->lfs_ivnode) {
lfs_writefile(fs, sp, vp);
lfs_writefile(fs, sp, vp);
}
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
#ifdef DEBUG
Use lockstatus instead of a homebrewed locking system to control LFCNWRAPSTOP and LFCNWRAPGO. Be less verbose about the various looping checks: use log() rather than printf(), and only log anything if we are really looping ("count = 2" is not an error condition). Allow dirops sleeping on available space to be interruptible.
2006-09-29 03:08:23 +04:00
if (++loopcount > 2)
log(LOG_NOTICE, "lfs_vflush: looping count=%d\n", loopcount);
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
} while (lfs_writeinode(fs, sp, ip));
} while (lfs_writeseg(fs, sp) && ip->i_number == LFS_IFILE_INUM);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (lfs_dostats) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.nwrites;
if (sp->seg_flags & SEGM_SYNC)
++lfs_stats.nsync_writes;
if (sp->seg_flags & SEGM_CKP)
++lfs_stats.ncheckpoints;
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/*
* If we were called from somewhere that has already held the seglock
* (e.g., lfs_markv()), the lfs_segunlock will not wait for
* the write to complete because we are still locked.
* Since lfs_vflush() must return the vnode with no dirty buffers,
* we must explicitly wait, if that is the case.
*
* We compare the iocount against 1, not 0, because it is
* artificially incremented by lfs_seglock().
*/
Add simple_lock protection for lfs_seglock and lfs_subsys_pages; these will be expanded to cover other per-fs and subsystem-wide data as well. Fix a case of IN_MODIFIED being set without updating lfs_uinodes, resulting in a "lfs_uinodes < 0" panic. Fix a deadlock in lfs_putpages arising from the need to busy all pages in a block; unbusy any that had already been busied before starting over.
2003-03-15 09:58:49 +03:00
simple_lock(&fs->lfs_interlock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if (fs->lfs_seglock > 1) {
while (fs->lfs_iocount > 1)
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
(void)ltsleep(&fs->lfs_iocount, PRIBIO + 1,
"lfs_vflush", 0, &fs->lfs_interlock);
}
simple_unlock(&fs->lfs_interlock);
Add simple_lock protection for lfs_seglock and lfs_subsys_pages; these will be expanded to cover other per-fs and subsystem-wide data as well. Fix a case of IN_MODIFIED being set without updating lfs_uinodes, resulting in a "lfs_uinodes < 0" panic. Fix a deadlock in lfs_putpages arising from the need to busy all pages in a block; unbusy any that had already been busied before starting over.
2003-03-15 09:58:49 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_segunlock(fs);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Make LFS work better (though still not "well") as an NFS-exported filesystem (and other things that needed to be fixed before the tests would complete), to wit: * Include the fs ident in the filehandle; improve stale filehandle checks. * Change definition of blksize() to use the on-dinode size instead of the inode's i_size, so that fsck_lfs will work properly again. * Use b_interlock in lfs_vtruncbuf. * Postpone dirop reclamation until after the seglock has been released, so that lfs_truncate is not called with the segment lock held. * Don't loop in lfs_fsync(), just write everything and wait. * Be more careful about the interlock/uobjlock in lfs_putpages: when we lose this lock, we have to resynchronize dirtiness of pages in each block. * Be sure to always write indirect blocks and update metadata in lfs_putpages; fixes a bug that caused blocks to be accounted to the wrong segment.
2003-04-23 11:20:37 +04:00
/* Wait for these buffers to be recovered by aiodoned */
s = splbio();
simple_lock(&global_v_numoutput_slock);
while (vp->v_numoutput > 0) {
set VBWAIT when waiting v_numoutput to be drained.
2003-12-17 13:38:39 +03:00
vp->v_flag |= VBWAIT;
Make LFS work better (though still not "well") as an NFS-exported filesystem (and other things that needed to be fixed before the tests would complete), to wit: * Include the fs ident in the filehandle; improve stale filehandle checks. * Change definition of blksize() to use the on-dinode size instead of the inode's i_size, so that fsck_lfs will work properly again. * Use b_interlock in lfs_vtruncbuf. * Postpone dirop reclamation until after the seglock has been released, so that lfs_truncate is not called with the segment lock held. * Don't loop in lfs_fsync(), just write everything and wait. * Be more careful about the interlock/uobjlock in lfs_putpages: when we lose this lock, we have to resynchronize dirtiness of pages in each block. * Be sure to always write indirect blocks and update metadata in lfs_putpages; fixes a bug that caused blocks to be accounted to the wrong segment.
2003-04-23 11:20:37 +04:00
ltsleep(&vp->v_numoutput, PRIBIO + 1, "lfs_vf2", 0,
&global_v_numoutput_slock);
}
simple_unlock(&global_v_numoutput_slock);
splx(s);
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when many inodes are cleaned at once. Make sure that we write all the pages on vnodes that are being flushed, even if we don't think there's room; drain v_numoutput before lfs_vflush() completes. Also, don't allow a vnode that is in the process of being cleaned to be chosen by getnewvnode(); this avoids a segment accounting panic in the case that a large number of inodes are fed to lfs_markv() all at once.
2006-05-13 03:36:11 +04:00
KASSERT(LIST_FIRST(&vp->v_dirtyblkhd) == NULL);
KASSERT(vp->v_numoutput == 0);
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
fs->lfs_flushvp = NULL;
KASSERT(fs->lfs_flushvp_fakevref == 0);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
return (0);
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writevnodes(struct lfs *fs, struct mount *mp, struct segment *sp, int op)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct inode *ip;
Replace the LIST structure mp->mnt_vnodelist to a TAILQ structure since all vnodes were synced and processed backwards. This meant that the last accessed node was processed first and the earlierst last. An extra benefit is the removal of the ugly hack from the Berkly days on LFS. In the proces, i've also replaced the various variations hand written loops by the TAILQ_FOREACH() macro's.
2006-10-20 22:58:12 +04:00
struct vnode *vp;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
int inodes_written = 0, only_cleaning;
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
int error = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Replace the LIST structure mp->mnt_vnodelist to a TAILQ structure since all vnodes were synced and processed backwards. This meant that the last accessed node was processed first and the earlierst last. An extra benefit is the removal of the ugly hack from the Berkly days on LFS. In the proces, i've also replaced the various variations hand written loops by the TAILQ_FOREACH() macro's.
2006-10-20 22:58:12 +04:00
loop:
Reverse the order of searching the vnode list in lfs_writevnodes(). This should speed up e.g. "chown -R" on LFS filesystems; e.g. it shows a 100% increase in the 'seq_stat' column of bonnie++.
2007-02-24 02:16:03 +03:00
/* start at last (newest) vnode. */
Replace the LIST structure mp->mnt_vnodelist to a TAILQ structure since all vnodes were synced and processed backwards. This meant that the last accessed node was processed first and the earlierst last. An extra benefit is the removal of the ugly hack from the Berkly days on LFS. In the proces, i've also replaced the various variations hand written loops by the TAILQ_FOREACH() macro's.
2006-10-20 22:58:12 +04:00
TAILQ_FOREACH_REVERSE(vp, &mp->mnt_vnodelist, vnodelst, v_mntvnodes) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If the vnode that we are about to sync is no longer
* associated with this mount point, start over.
*/
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
if (vp->v_mount != mp) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_VNODE, "lfs_writevnodes: starting over\n"));
lfs_writevnodes: in the case of "starting over", kick lfs_writeseg in order to avoid deadlock in check_dirty.
2003-03-20 17:17:21 +03:00
/*
* After this, pages might be busy
* due to our own previous putpages.
* Start actual segment write here to avoid deadlock.
*/
(void)lfs_writeseg(fs, sp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
goto loop;
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
in lfs_writefile, check v_type==VNON earlier. to avoid null dereference with DEBUG_LFS_VERBOSE.
2002-12-14 14:54:47 +03:00
if (vp->v_type == VNON) {
continue;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
ip = VTOI(vp);
if ((op == VN_DIROP && !(vp->v_flag & VDIROP)) ||
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
(op != VN_DIROP && op != VN_CLEAN &&
(vp->v_flag & VDIROP))) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
vndebug(vp,"dirop");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (op == VN_EMPTY && !VPISEMPTY(vp)) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
vndebug(vp,"empty");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (op == VN_CLEAN && ip->i_number != LFS_IFILE_INUM
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
&& vp != fs->lfs_flushvp
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
&& !(ip->i_flag & IN_CLEANING)) {
vndebug(vp,"cleaning");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
if (lfs_vref(vp)) {
vndebug(vp,"vref");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Add initialization to quell compiler warning (only on some platforms?)
1999-03-30 20:11:43 +04:00
only_cleaning = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
* Write the inode/file if dirty and it's not the IFILE.
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if ((ip->i_flag & IN_ALLMOD) || !VPISEMPTY(vp)) {
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
only_cleaning =
((ip->i_flag & IN_ALLMOD) == IN_CLEANING);
Make sysctl variable lfs_clean_vnhead do what it was supposed to do, namely, toggle whether vnodes loaded only for cleaning (as opposed to normal filesystem use) are freed to the *head* of the vnode free list, rather than the tail. This should avoid a possible cache flushing effect, if the cleaner cleans a segment containing a large number of live inodes.
1999-03-26 01:38:28 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
if (ip->i_number != LFS_IFILE_INUM) {
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
error = lfs_writefile(fs, sp, vp);
if (error) {
lfs_vunref(vp);
if (error == EAGAIN) {
/*
* This error from lfs_putpages
* indicates we need to drop
* the segment lock and start
* over after the cleaner has
* had a chance to run.
*/
Postpone the segment accounting changes coming from truncation until the inode that makes those changes valid is either written to disk by lfs_writeinode() or discarded by lfs_vfree(). A couple of locking fixes are also included as well.
2006-05-01 01:19:42 +04:00
lfs_writeinode(fs, sp, ip);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
lfs_writeseg(fs, sp);
Several minor bug fixes: * Correct (weak) segment lock assertions in lfs_fragextend and lfs_putpages. * Keep IN_MODIFIED set if we run out of avail in lfs_putpages. * Don't try to (re)write buffers on a VBLK vnode; fixes a panic I found while running with an LFS root. * Raise priority of LFCNSEGWAIT to PVFS; PUSER is way too low for something the pagedaemon is relying on.
2006-04-08 03:59:28 +04:00
if (!VPISEMPTY(vp) &&
!WRITEINPROG(vp) &&
!(ip->i_flag & IN_ALLMOD))
LFS_SET_UINO(ip, IN_MODIFIED);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
break;
}
error = 0; /* XXX not quite right */
continue;
}
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
if (!VPISEMPTY(vp)) {
if (WRITEINPROG(vp)) {
ivndebug(vp,"writevnodes/write2");
} else if (!(ip->i_flag & IN_ALLMOD)) {
LFS_SET_UINO(ip, IN_MODIFIED);
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
(void) lfs_writeinode(fs, sp, ip);
inodes_written++;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Fixes associated with filling an LFS: Change the space computation to appear to change the size of the *disk* rather than the *bytes used* when more segment summaries and inode blocks are written. Try to estimate the amount of space that these will take up when more files are written, so the disk size doesn't change too much. Regularize error returns from lfs_valloc, lfs_balloc, lfs_truncate: they now fail entirely, rather than succeeding half-way and leaving the fs in an inconsistent state. Rewrite lfs_truncate, mostly stealing from ffs_truncate. The old lfs_truncate had difficulty truncating a large file to a non-zero size (indirect blocks were not handled appropriately). Unmark VDIROP on fvp after ufs_remove, ufs_rmdir, so these can be reclaimed immediately: this vnode would not be written to disk again anyway if the removal succeeded, and if it failed, no directory operation occurred. ufs_makeinode and ufs_mkdir now remove IN_ADIROP on error.
2000-06-28 00:57:11 +04:00
if (lfs_clean_vnhead && only_cleaning)
Make sysctl variable lfs_clean_vnhead do what it was supposed to do, namely, toggle whether vnodes loaded only for cleaning (as opposed to normal filesystem use) are freed to the *head* of the vnode free list, rather than the tail. This should avoid a possible cache flushing effect, if the cleaner cleans a segment containing a large number of live inodes.
1999-03-26 01:38:28 +03:00
lfs_vunref_head(vp);
else
lfs_vunref(vp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
return error;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
/*
* Do a checkpoint.
*/
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_segwrite(struct mount *mp, int flags)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct buf *bp;
struct inode *ip;
struct lfs *fs;
struct segment *sp;
struct vnode *vp;
SEGUSE *segusep;
The checkpoint loop always used (multiples of) lfs_sepb as the number of segments to mark. However, this may be much more than lfs_nseg. Originally this wasn't a big problem, since only the structures in the diskblock were changed, but nowadays there's a mirror of the segflags in the in-core superblock. This problem caused the code to walk way past the end of that allocated area, causing memory corruption in other kernel structures. So, use lfs_nseg as the maximum, as it should be. While here, simplify the loop; it had become an obfuscated piece of code overtime.
2003-03-29 01:39:42 +03:00
int do_ckp, did_ckp, error, s;
unsigned n, segleft, maxseg, sn, i, curseg;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int writer_set = 0;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
int dirty;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
int redo;
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
int um_error;
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
int loopcount;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
fs = VFSTOUFS(mp)->um_lfs;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_MAYBE_SEGLOCK(fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
if (fs->lfs_ronly)
return EROFS;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
lfs_imtime(fs);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Allocate a segment structure and enough space to hold pointers to
* the maximum possible number of buffers which can be described in a
* single summary block.
*/
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
do_ckp = LFS_SHOULD_CHECKPOINT(fs, flags);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_seglock(fs, flags | (do_ckp ? SEGM_CKP : 0));
sp = fs->lfs_sp;
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
if (sp->seg_flags & (SEGM_CLEAN | SEGM_CKP))
do_ckp = 1;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
* If lfs_flushvp is non-NULL, we are called from lfs_vflush,
* in which case we have to flush *all* buffers off of this vnode.
Handle the case of a vnode flush while dirops are active correctly in lfs_segwrite. Also, make sure a flush is called in SET_DIROP before sleeping on its results. Addresses PR #8863.
1999-12-04 00:47:44 +03:00
* We don't care about other nodes, but write any non-dirop nodes
* anyway in anticipation of another getnewvnode().
*
* If we're cleaning we only write cleaning and ifile blocks, and
* no dirops, since otherwise we'd risk corruption in a crash.
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
*/
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (sp->seg_flags & SEGM_CLEAN)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
lfs_writevnodes(fs, mp, sp, VN_CLEAN);
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
else if (!(sp->seg_flags & SEGM_FORCE_CKP)) {
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
do {
um_error = lfs_writevnodes(fs, mp, sp, VN_REG);
Install a new sysctl, vfs.lfs.ignore_lazy_sync, which causes LFS to ignore the "smooth" syncer, as if vfs.sync.*delay = 0, but only for LFS. The default is "on", i.e., ignore lazy sync. Reduce the amount of polling/busy-waiting done by lfs_putpages(). To accomplish this, copied genfs_putpages() and modified it to indicate which page it was that caused it to return with EDEADLK. fsync()/fdatasync() should no longer ever fail with EAGAIN, and should not consume huge quantities of cpu. Also, try to make dirops less likely to be written as the result of a VOP_PUTPAGES(), while ensuring that they are written regularly.
2007-04-17 05:16:46 +04:00
if (do_ckp ||
(sp->seg_flags & SEGM_W_DIROPS && !fs->lfs_dirops)) {
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
if (!writer_set) {
lfs_writer_enter(fs, "lfs writer");
writer_set = 1;
}
error = lfs_writevnodes(fs, mp, sp, VN_DIROP);
if (um_error == 0)
um_error = error;
Don't ever partially write dirops, even if we need the cleaner to run. This increases the chances of the "no clean segments" panic slightly, but allows us to run the ckckp regression test successfully to completion.
2006-05-01 23:47:29 +04:00
/* In case writevnodes errored out */
lfs_flush_dirops(fs);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
((SEGSUM *)(sp->segsum))->ss_flags &= ~(SS_CONT);
Postpone the segment accounting changes coming from truncation until the inode that makes those changes valid is either written to disk by lfs_writeinode() or discarded by lfs_vfree(). A couple of locking fixes are also included as well.
2006-05-01 01:19:42 +04:00
lfs_finalize_fs_seguse(fs);
- add a new functions, lfs_writer_enter/leave, and use them instead of duplicated code fragments. - add an assertion.
2003-07-02 17:40:51 +04:00
}
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
if (do_ckp && um_error) {
lfs_segunlock_relock(fs);
sp = fs->lfs_sp;
}
} while (do_ckp && um_error != 0);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If we are doing a checkpoint, mark everything since the
* last checkpoint as no longer ACTIVE.
*/
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
if (do_ckp || fs->lfs_doifile) {
The checkpoint loop always used (multiples of) lfs_sepb as the number of segments to mark. However, this may be much more than lfs_nseg. Originally this wasn't a big problem, since only the structures in the diskblock were changed, but nowadays there's a mirror of the segflags in the in-core superblock. This problem caused the code to walk way past the end of that allocated area, causing memory corruption in other kernel structures. So, use lfs_nseg as the maximum, as it should be. While here, simplify the loop; it had become an obfuscated piece of code overtime.
2003-03-29 01:39:42 +03:00
segleft = fs->lfs_nseg;
curseg = 0;
for (n = 0; n < fs->lfs_segtabsz; n++) {
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
dirty = 0;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
if (bread(fs->lfs_ivnode,
The checkpoint loop always used (multiples of) lfs_sepb as the number of segments to mark. However, this may be much more than lfs_nseg. Originally this wasn't a big problem, since only the structures in the diskblock were changed, but nowadays there's a mirror of the segflags in the in-core superblock. This problem caused the code to walk way past the end of that allocated area, causing memory corruption in other kernel structures. So, use lfs_nseg as the maximum, as it should be. While here, simplify the loop; it had become an obfuscated piece of code overtime.
2003-03-29 01:39:42 +03:00
fs->lfs_cleansz + n, fs->lfs_bsize, NOCRED, &bp))
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
panic("lfs_segwrite: ifile read");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
segusep = (SEGUSE *)bp->b_data;
The checkpoint loop always used (multiples of) lfs_sepb as the number of segments to mark. However, this may be much more than lfs_nseg. Originally this wasn't a big problem, since only the structures in the diskblock were changed, but nowadays there's a mirror of the segflags in the in-core superblock. This problem caused the code to walk way past the end of that allocated area, causing memory corruption in other kernel structures. So, use lfs_nseg as the maximum, as it should be. While here, simplify the loop; it had become an obfuscated piece of code overtime.
2003-03-29 01:39:42 +03:00
maxseg = min(segleft, fs->lfs_sepb);
for (i = 0; i < maxseg; i++) {
sn = curseg + i;
when inactivating segments, compare segment numbers correctly.
2003-10-03 17:02:54 +04:00
if (sn != dtosn(fs, fs->lfs_curseg) &&
The checkpoint loop always used (multiples of) lfs_sepb as the number of segments to mark. However, this may be much more than lfs_nseg. Originally this wasn't a big problem, since only the structures in the diskblock were changed, but nowadays there's a mirror of the segflags in the in-core superblock. This problem caused the code to walk way past the end of that allocated area, causing memory corruption in other kernel structures. So, use lfs_nseg as the maximum, as it should be. While here, simplify the loop; it had become an obfuscated piece of code overtime.
2003-03-29 01:39:42 +03:00
segusep->su_flags & SEGUSE_ACTIVE) {
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
segusep->su_flags &= ~SEGUSE_ACTIVE;
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
--fs->lfs_nactive;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
++dirty;
}
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
fs->lfs_suflags[fs->lfs_activesb][sn] =
segusep->su_flags;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (fs->lfs_version > 1)
++segusep;
else
segusep = (SEGUSE *)
((SEGUSE_V1 *)segusep + 1);
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
if (dirty)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
error = LFS_BWRITE_LOG(bp); /* Ifile */
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
else
brelse(bp);
The checkpoint loop always used (multiples of) lfs_sepb as the number of segments to mark. However, this may be much more than lfs_nseg. Originally this wasn't a big problem, since only the structures in the diskblock were changed, but nowadays there's a mirror of the segflags in the in-core superblock. This problem caused the code to walk way past the end of that allocated area, causing memory corruption in other kernel structures. So, use lfs_nseg as the maximum, as it should be. While here, simplify the loop; it had become an obfuscated piece of code overtime.
2003-03-29 01:39:42 +03:00
segleft -= fs->lfs_sepb;
curseg += fs->lfs_sepb;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
LOCK_ASSERT(LFS_SEGLOCK_HELD(fs));
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
did_ckp = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (do_ckp || fs->lfs_doifile) {
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
vp = fs->lfs_ivnode;
vn_lock(vp, LK_EXCLUSIVE);
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
loopcount = 0;
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
do {
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#ifdef DEBUG
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
LFS_ENTER_LOG("pretend", __FILE__, __LINE__, 0, 0, curproc->p_pid);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#endif
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&fs->lfs_interlock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
fs->lfs_flags &= ~LFS_IFDIRTY;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
ip = VTOI(vp);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
if (LIST_FIRST(&vp->v_dirtyblkhd) != NULL) {
/*
* Ifile has no pages, so we don't need
* to check error return here.
*/
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
lfs_writefile(fs, sp, vp);
Introduce two fcntl calls that freeze the filesystem right at the point where segment 0 is being considered for writing. This allows for automated checkpoint vailidity scanning, and could be used (in conjunction with the existing LFCNREWIND) for e.g. snapshot dumps as well. Include a regression test that does such scanning. When writing the Ifile, loop through the dirty block list three times to make sure that the checkpoint is always consistent (the first and second times the Ifile blocks can cross a segment boundary; not so the third time unless the segments are very small). Discovered by using the aforementioned regression test.
2006-04-18 00:02:34 +04:00
/*
* Ensure the Ifile takes the current segment
* into account. See comment in lfs_vflush.
*/
lfs_writefile(fs, sp, vp);
lfs_writefile(fs, sp, vp);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
if (ip->i_flag & IN_ALLMOD)
++did_ckp;
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
#if 0
redo = (do_ckp ? lfs_writeinode(fs, sp, ip) : 0);
#else
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
redo = lfs_writeinode(fs, sp, ip);
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
#endif
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
redo += lfs_writeseg(fs, sp);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&fs->lfs_interlock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
redo += (fs->lfs_flags & LFS_IFDIRTY);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
#ifdef DEBUG
Use lockstatus instead of a homebrewed locking system to control LFCNWRAPSTOP and LFCNWRAPGO. Be less verbose about the various looping checks: use log() rather than printf(), and only log anything if we are really looping ("count = 2" is not an error condition). Allow dirops sleeping on available space to be interruptible.
2006-09-29 03:08:23 +04:00
if (++loopcount > 2)
log(LOG_NOTICE, "lfs_segwrite: looping count=%d\n",
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
loopcount);
#endif
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
} while (redo && do_ckp);
/*
* Unless we are unmounting, the Ifile may continue to have
* dirty blocks even after a checkpoint, due to changes to
* inodes' atime. If we're checkpointing, it's "impossible"
* for other parts of the Ifile to be dirty after the loop
* above, since we hold the segment lock.
*/
Use VONWORKLST as a heuristic for vnode emptiness, rather than exhaustively checking the memq. Take greater care not to dirty the Ifile vnode when unmounting the filesystem. This should fix a "(vp->v_flag & VONWORKLST) == 0" assertion panic in vgonel that could occur when unmounting. Do not allow the Ifile to be mapped for writing.
2003-03-21 09:16:53 +03:00
s = splbio();
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
if (LIST_EMPTY(&vp->v_dirtyblkhd)) {
LFS_CLR_UINO(ip, IN_ALLMOD);
}
#ifdef DIAGNOSTIC
else if (do_ckp) {
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
int do_panic = 0;
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
LIST_FOREACH(bp, &vp->v_dirtyblkhd, b_vnbufs) {
if (bp->b_lblkno < fs->lfs_cleansz +
fs->lfs_segtabsz &&
!(bp->b_flags & B_GATHERED)) {
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
printf("ifile lbn %ld still dirty (flags %lx)\n",
(long)bp->b_lblkno,
(long)bp->b_flags);
++do_panic;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
}
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
if (do_panic)
panic("dirty blocks");
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
#endif
Use VONWORKLST as a heuristic for vnode emptiness, rather than exhaustively checking the memq. Take greater care not to dirty the Ifile vnode when unmounting the filesystem. This should fix a "(vp->v_flag & VONWORKLST) == 0" assertion panic in vgonel that could occur when unmounting. Do not allow the Ifile to be mapped for writing.
2003-03-21 09:16:53 +03:00
splx(s);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
VOP_UNLOCK(vp, 0);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
} else {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
(void) lfs_writeseg(fs, sp);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
/* Note Ifile no longer needs to be written */
fs->lfs_doifile = 0;
- add a new functions, lfs_writer_enter/leave, and use them instead of duplicated code fragments. - add an assertion.
2003-07-02 17:40:51 +04:00
if (writer_set)
lfs_writer_leave(fs);
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
/*
* If we didn't write the Ifile, we didn't really do anything.
* That means that (1) there is a checkpoint on disk and (2)
* nothing has changed since it was written.
*
* Take the flags off of the segment so that lfs_segunlock
* doesn't have to write the superblock either.
*/
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
if (do_ckp && !did_ckp) {
sp->seg_flags &= ~SEGM_CKP;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
}
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (lfs_dostats) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.nwrites;
if (sp->seg_flags & SEGM_SYNC)
++lfs_stats.nsync_writes;
if (sp->seg_flags & SEGM_CKP)
++lfs_stats.ncheckpoints;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_segunlock(fs);
return (0);
}
/*
* Write the dirty blocks associated with a vnode.
*/
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writefile(struct lfs *fs, struct segment *sp, struct vnode *vp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct finfo *fip;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
struct inode *ip;
Fix a bug in which FINFOs were written with a version number of zero. Add assertions and add this to the DEBUG fip test in lfs_writeseg.
2006-05-20 05:10:18 +04:00
int i, frag;
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
int error;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
error = 0;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
ip = VTOI(vp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
fip = sp->fip;
Fix a bug in which FINFOs were written with a version number of zero. Add assertions and add this to the DEBUG fip test in lfs_writeseg.
2006-05-20 05:10:18 +04:00
lfs_acquire_finfo(fs, ip->i_number, ip->i_gen);
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
if (vp->v_flag & VDIROP)
((SEGSUM *)(sp->segsum))->ss_flags |= (SS_DIROP|SS_CONT);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if (sp->seg_flags & SEGM_CLEAN) {
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
lfs_gather(fs, sp, vp, lfs_match_fake);
/*
* For a file being flushed, we need to write *all* blocks.
* This means writing the cleaning blocks first, and then
* immediately following with any non-cleaning blocks.
* The same is true of the Ifile since checkpoints assume
* that all valid Ifile blocks are written.
*/
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when many inodes are cleaned at once. Make sure that we write all the pages on vnodes that are being flushed, even if we don't think there's room; drain v_numoutput before lfs_vflush() completes. Also, don't allow a vnode that is in the process of being cleaned to be chosen by getnewvnode(); this avoids a segment accounting panic in the case that a large number of inodes are fed to lfs_markv() all at once.
2006-05-13 03:36:11 +04:00
if (IS_FLUSHING(fs, vp) || vp == fs->lfs_ivnode) {
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
lfs_gather(fs, sp, vp, lfs_match_data);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/*
* Don't call VOP_PUTPAGES: if we're flushing,
* we've already done it, and the Ifile doesn't
* use the page cache.
*/
}
} else {
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
lfs_gather(fs, sp, vp, lfs_match_data);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/*
* If we're flushing, we've already called VOP_PUTPAGES
* so don't do it again. Otherwise, we want to write
* everything we've got.
*/
if (!IS_FLUSHING(fs, vp)) {
acquire v_interlock before calling VOP_PUTPAGES.
2003-02-19 15:02:38 +03:00
simple_lock(&vp->v_interlock);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
error = VOP_PUTPAGES(vp, 0, 0,
PGO_CLEANIT | PGO_ALLPAGES | PGO_LOCKED);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
}
}
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* It may not be necessary to write the meta-data blocks at this point,
* as the roll-forward recovery code should be able to reconstruct the
* list.
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
*
* We have to write them anyway, though, under two conditions: (1) the
* vnode is being flushed (for reuse by vinvalbuf); or (2) we are
* checkpointing.
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
*
* BUT if we are cleaning, we might have indirect blocks that refer to
* new blocks not being written yet, in addition to fragments being
* moved out of a cleaned segment. If that is the case, don't
* write the indirect blocks, or the finfo will have a small block
* in the middle of it!
* XXX in this case isn't the inode size wrong too?
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
frag = 0;
if (sp->seg_flags & SEGM_CLEAN) {
for (i = 0; i < NDADDR; i++)
if (ip->i_lfs_fragsize[i] > 0 &&
ip->i_lfs_fragsize[i] < fs->lfs_bsize)
++frag;
}
#ifdef DIAGNOSTIC
if (frag > 1)
panic("lfs_writefile: more than one fragment!");
#endif
if (IS_FLUSHING(fs, vp) ||
(frag == 0 && (lfs_writeindir || (sp->seg_flags & SEGM_CKP)))) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
lfs_gather(fs, sp, vp, lfs_match_indir);
lfs_gather(fs, sp, vp, lfs_match_dindir);
lfs_gather(fs, sp, vp, lfs_match_tindir);
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
fip = sp->fip;
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
lfs_release_finfo(fs);
Improvements to LFS's paging mechanism, to wit: * Acknowledge that sometimes there are more dirty pages to be written to disk than clean segments. When we reach the danger line, lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if it holds the segment lock, drops it and waits for the cleaner to make room before continuing. * Note and avoid a three-way deadlock in lfs_putpages (a writer holding a page busy blocks on the cleaner while the cleaner blocks on the segment lock while lfs_putpages blocks on the page).
2006-03-24 23:05:32 +03:00
return error;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
/*
* Update segment accounting to reflect this inode's change of address.
*/
static int
lfs_update_iaddr(struct lfs *fs, struct segment *sp, struct inode *ip, daddr_t ndaddr)
{
struct buf *bp;
daddr_t daddr;
IFILE *ifp;
SEGUSE *sup;
ino_t ino;
int redo_ifile, error;
u_int32_t sn;
redo_ifile = 0;
/*
* If updating the ifile, update the super-block. Update the disk
* address and access times for this inode in the ifile.
*/
ino = ip->i_number;
if (ino == LFS_IFILE_INUM) {
daddr = fs->lfs_idaddr;
fs->lfs_idaddr = dbtofsb(fs, ndaddr);
} else {
LFS_IENTRY(ifp, fs, ino, bp);
daddr = ifp->if_daddr;
ifp->if_daddr = dbtofsb(fs, ndaddr);
error = LFS_BWRITE_LOG(bp); /* Ifile */
}
/*
* If this is the Ifile and lfs_offset is set to the first block
* in the segment, dirty the new segment's accounting block
* (XXX should already be dirty?) and tell the caller to do it again.
*/
if (ip->i_number == LFS_IFILE_INUM) {
sn = dtosn(fs, fs->lfs_offset);
if (sntod(fs, sn) + btofsb(fs, fs->lfs_sumsize) ==
fs->lfs_offset) {
LFS_SEGENTRY(sup, fs, sn, bp);
KASSERT(bp->b_flags & B_DELWRI);
LFS_WRITESEGENTRY(sup, fs, sn, bp);
/* fs->lfs_flags |= LFS_IFDIRTY; */
redo_ifile |= 1;
}
}
/*
* The inode's last address should not be in the current partial
* segment, except under exceptional circumstances (lfs_writevnodes
* had to start over, and in the meantime more blocks were written
* to a vnode). Both inodes will be accounted to this segment
* in lfs_writeseg so we need to subtract the earlier version
* here anyway. The segment count can temporarily dip below
* zero here; keep track of how many duplicates we have in
* "dupino" so we don't panic below.
*/
if (daddr >= fs->lfs_lastpseg && daddr <= fs->lfs_offset) {
++sp->ndupino;
DLOG((DLOG_SEG, "lfs_writeinode: last inode addr in current pseg "
"(ino %d daddr 0x%llx) ndupino=%d\n", ino,
(long long)daddr, sp->ndupino));
}
/*
* Account the inode: it no longer belongs to its former segment,
* though it will not belong to the new segment until that segment
* is actually written.
*/
if (daddr != LFS_UNUSED_DADDR) {
u_int32_t oldsn = dtosn(fs, daddr);
#ifdef DIAGNOSTIC
int ndupino = (sp->seg_number == oldsn) ? sp->ndupino : 0;
#endif
LFS_SEGENTRY(sup, fs, oldsn, bp);
#ifdef DIAGNOSTIC
if (sup->su_nbytes +
sizeof (struct ufs1_dinode) * ndupino
< sizeof (struct ufs1_dinode)) {
printf("lfs_writeinode: negative bytes "
"(segment %" PRIu32 " short by %d, "
"oldsn=%" PRIu32 ", cursn=%" PRIu32
", daddr=%" PRId64 ", su_nbytes=%u, "
"ndupino=%d)\n",
dtosn(fs, daddr),
(int)sizeof (struct ufs1_dinode) *
(1 - sp->ndupino) - sup->su_nbytes,
oldsn, sp->seg_number, daddr,
(unsigned int)sup->su_nbytes,
sp->ndupino);
panic("lfs_writeinode: negative bytes");
sup->su_nbytes = sizeof (struct ufs1_dinode);
}
#endif
DLOG((DLOG_SU, "seg %d -= %d for ino %d inode\n",
dtosn(fs, daddr), sizeof (struct ufs1_dinode), ino));
sup->su_nbytes -= sizeof (struct ufs1_dinode);
redo_ifile |=
(ino == LFS_IFILE_INUM && !(bp->b_flags & B_GATHERED));
if (redo_ifile) {
simple_lock(&fs->lfs_interlock);
fs->lfs_flags |= LFS_IFDIRTY;
simple_unlock(&fs->lfs_interlock);
/* Don't double-account */
fs->lfs_idaddr = 0x0;
}
LFS_WRITESEGENTRY(sup, fs, oldsn, bp); /* Ifile */
}
return redo_ifile;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writeinode(struct lfs *fs, struct segment *sp, struct inode *ip)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
struct buf *bp;
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
struct ufs1_dinode *cdp;
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t daddr;
int32_t *daddrp; /* XXX ondisk32 */
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
int i, ndx;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int redo_ifile = 0;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
int gotblk = 0;
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
int count;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
update for IN_ACCESSED changes
2000-05-31 05:40:01 +04:00
if (!(ip->i_flag & IN_ALLMOD))
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (0);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
/* Can't write ifile when writer is not set */
KASSERT(ip->i_number != LFS_IFILE_INUM || fs->lfs_writer > 0 ||
(sp->seg_flags & SEGM_CLEAN));
/*
* If this is the Ifile, see if writing it here will generate a
* temporary misaccounting. If it will, do the accounting and write
* the blocks, postponing the inode write until the accounting is
* solid.
*/
count = 0;
while (ip->i_number == LFS_IFILE_INUM) {
int redo = 0;
if (sp->idp == NULL && sp->ibp == NULL &&
(sp->seg_bytes_left < fs->lfs_ibsize ||
sp->sum_bytes_left < sizeof(int32_t))) {
(void) lfs_writeseg(fs, sp);
continue;
}
/* Look for dirty Ifile blocks */
LIST_FOREACH(bp, &fs->lfs_ivnode->v_dirtyblkhd, b_vnbufs) {
if (!(bp->b_flags & B_GATHERED)) {
redo = 1;
break;
}
}
if (redo == 0)
redo = lfs_update_iaddr(fs, sp, ip, 0x0);
if (redo == 0)
break;
if (sp->idp) {
sp->idp->di_inumber = 0;
sp->idp = NULL;
}
++count;
Use lockstatus instead of a homebrewed locking system to control LFCNWRAPSTOP and LFCNWRAPGO. Be less verbose about the various looping checks: use log() rather than printf(), and only log anything if we are really looping ("count = 2" is not an error condition). Allow dirops sleeping on available space to be interruptible.
2006-09-29 03:08:23 +04:00
if (count > 2)
log(LOG_NOTICE, "lfs_writeinode: looping count=%d\n", count);
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
lfs_writefile(fs, sp, fs->lfs_ivnode);
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Allocate a new inode block if necessary. */
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
if ((ip->i_number != LFS_IFILE_INUM || sp->idp == NULL) &&
sp->ibp == NULL) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Allocate a new segment if necessary. */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (sp->seg_bytes_left < fs->lfs_ibsize ||
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
sp->sum_bytes_left < sizeof(int32_t))
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
(void) lfs_writeseg(fs, sp);
/* Get next inode block. */
daddr = fs->lfs_offset;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_offset += btofsb(fs, fs->lfs_ibsize);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->ibp = *sp->cbpp++ =
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
getblk(VTOI(fs->lfs_ivnode)->i_devvp,
fsbtodb(fs, daddr), fs->lfs_ibsize, 0, 0);
Fix "lfs_ifind: dinode xxx not found" panic. When inodes were freed, then immediately reloaded, their dinodes were located in an inode block which was not on disk at the advertized location, nor in the cache (although it would be flushed to disk next segment write). Fix this by using getblk() instead of lfs_newbuf() for inode blocks.
1999-04-12 04:04:21 +04:00
gotblk++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Zero out inode numbers */
for (i = 0; i < INOPB(fs); ++i)
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
((struct ufs1_dinode *)sp->ibp->b_data)[i].di_inumber =
0;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
++sp->start_bpp;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_avail -= btofsb(fs, fs->lfs_ibsize);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Set remaining space counters. */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_bytes_left -= fs->lfs_ibsize;
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
sp->sum_bytes_left -= sizeof(int32_t);
ndx = fs->lfs_sumsize / sizeof(int32_t) -
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
sp->ninodes / INOPB(fs) - 1;
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
((int32_t *)(sp->segsum))[ndx] = daddr;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
/* Check VDIROP in case there is a new file with no data blocks */
if (ITOV(ip)->v_flag & VDIROP)
((SEGSUM *)(sp->segsum))->ss_flags |= (SS_DIROP|SS_CONT);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Update the inode times and copy the inode onto the inode page. */
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/* XXX kludge --- don't redirty the ifile just to put times on it */
if (ip->i_number != LFS_IFILE_INUM)
Use nanotime() to update the time fields in filesystems. Convert the code from macros to real functions. Original patch and review from chuq. Note: ext2fs only keeps seconds in the on-disk inode, and msdosfs does not have enough precision for all fields, so this is not very useful for those two.
2005-09-12 20:24:41 +04:00
LFS_ITIMES(ip, NULL, NULL, NULL);
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
/*
* If this is the Ifile, and we've already written the Ifile in this
* partial segment, just overwrite it (it's not on disk yet) and
* continue.
*
* XXX we know that the bp that we get the second time around has
* already been gathered.
*/
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ip->i_number == LFS_IFILE_INUM && sp->idp) {
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
*(sp->idp) = *ip->i_din.ffs1_din;
ip->i_lfs_osize = ip->i_size;
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
return 0;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
bp = sp->ibp;
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
cdp = ((struct ufs1_dinode *)bp->b_data) + (sp->ninodes % INOPB(fs));
*cdp = *ip->i_din.ffs1_din;
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
/*
* If cleaning, link counts and directory file sizes cannot change,
* since those would be directory operations---even if the file
* we are writing is marked VDIROP we should write the old values.
* If we're not cleaning, of course, update the values so we get
* current values the next time we clean.
*/
if (sp->seg_flags & SEGM_CLEAN) {
if (ITOV(ip)->v_flag & VDIROP) {
cdp->di_nlink = ip->i_lfs_odnlink;
/* if (ITOV(ip)->v_type == VDIR) */
cdp->di_size = ip->i_lfs_osize;
}
} else {
ip->i_lfs_odnlink = cdp->di_nlink;
ip->i_lfs_osize = ip->i_size;
}
Postpone the segment accounting changes coming from truncation until the inode that makes those changes valid is either written to disk by lfs_writeinode() or discarded by lfs_vfree(). A couple of locking fixes are also included as well.
2006-05-01 01:19:42 +04:00
/* We can finish the segment accounting for truncations now */
lfs_finalize_ino_seguse(fs, ip);
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
/*
* If we are cleaning, ensure that we don't write UNWRITTEN disk
Keep track of the highest block held by an LFS inode, so that we can be assured that the last byte of a file is always allocated. Previously a file extension could cause the filesystem to be flushed, writing an inconsistent inode to disk. Although this condition would be corrected the next time blocks were written to disk, an intervening crash would leave the filesystem in an inconsistent state, leaving fsck_lfs to complain of an inode "partially truncated".
2005-04-14 04:02:46 +04:00
* addresses to disk; possibly change the on-disk record of
* the inode size, either by reverting to the previous size
* (in the case of cleaning) or by verifying the inode's block
* holdings (in the case of files being allocated as they are being
* written).
* XXX By not writing UNWRITTEN blocks, we are making the lfs_avail
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* XXX count on disk wrong by the same amount. We should be
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* XXX able to "borrow" from lfs_avail and return it after the
* XXX Ifile is written. See also in lfs_writeseg.
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
*/
Keep track of the highest block held by an LFS inode, so that we can be assured that the last byte of a file is always allocated. Previously a file extension could cause the filesystem to be flushed, writing an inconsistent inode to disk. Although this condition would be corrected the next time blocks were written to disk, an intervening crash would leave the filesystem in an inconsistent state, leaving fsck_lfs to complain of an inode "partially truncated".
2005-04-14 04:02:46 +04:00
/* Check file size based on highest allocated block */
if (((ip->i_ffs1_mode & IFMT) == IFREG ||
Check the to-be-on-disk consistency of directories as well (correct a typo in an earlier commit).
2005-04-19 03:03:08 +04:00
(ip->i_ffs1_mode & IFMT) == IFDIR) &&
Keep track of the highest block held by an LFS inode, so that we can be assured that the last byte of a file is always allocated. Previously a file extension could cause the filesystem to be flushed, writing an inconsistent inode to disk. Although this condition would be corrected the next time blocks were written to disk, an intervening crash would leave the filesystem in an inconsistent state, leaving fsck_lfs to complain of an inode "partially truncated".
2005-04-14 04:02:46 +04:00
ip->i_size > ((ip->i_lfs_hiblk + 1) << fs->lfs_bshift)) {
cdp->di_size = (ip->i_lfs_hiblk + 1) << fs->lfs_bshift;
DLOG((DLOG_SEG, "lfs_writeinode: ino %d size %" PRId64 " -> %"
PRId64 "\n", (int)ip->i_number, ip->i_size, cdp->di_size));
}
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
if (ip->i_lfs_effnblks != ip->i_ffs1_blocks) {
Keep track of the highest block held by an LFS inode, so that we can be assured that the last byte of a file is always allocated. Previously a file extension could cause the filesystem to be flushed, writing an inconsistent inode to disk. Although this condition would be corrected the next time blocks were written to disk, an intervening crash would leave the filesystem in an inconsistent state, leaving fsck_lfs to complain of an inode "partially truncated".
2005-04-14 04:02:46 +04:00
DLOG((DLOG_SEG, "lfs_writeinode: cleansing ino %d eff %d != nblk %d)"
" at %x\n", ip->i_number, ip->i_lfs_effnblks,
ip->i_ffs1_blocks, fs->lfs_offset));
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
for (daddrp = cdp->di_db; daddrp < cdp->di_ib + NIADDR;
daddrp++) {
if (*daddrp == UNWRITTEN) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "lfs_writeinode: wiping UNWRITTEN\n"));
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
*daddrp = 0;
}
}
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Keep track of the highest block held by an LFS inode, so that we can be assured that the last byte of a file is always allocated. Previously a file extension could cause the filesystem to be flushed, writing an inconsistent inode to disk. Although this condition would be corrected the next time blocks were written to disk, an intervening crash would leave the filesystem in an inconsistent state, leaving fsck_lfs to complain of an inode "partially truncated".
2005-04-14 04:02:46 +04:00
#ifdef DIAGNOSTIC
/*
* Check dinode held blocks against dinode size.
* This should be identical to the check in lfs_vget().
*/
for (i = (cdp->di_size + fs->lfs_bsize - 1) >> fs->lfs_bshift;
i < NDADDR; i++) {
KASSERT(i >= 0);
if ((cdp->di_mode & IFMT) == IFLNK)
continue;
if (((cdp->di_mode & IFMT) == IFBLK ||
(cdp->di_mode & IFMT) == IFCHR) && i == 0)
continue;
if (cdp->di_db[i] != 0) {
# ifdef DEBUG
lfs_dump_dinode(cdp);
# endif
panic("writing inconsistent inode");
}
}
#endif /* DIAGNOSTIC */
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ip->i_flag & IN_CLEANING)
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
LFS_CLR_UINO(ip, IN_CLEANING);
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
else {
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
/* XXX IN_ALLMOD */
LFS_CLR_UINO(ip, IN_ACCESSED | IN_ACCESS | IN_CHANGE |
Add a new flag, IN_MODIFY. This is like IN_UPDATE|IN_CHANGE, but unlike setting those flags, it does not cause the inode to be written in the periodic sync. This is used for writes to special files (devices and named pipes) and FIFOs. Do not preemptively sync updates to access times and modification times. They are now updated in the inode only opportunistically, or when the file or device is closed. (Really, it should be delayed beyond close, but this is enough to help substantially with device nodes.) And the most amusing part: Trickle sync was broken on both FFS and ext2fs, in different ways. In FFS, the periodic call to VFS_SYNC(MNT_LAZY) was still causing all file data to be synced. In ext2fs, it was causing the metadata to *not* be synced. We now only call VOP_UPDATE() on the node if we're doing MNT_LAZY. I've confirmed that we do in fact trickle correctly now.
2004-08-14 05:08:02 +04:00
IN_UPDATE | IN_MODIFY);
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
if (ip->i_lfs_effnblks == ip->i_ffs1_blocks)
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
LFS_CLR_UINO(ip, IN_MODIFIED);
fix empty if
2006-10-04 19:53:24 +04:00
else {
DLOG((DLOG_VNODE, "lfs_writeinode: ino %d: real "
"blks=%d, eff=%d\n", ip->i_number,
ip->i_ffs1_blocks, ip->i_lfs_effnblks));
}
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
}
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
if (ip->i_number == LFS_IFILE_INUM) {
/* We know sp->idp == NULL */
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
sp->idp = ((struct ufs1_dinode *)bp->b_data) +
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
(sp->ninodes % INOPB(fs));
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
/* Not dirty any more */
simple_lock(&fs->lfs_interlock);
fs->lfs_flags &= ~LFS_IFDIRTY;
simple_unlock(&fs->lfs_interlock);
}
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (gotblk) {
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
LFS_LOCK_BUF(bp);
Fix "lfs_ifind: dinode xxx not found" panic. When inodes were freed, then immediately reloaded, their dinodes were located in an inode block which was not on disk at the advertized location, nor in the cache (although it would be flushed to disk next segment write). Fix this by using getblk() instead of lfs_newbuf() for inode blocks.
1999-04-12 04:04:21 +04:00
brelse(bp);
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Increment inode count in segment summary block. */
++((SEGSUM *)(sp->segsum))->ss_ninos;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* If this page is full, set flag to allocate a new page. */
if (++sp->ninodes % INOPB(fs) == 0)
sp->ibp = NULL;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
redo_ifile = lfs_update_iaddr(fs, sp, ip, bp->b_blkno);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
KASSERT(redo_ifile == 0);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
return (redo_ifile);
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_gatherblock(struct segment *sp, struct buf *bp, int *sptr)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct lfs *fs;
- sprinkle const - avoid shadow variables.
2005-05-30 01:25:24 +04:00
int vers;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
int j, blksinblk;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(sp->fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If full, finish this segment. We may be doing I/O, so
* release and reacquire the splbio().
*/
#ifdef DIAGNOSTIC
if (sp->vp == NULL)
panic ("lfs_gatherblock: Null vp in segment");
#endif
fs = sp->fs;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
blksinblk = howmany(bp->b_bcount, fs->lfs_bsize);
if (sp->sum_bytes_left < sizeof(int32_t) * blksinblk ||
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
sp->seg_bytes_left < bp->b_bcount) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (sptr)
splx(*sptr);
lfs_updatemeta(sp);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
- sprinkle const - avoid shadow variables.
2005-05-30 01:25:24 +04:00
vers = sp->fip->fi_version;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
(void) lfs_writeseg(fs, sp);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Add the current file to the segment summary. */
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
lfs_acquire_finfo(fs, VTOI(sp->vp)->i_number, vers);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (sptr)
*sptr = splbio();
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (1);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (bp->b_flags & B_GATHERED) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "lfs_gatherblock: already gathered! Ino %d,"
" lbn %" PRId64 "\n",
sp->fip->fi_ino, bp->b_lblkno));
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (0);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Insert into the buffer list, update the FINFO block. */
bp->b_flags |= B_GATHERED;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*sp->cbpp++ = bp;
Keep per-inode, per-fs, and subsystem-wide counts of blocks allocated through lfs_balloc(), and use that to estimate the number of dirty pages belonging to LFS (subsystem or filesystem). This is almost certainly wrong for the case of a large mmap()ed region, but the accounting is tighter than what we had before, and performs much better in the typical case of pages dirtied through write().
2005-04-20 00:59:05 +04:00
for (j = 0; j < blksinblk; j++) {
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
sp->fip->fi_blocks[sp->fip->fi_nblocks++] = bp->b_lblkno + j;
Keep per-inode, per-fs, and subsystem-wide counts of blocks allocated through lfs_balloc(), and use that to estimate the number of dirty pages belonging to LFS (subsystem or filesystem). This is almost certainly wrong for the case of a large mmap()ed region, but the accounting is tighter than what we had before, and performs much better in the typical case of pages dirtied through write().
2005-04-20 00:59:05 +04:00
/* This block's accounting moves from lfs_favail to lfs_avail */
lfs_deregister_block(sp->vp, bp->b_lblkno + j);
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
sp->sum_bytes_left -= sizeof(int32_t) * blksinblk;
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
sp->seg_bytes_left -= bp->b_bcount;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (0);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
lfs_gather(struct lfs *fs, struct segment *sp, struct vnode *vp,
int (*match)(struct lfs *, struct buf *))
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
struct buf *bp, *nbp;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
int s, count = 0;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Several minor bug fixes: * Correct (weak) segment lock assertions in lfs_fragextend and lfs_putpages. * Keep IN_MODIFIED set if we run out of avail in lfs_putpages. * Don't try to (re)write buffers on a VBLK vnode; fixes a panic I found while running with an LFS root. * Raise priority of LFCNSEGWAIT to PVFS; PUSER is way too low for something the pagedaemon is relying on.
2006-04-08 03:59:28 +04:00
if (vp->v_type == VBLK)
return 0;
be more strict about sa->vp. (make sure the last lfs_updatemata in lfs_putpages takes effect.)
2003-10-18 19:52:42 +04:00
KASSERT(sp->vp == NULL);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->vp = vp;
s = splbio();
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#ifndef LFS_NO_BACKBUF_HACK
/* This is a hack to see if ordering the blocks in LFS makes a difference. */
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
# define BUF_OFFSET \
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
(((char *)&LIST_NEXT(bp, b_vnbufs)) - (char *)bp)
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
# define BACK_BUF(BP) \
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
((struct buf *)(((char *)(BP)->b_vnbufs.le_prev) - BUF_OFFSET))
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
# define BEG_OF_LIST \
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
((struct buf *)(((char *)&LIST_FIRST(&vp->v_dirtyblkhd)) - BUF_OFFSET))
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
loop:
/* Find last buffer. */
for (bp = LIST_FIRST(&vp->v_dirtyblkhd);
bp && LIST_NEXT(bp, b_vnbufs) != NULL;
bp = LIST_NEXT(bp, b_vnbufs))
/* nothing */;
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
for (; bp && bp != BEG_OF_LIST; bp = nbp) {
nbp = BACK_BUF(bp);
#else /* LFS_NO_BACKBUF_HACK */
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
loop:
for (bp = LIST_FIRST(&vp->v_dirtyblkhd); bp; bp = nbp) {
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
nbp = LIST_NEXT(bp, b_vnbufs);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#endif /* LFS_NO_BACKBUF_HACK */
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if ((bp->b_flags & (B_BUSY|B_GATHERED)) || !match(fs, bp)) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
#ifdef DEBUG
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
if (vp == fs->lfs_ivnode &&
(bp->b_flags & (B_BUSY|B_GATHERED)) == B_BUSY)
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
log(LOG_NOTICE, "lfs_gather: ifile lbn %"
PRId64 " busy (%x) at 0x%x",
bp->b_lblkno, bp->b_flags,
(unsigned)fs->lfs_offset);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#ifdef DIAGNOSTIC
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
# ifdef LFS_USE_B_INVAL
Several minor bug fixes: * Correct (weak) segment lock assertions in lfs_fragextend and lfs_putpages. * Keep IN_MODIFIED set if we run out of avail in lfs_putpages. * Don't try to (re)write buffers on a VBLK vnode; fixes a panic I found while running with an LFS root. * Raise priority of LFCNSEGWAIT to PVFS; PUSER is way too low for something the pagedaemon is relying on.
2006-04-08 03:59:28 +04:00
if ((bp->b_flags & (B_CALL|B_INVAL)) == B_INVAL) {
DLOG((DLOG_SEG, "lfs_gather: lbn %" PRId64
" is B_INVAL\n", bp->b_lblkno));
VOP_PRINT(bp->b_vp);
}
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
# endif /* LFS_USE_B_INVAL */
Several minor bug fixes: * Correct (weak) segment lock assertions in lfs_fragextend and lfs_putpages. * Keep IN_MODIFIED set if we run out of avail in lfs_putpages. * Don't try to (re)write buffers on a VBLK vnode; fixes a panic I found while running with an LFS root. * Raise priority of LFCNSEGWAIT to PVFS; PUSER is way too low for something the pagedaemon is relying on.
2006-04-08 03:59:28 +04:00
if (!(bp->b_flags & B_DELWRI))
panic("lfs_gather: bp not B_DELWRI");
if (!(bp->b_flags & B_LOCKED)) {
DLOG((DLOG_SEG, "lfs_gather: lbn %" PRId64
" blk %" PRId64 " not B_LOCKED\n",
bp->b_lblkno,
dbtofsb(fs, bp->b_blkno)));
VOP_PRINT(bp->b_vp);
panic("lfs_gather: bp not B_LOCKED");
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#endif
Several minor bug fixes: * Correct (weak) segment lock assertions in lfs_fragextend and lfs_putpages. * Keep IN_MODIFIED set if we run out of avail in lfs_putpages. * Don't try to (re)write buffers on a VBLK vnode; fixes a panic I found while running with an LFS root. * Raise priority of LFCNSEGWAIT to PVFS; PUSER is way too low for something the pagedaemon is relying on.
2006-04-08 03:59:28 +04:00
if (lfs_gatherblock(sp, bp, &s)) {
goto loop;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
count++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
splx(s);
lfs_updatemeta(sp);
be more strict about sa->vp. (make sure the last lfs_updatemata in lfs_putpages takes effect.)
2003-10-18 19:52:42 +04:00
KASSERT(sp->vp == vp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->vp = NULL;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
return count;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#if DEBUG
# define DEBUG_OOFF(n) do { \
if (ooff == 0) { \
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "lfs_updatemeta[%d]: warning: writing " \
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
"ino %d lbn %" PRId64 " at 0x%" PRIx32 \
", was 0x0 (or %" PRId64 ")\n", \
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
(n), ip->i_number, lbn, ndaddr, daddr)); \
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
} \
KNF (space after keywords).
2003-03-21 09:26:36 +03:00
} while (0)
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#else
# define DEBUG_OOFF(n)
#endif
/*
* Change the given block's address to ndaddr, finding its previous
* location using ufs_bmaparray().
*
* Account for this change in the segment table.
- tweak lfs_update_single()'s prototype so that it can be used by roll-forward code. - reduce code duplication using the above in update_meta() this also fixes fragment accounting.
2003-11-07 20:55:29 +03:00
*
* called with sp == NULL by roll-forwarding code.
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
*/
void
__unused removal on arguments; approved by core.
2006-11-16 04:32:37 +03:00
lfs_update_single(struct lfs *fs, struct segment *sp,
- sprinkle __unused on function decls. - fix a couple of unused bugs - no more -Wno-unused for i386
2006-10-12 05:30:41 +04:00
struct vnode *vp, daddr_t lbn, int32_t ndaddr, int size)
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
{
SEGUSE *sup;
struct buf *bp;
struct indir a[NIADDR + 2], *ap;
struct inode *ip;
daddr_t daddr, ooff;
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
int num, error;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
int bb, osize, obb;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
- tweak lfs_update_single()'s prototype so that it can be used by roll-forward code. - reduce code duplication using the above in update_meta() this also fixes fragment accounting.
2003-11-07 20:55:29 +03:00
KASSERT(sp == NULL || sp->vp == vp);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
ip = VTOI(vp);
make is_sequential a callback in order to achieve better lfs write clustering. since lfs always rewrite blocks into the new segment, current on-disk place of the block doesn't affect to write clustering. ok'ed by Konrad Schroder.
2003-05-18 16:59:05 +04:00
error = ufs_bmaparray(vp, lbn, &daddr, a, &num, NULL, NULL);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (error)
panic("lfs_updatemeta: ufs_bmaparray returned %d", error);
Add a sleeper count, to prevent the cleaner from panicing the kernel when the filesystem is unmounted, relocking the Ifile when its lock is draining. (We can't use vfs_busy() since the process is sleeping for a good long time.) Clean up / organize lfs.h, while I'm here. In lfs_update_single, assert that disk addresses are either negative, or are still positive when converted to int32_t, to prevent recurrence of a negative/positive block problem.
2003-03-28 11:03:38 +03:00
Various minor LFS improvements: * Note when lfs_putpages(9) thinks it is not going to be writing any pages before calling genfs_putpages(9). This prevents a situation in which blocks can be queued for writing without a segment header. * Correct computation of NRESERVE(), though it is still a gross overestimate in most cases. Note that if NRESERVE() is too high, it may be impossible to create files on the filesystem. We catch this case on filesystem mount and refuse to mount r/w. * Allow filesystems to be mounted whose block size is == MAXBSIZE. * Somewhere along the line, ufs_bmaparray(9) started mangling UNWRITTEN entries in indirect blocks again, triggering a failed assertion "daddr <= LFS_MAX_DADDR". Explicitly convert to and from int32_t to correct this. * Add a high-water mark for the number of dirty pages any given LFS can hold before triggering a flush. This is settable by sysctl, but off (zero) by default. * Be more careful about the MAX_BYTES and MAX_BUFS computations so we shouldn't see "please increase to at least zero" messages. * Note that VBLK and VCHR vnodes can have nonzero values in di_db[0] even though their v_size == 0. Don't panic when we see this. * Change lfs_bfree to a signed quantity. The manner in which it is processed before being passed to the cleaner means that sometimes it may drop below zero, and the cleaner must be aware of this. * Never report bfree < 0 (or higher than lfs_dsize) through lfs_statvfs(9). This prevents df(1) from ever telling us that our full filesystems have 16TB free. * Account space allocated through lfs_balloc(9) that does not have associated buffer headers, so that the pagedaemon doesn't run us out of segments. * Return ENOSPC from lfs_balloc(9) when bfree drops to zero. * Address a deadlock in lfs_bmapv/lfs_markv when the filesystem is being unmounted. Because vfs_busy() is a shared lock, and lfs_bmapv/lfs_markv mark the filesystem vfs_busy(), the cleaner can be holding the lock that umount() is blocking on, then try to vfs_busy() again in getnewvnode().
2005-02-26 08:40:42 +03:00
daddr = (daddr_t)((int32_t)daddr); /* XXX ondisk32 */
Add a sleeper count, to prevent the cleaner from panicing the kernel when the filesystem is unmounted, relocking the Ifile when its lock is draining. (We can't use vfs_busy() since the process is sleeping for a good long time.) Clean up / organize lfs.h, while I'm here. In lfs_update_single, assert that disk addresses are either negative, or are still positive when converted to int32_t, to prevent recurrence of a negative/positive block problem.
2003-03-28 11:03:38 +03:00
KASSERT(daddr <= LFS_MAX_DADDR);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (daddr > 0)
daddr = dbtofsb(fs, daddr);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
bb = fragstofsb(fs, numfrags(fs, size));
switch (num) {
case 0:
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
ooff = ip->i_ffs1_db[lbn];
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
DEBUG_OOFF(0);
if (ooff == UNWRITTEN)
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
ip->i_ffs1_blocks += bb;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
else {
/* possible fragment truncation or extension */
obb = btofsb(fs, ip->i_lfs_fragsize[lbn]);
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
ip->i_ffs1_blocks += (bb - obb);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
}
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
ip->i_ffs1_db[lbn] = ndaddr;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
break;
case 1:
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
ooff = ip->i_ffs1_ib[a[0].in_off];
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
DEBUG_OOFF(1);
if (ooff == UNWRITTEN)
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
ip->i_ffs1_blocks += bb;
ip->i_ffs1_ib[a[0].in_off] = ndaddr;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
break;
default:
ap = &a[num - 1];
if (bread(vp, ap->in_lbn, fs->lfs_bsize, NOCRED, &bp))
panic("lfs_updatemeta: bread bno %" PRId64,
ap->in_lbn);
/* XXX ondisk32 */
ooff = ((int32_t *)bp->b_data)[ap->in_off];
DEBUG_OOFF(num);
if (ooff == UNWRITTEN)
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
ip->i_ffs1_blocks += bb;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/* XXX ondisk32 */
((int32_t *)bp->b_data)[ap->in_off] = ndaddr;
(void) VOP_BWRITE(bp);
}
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
lfs_update_single: add an assertion.
2004-01-29 15:10:07 +03:00
KASSERT(ooff == 0 || ooff == UNWRITTEN || ooff == daddr);
Keep track of the highest block held by an LFS inode, so that we can be assured that the last byte of a file is always allocated. Previously a file extension could cause the filesystem to be flushed, writing an inconsistent inode to disk. Although this condition would be corrected the next time blocks were written to disk, an intervening crash would leave the filesystem in an inconsistent state, leaving fsck_lfs to complain of an inode "partially truncated".
2005-04-14 04:02:46 +04:00
/* Update hiblk when extending the file */
if (lbn > ip->i_lfs_hiblk)
ip->i_lfs_hiblk = lbn;
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
/*
* Though we'd rather it couldn't, this *can* happen right now
* if cleaning blocks and regular blocks coexist.
*/
/* KASSERT(daddr < fs->lfs_lastpseg || daddr > ndaddr); */
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/*
* Update segment usage information, based on old size
* and location.
*/
if (daddr > 0) {
u_int32_t oldsn = dtosn(fs, daddr);
#ifdef DIAGNOSTIC
- tweak lfs_update_single()'s prototype so that it can be used by roll-forward code. - reduce code duplication using the above in update_meta() this also fixes fragment accounting.
2003-11-07 20:55:29 +03:00
int ndupino;
if (sp && sp->seg_number == oldsn) {
ndupino = sp->ndupino;
} else {
ndupino = 0;
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#endif
remove impossible test
2006-09-02 10:46:04 +04:00
KASSERT(oldsn < fs->lfs_nseg);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (lbn >= 0 && lbn < NDADDR)
osize = ip->i_lfs_fragsize[lbn];
else
osize = fs->lfs_bsize;
LFS_SEGENTRY(sup, fs, oldsn, bp);
#ifdef DIAGNOSTIC
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
if (sup->su_nbytes + sizeof (struct ufs1_dinode) * ndupino
< osize) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
printf("lfs_updatemeta: negative bytes "
"(segment %" PRIu32 " short by %" PRId64
")\n", dtosn(fs, daddr),
(int64_t)osize -
- tweak lfs_update_single()'s prototype so that it can be used by roll-forward code. - reduce code duplication using the above in update_meta() this also fixes fragment accounting.
2003-11-07 20:55:29 +03:00
(sizeof (struct ufs1_dinode) * ndupino +
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
sup->su_nbytes));
64 bit inode changes.
2005-08-19 06:04:03 +04:00
printf("lfs_updatemeta: ino %llu, lbn %" PRId64
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
", addr = 0x%" PRIx64 "\n",
64 bit inode changes.
2005-08-19 06:04:03 +04:00
(unsigned long long)ip->i_number, lbn, daddr);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
printf("lfs_updatemeta: ndupino=%d\n", ndupino);
panic("lfs_updatemeta: negative bytes");
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
sup->su_nbytes = osize -
- tweak lfs_update_single()'s prototype so that it can be used by roll-forward code. - reduce code duplication using the above in update_meta() this also fixes fragment accounting.
2003-11-07 20:55:29 +03:00
sizeof (struct ufs1_dinode) * ndupino;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
}
#endif
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SU, "seg %" PRIu32 " -= %d for ino %d lbn %" PRId64
" db 0x%" PRIx64 "\n",
dtosn(fs, daddr), osize,
ip->i_number, lbn, daddr));
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
sup->su_nbytes -= osize;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
if (!(bp->b_flags & B_GATHERED)) {
simple_lock(&fs->lfs_interlock);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
fs->lfs_flags |= LFS_IFDIRTY;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
LFS_WRITESEGENTRY(sup, fs, oldsn, bp);
}
/*
* Now that this block has a new address, and its old
* segment no longer owns it, we can forget about its
* old size.
*/
if (lbn >= 0 && lbn < NDADDR)
ip->i_lfs_fragsize[lbn] = size;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Update the metadata that points to the blocks listed in the FINFO
* array.
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_updatemeta(struct segment *sp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
struct buf *sbp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
struct lfs *fs;
struct vnode *vp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
daddr_t lbn;
int i, nblocks, num;
int bb;
int bytesleft, size;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(sp->fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
vp = sp->vp;
nblocks = &sp->fip->fi_blocks[sp->fip->fi_nblocks] - sp->start_lbp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
KASSERT(nblocks >= 0);
be more strict about sa->vp. (make sure the last lfs_updatemata in lfs_putpages takes effect.)
2003-10-18 19:52:42 +04:00
KASSERT(vp != NULL);
if (nblocks == 0)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
return;
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
/*
* This count may be high due to oversize blocks from lfs_gop_write.
* Correct for this. (XXX we should be able to keep track of these.)
*/
fs = sp->fs;
for (i = 0; i < nblocks; i++) {
if (sp->start_bpp[i] == NULL) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "lfs_updatemeta: nblocks = %d, not %d\n", i, nblocks));
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
nblocks = i;
break;
}
num = howmany(sp->start_bpp[i]->b_bcount, fs->lfs_bsize);
add assertions and a debug check.
2003-04-01 18:58:43 +04:00
KASSERT(sp->start_bpp[i]->b_lblkno >= 0 || num == 1);
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
nblocks -= num - 1;
}
add assertions and a debug check.
2003-04-01 18:58:43 +04:00
KASSERT(vp->v_type == VREG ||
nblocks == &sp->fip->fi_blocks[sp->fip->fi_nblocks] - sp->start_lbp);
KASSERT(nblocks == sp->cbpp - sp->start_bpp);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* Sort the blocks.
*
* We have to sort even if the blocks come from the
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
* cleaner, because there might be other pending blocks on the
* same inode...and if we don't sort, and there are fragments
* present, blocks may be written in the wrong place.
*/
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
lfs_shellsort(sp->start_bpp, sp->start_lbp, nblocks, fs->lfs_bsize);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
/*
* Record the length of the last block in case it's a fragment.
* If there are indirect blocks present, they sort last. An
* indirect block will be lfs_bsize and its presence indicates
* that you cannot have fragments.
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
*
* XXX This last is a lie. A cleaned fragment can coexist with
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* XXX a later indirect block. This will continue to be
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
* XXX true until lfs_markv is fixed to do everything with
* XXX fake blocks (including fake inodes and fake indirect blocks).
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
sp->fip->fi_lastlength = ((sp->start_bpp[nblocks - 1]->b_bcount - 1) &
fs->lfs_bmask) + 1;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Assign disk addresses, and update references to the logical
* block and the segment usage information.
*/
for (i = nblocks; i--; ++sp->start_bpp) {
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
sbp = *sp->start_bpp;
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
lbn = *sp->start_lbp;
add assertions and a debug check.
2003-04-01 18:58:43 +04:00
KASSERT(sbp->b_lblkno == lbn);
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
sbp->b_blkno = fsbtodb(fs, fs->lfs_offset);
/*
* If we write a frag in the wrong place, the cleaner won't
* be able to correctly identify its size later, and the
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* segment will be uncleanable. (Even worse, it will assume
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
* that the indirect block that actually ends the list
* is of a smaller size!)
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if ((sbp->b_bcount & fs->lfs_bmask) && i != 0)
remove trailing \n in panic(). approved perry.
2002-09-27 19:35:29 +04:00
panic("lfs_updatemeta: fragment is not last block");
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
/*
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* For each subblock in this possibly oversized block,
* update its address on disk.
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
KASSERT(lbn >= 0 || sbp->b_bcount == fs->lfs_bsize);
be more strict about sa->vp. (make sure the last lfs_updatemata in lfs_putpages takes effect.)
2003-10-18 19:52:42 +04:00
KASSERT(vp == sbp->b_vp);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
for (bytesleft = sbp->b_bcount; bytesleft > 0;
bytesleft -= fs->lfs_bsize) {
size = MIN(bytesleft, fs->lfs_bsize);
bb = fragstofsb(fs, numfrags(fs, size));
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
lbn = *sp->start_lbp++;
- tweak lfs_update_single()'s prototype so that it can be used by roll-forward code. - reduce code duplication using the above in update_meta() this also fixes fragment accounting.
2003-11-07 20:55:29 +03:00
lfs_update_single(fs, sp, sp->vp, lbn, fs->lfs_offset,
size);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
fs->lfs_offset += bb;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Install a new sysctl, vfs.lfs.ignore_lazy_sync, which causes LFS to ignore the "smooth" syncer, as if vfs.sync.*delay = 0, but only for LFS. The default is "on", i.e., ignore lazy sync. Reduce the amount of polling/busy-waiting done by lfs_putpages(). To accomplish this, copied genfs_putpages() and modified it to indicate which page it was that caused it to return with EDEADLK. fsync()/fdatasync() should no longer ever fail with EAGAIN, and should not consume huge quantities of cpu. Also, try to make dirops less likely to be written as the result of a VOP_PUTPAGES(), while ensuring that they are written regularly.
2007-04-17 05:16:46 +04:00
/* This inode has been modified */
LFS_SET_UINO(VTOI(vp), IN_MODIFIED);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
/*
* Move lfs_offset to a segment earlier than sn.
*/
int
lfs_rewind(struct lfs *fs, int newsn)
{
int sn, osn, isdirty;
struct buf *bp;
SEGUSE *sup;
ASSERT_SEGLOCK(fs);
osn = dtosn(fs, fs->lfs_offset);
if (osn < newsn)
return 0;
/* lfs_avail eats the remaining space in this segment */
fs->lfs_avail -= fs->lfs_fsbpseg - (fs->lfs_offset - fs->lfs_curseg);
/* Find a low-numbered segment */
for (sn = 0; sn < fs->lfs_nseg; ++sn) {
LFS_SEGENTRY(sup, fs, sn, bp);
isdirty = sup->su_flags & SEGUSE_DIRTY;
brelse(bp);
if (!isdirty)
break;
}
if (sn == fs->lfs_nseg)
panic("lfs_rewind: no clean segments");
Change LFCNWRAP{STOP,GO} to make them more suitable for snapshotting; in particular, the caller can now choose whether to wait for the condition to be met, and if the caller of LFCNWRAPSTOP dies or otherwise closes the descriptor, the filesystem is started again. Updated the ckckp regression test to use the new semantics. dump_lfs(8) now uses the fcntls to implement LFS-style snapshotting through the -X flag, addressing PR#33457 albeit not using fss(4). Fixed a couple other problems with dump_lfs that manifested themselves during testing.
2006-06-24 09:28:54 +04:00
if (newsn >= 0 && sn >= newsn)
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
return ENOENT;
fs->lfs_nextseg = sn;
lfs_newseg(fs);
fs->lfs_offset = fs->lfs_curseg;
return 0;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
add comments and tweak code a little for readability. (no behaviour changes)
2003-10-17 18:20:12 +04:00
* Start a new partial segment.
*
* Return 1 when we entered to a new segment.
* Otherwise, return 0.
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_initseg(struct lfs *fs)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
add comments and tweak code a little for readability. (no behaviour changes)
2003-10-17 18:20:12 +04:00
struct segment *sp = fs->lfs_sp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
SEGSUM *ssp;
add comments and tweak code a little for readability. (no behaviour changes)
2003-10-17 18:20:12 +04:00
struct buf *sbp; /* buffer for SEGSUM */
int repeat = 0; /* return value */
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Advance to the next segment. */
if (!LFS_PARTIAL_FITS(fs)) {
add comments and tweak code a little for readability. (no behaviour changes)
2003-10-17 18:20:12 +04:00
SEGUSE *sup;
struct buf *bp;
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
/* lfs_avail eats the remaining space */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_avail -= fs->lfs_fsbpseg - (fs->lfs_offset -
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
fs->lfs_curseg);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Wake up any cleaning procs waiting on this file system. */
Don't wake up the cleaner if the filesystem is unwrappable, and fix the compatibility fcntls. Also includes one-line fixes for an MP locking bug and a zero-length FINFO problem that manifested during testing.
2006-06-29 23:28:21 +04:00
lfs_wakeup_cleaner(fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_newseg(fs);
repeat = 1;
fs->lfs_offset = fs->lfs_curseg;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_number = dtosn(fs, fs->lfs_curseg);
sp->seg_bytes_left = fsbtob(fs, fs->lfs_fsbpseg);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If the segment contains a superblock, update the offset
* and summary address to skip over it.
*/
LFS_SEGENTRY(sup, fs, sp->seg_number, bp);
if (sup->su_flags & SEGUSE_SUPERBLOCK) {
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_offset += btofsb(fs, LFS_SBPAD);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->seg_bytes_left -= LFS_SBPAD;
}
brelse(bp);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
/* Segment zero could also contain the labelpad */
if (fs->lfs_version > 1 && sp->seg_number == 0 &&
fs->lfs_start < btofsb(fs, LFS_LABELPAD)) {
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
fs->lfs_offset +=
btofsb(fs, LFS_LABELPAD) - fs->lfs_start;
sp->seg_bytes_left -=
LFS_LABELPAD - fsbtob(fs, fs->lfs_start);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
} else {
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_number = dtosn(fs, fs->lfs_curseg);
sp->seg_bytes_left = fsbtob(fs, fs->lfs_fsbpseg -
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
(fs->lfs_offset - fs->lfs_curseg));
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
fs->lfs_lastpseg = fs->lfs_offset;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
/* Record first address of this partial segment */
if (sp->seg_flags & SEGM_CLEAN) {
fs->lfs_cleanint[fs->lfs_cleanind] = fs->lfs_offset;
if (++fs->lfs_cleanind >= LFS_MAX_CLEANIND) {
/* "1" is the artificial inc in lfs_seglock */
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&fs->lfs_interlock);
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
while (fs->lfs_iocount > 1) {
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ltsleep(&fs->lfs_iocount, PRIBIO + 1,
"lfs_initseg", 0, &fs->lfs_interlock);
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
}
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
fs->lfs_cleanind = 0;
}
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->fs = fs;
sp->ibp = NULL;
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
sp->idp = NULL;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->ninodes = 0;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
sp->ndupino = 0;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->cbpp = sp->bpp;
add comments and tweak code a little for readability. (no behaviour changes)
2003-10-17 18:20:12 +04:00
/* Get a new buffer for SEGSUM */
remove a redundant substitution.
2003-12-17 10:14:03 +03:00
sbp = lfs_newbuf(fs, VTOI(fs->lfs_ivnode)->i_devvp,
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
fsbtodb(fs, fs->lfs_offset), fs->lfs_sumsize, LFS_NB_SUMMARY);
add comments and tweak code a little for readability. (no behaviour changes)
2003-10-17 18:20:12 +04:00
/* ... and enter it into the buffer list. */
*sp->cbpp = sbp;
sp->cbpp++;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_offset += btofsb(fs, fs->lfs_sumsize);
add comments and tweak code a little for readability. (no behaviour changes)
2003-10-17 18:20:12 +04:00
sp->start_bpp = sp->cbpp;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Set point to SEGSUM, initialize it. */
add comments and tweak code a little for readability. (no behaviour changes)
2003-10-17 18:20:12 +04:00
ssp = sp->segsum = sbp->b_data;
memset(ssp, 0, fs->lfs_sumsize);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
ssp->ss_next = fs->lfs_nextseg;
ssp->ss_nfinfo = ssp->ss_ninos = 0;
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
ssp->ss_magic = SS_MAGIC;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Set pointer to first FINFO, initialize it. */
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
sp->fip = (struct finfo *)((char *)sp->segsum + SEGSUM_SIZE(fs));
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->fip->fi_nblocks = 0;
sp->start_lbp = &sp->fip->fi_blocks[0];
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
sp->fip->fi_lastlength = 0;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_bytes_left -= fs->lfs_sumsize;
sp->sum_bytes_left = fs->lfs_sumsize - SEGSUM_SIZE(fs);
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (repeat);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
/*
* Remove SEGUSE_INVAL from all segments.
*/
void
lfs_unset_inval_all(struct lfs *fs)
{
SEGUSE *sup;
struct buf *bp;
int i;
for (i = 0; i < fs->lfs_nseg; i++) {
LFS_SEGENTRY(sup, fs, i, bp);
if (sup->su_flags & SEGUSE_INVAL) {
sup->su_flags &= ~SEGUSE_INVAL;
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
LFS_WRITESEGENTRY(sup, fs, i, bp);
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
} else
brelse(bp);
}
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Return the next segment to write.
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_newseg(struct lfs *fs)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
CLEANERINFO *cip;
SEGUSE *sup;
struct buf *bp;
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
int curseg, isdirty, sn, skip_inval;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Regression test improvements: Move the stop for LFCNWRAPSTOP to the point at which writing at segment 0 is really about to commence, since this is what the test expects (and incidentally what a snapshotting utility wants as well). More correctly reconstruct the on-disk state at every checkpoint, rather than relying on the entire state at the point of wrapping to be accurate (that is only true the first time we wrap). Add a "make abort" target to make rerunning the test more convenient when it has failed and we're done analyzing the failure.
2006-04-22 04:10:54 +04:00
/* Honor LFCNWRAPSTOP */
simple_lock(&fs->lfs_interlock);
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
while (fs->lfs_nextseg < fs->lfs_curseg && fs->lfs_nowrap) {
if (fs->lfs_wrappass) {
log(LOG_NOTICE, "%s: wrappass=%d\n",
fs->lfs_fsmnt, fs->lfs_wrappass);
fs->lfs_wrappass = 0;
break;
}
fs->lfs_wrapstatus = LFS_WRAP_WAITING;
Regression test improvements: Move the stop for LFCNWRAPSTOP to the point at which writing at segment 0 is really about to commence, since this is what the test expects (and incidentally what a snapshotting utility wants as well). More correctly reconstruct the on-disk state at every checkpoint, rather than relying on the entire state at the point of wrapping to be accurate (that is only true the first time we wrap). Add a "make abort" target to make rerunning the test more convenient when it has failed and we're done analyzing the failure.
2006-04-22 04:10:54 +04:00
wakeup(&fs->lfs_nowrap);
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
log(LOG_NOTICE, "%s: waiting at log wrap\n", fs->lfs_fsmnt);
ltsleep(&fs->lfs_wrappass, PVFS, "newseg", 10 * hz,
Regression test improvements: Move the stop for LFCNWRAPSTOP to the point at which writing at segment 0 is really about to commence, since this is what the test expects (and incidentally what a snapshotting utility wants as well). More correctly reconstruct the on-disk state at every checkpoint, rather than relying on the entire state at the point of wrapping to be accurate (that is only true the first time we wrap). Add a "make abort" target to make rerunning the test more convenient when it has failed and we're done analyzing the failure.
2006-04-22 04:10:54 +04:00
&fs->lfs_interlock);
}
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
fs->lfs_wrapstatus = LFS_WRAP_GOING;
Regression test improvements: Move the stop for LFCNWRAPSTOP to the point at which writing at segment 0 is really about to commence, since this is what the test expects (and incidentally what a snapshotting utility wants as well). More correctly reconstruct the on-disk state at every checkpoint, rather than relying on the entire state at the point of wrapping to be accurate (that is only true the first time we wrap). Add a "make abort" target to make rerunning the test more convenient when it has failed and we're done analyzing the failure.
2006-04-22 04:10:54 +04:00
simple_unlock(&fs->lfs_interlock);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
LFS_SEGENTRY(sup, fs, dtosn(fs, fs->lfs_nextseg), bp);
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SU, "lfs_newseg: seg %d := 0 in newseg\n",
dtosn(fs, fs->lfs_nextseg)));
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
sup->su_flags |= SEGUSE_DIRTY | SEGUSE_ACTIVE;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sup->su_nbytes = 0;
sup->su_nsums = 0;
sup->su_ninos = 0;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
LFS_WRITESEGENTRY(sup, fs, dtosn(fs, fs->lfs_nextseg), bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
LFS_CLEANERINFO(cip, fs, bp);
--cip->clean;
++cip->dirty;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
fs->lfs_nclean = cip->clean;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
LFS_SYNC_CLEANERINFO(cip, fs, bp, 1);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
fs->lfs_lastseg = fs->lfs_curseg;
fs->lfs_curseg = fs->lfs_nextseg;
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
skip_inval = 1;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
for (sn = curseg = dtosn(fs, fs->lfs_curseg) + fs->lfs_interleave;;) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sn = (sn + 1) % fs->lfs_nseg;
Introduce two fcntl calls that freeze the filesystem right at the point where segment 0 is being considered for writing. This allows for automated checkpoint vailidity scanning, and could be used (in conjunction with the existing LFCNREWIND) for e.g. snapshot dumps as well. Include a regression test that does such scanning. When writing the Ifile, loop through the dirty block list three times to make sure that the checkpoint is always consistent (the first and second times the Ifile blocks can cross a segment boundary; not so the third time unless the segments are very small). Discovered by using the aforementioned regression test.
2006-04-18 00:02:34 +04:00
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
if (sn == curseg) {
if (skip_inval)
skip_inval = 0;
else
panic("lfs_nextseg: no clean segments");
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
LFS_SEGENTRY(sup, fs, sn, bp);
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
isdirty = sup->su_flags & (SEGUSE_DIRTY | (skip_inval ? SEGUSE_INVAL : 0));
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/* Check SEGUSE_EMPTY as we go along */
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
if (isdirty && sup->su_nbytes == 0 &&
!(sup->su_flags & SEGUSE_EMPTY))
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
LFS_WRITESEGENTRY(sup, fs, sn, bp);
else
brelse(bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (!isdirty)
break;
}
Provide a resize_lfs(8), including kernel and cleaner support. The current implementation requires the fs to be mounted while resizing. Tested in both directions, and everything appears to work happily, but ymmv.
2005-04-23 23:47:51 +04:00
if (skip_inval == 0)
lfs_unset_inval_all(fs);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
++fs->lfs_nactive;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_nextseg = sntod(fs, sn);
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (lfs_dostats) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.segsused;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
static struct buf *
- sprinkle __unused on function decls. - fix a couple of unused bugs - no more -Wno-unused for i386
2006-10-12 05:30:41 +04:00
lfs_newclusterbuf(struct lfs *fs, struct vnode *vp, daddr_t addr,
__unused removal on arguments; approved by core.
2006-11-16 04:32:37 +03:00
int n)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
{
struct lfs_cluster *cl;
struct buf **bpp, *bp;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
cl = (struct lfs_cluster *)pool_get(&fs->lfs_clpool, PR_WAITOK);
bpp = (struct buf **)pool_get(&fs->lfs_bpppool, PR_WAITOK);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
memset(cl, 0, sizeof(*cl));
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
cl->fs = fs;
cl->bpp = bpp;
cl->bufcount = 0;
cl->bufsize = 0;
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
/* If this segment is being written synchronously, note that */
if (fs->lfs_sp->seg_flags & SEGM_SYNC) {
cl->flags |= LFS_CL_SYNC;
cl->seg = fs->lfs_sp;
++cl->seg->seg_iocount;
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/* Get an empty buffer header, or maybe one with something on it */
- add simple functions to allocate/free a buffer for i/o. - make bufpool static.
2006-01-04 13:13:05 +03:00
bp = getiobuf();
using normal bufcache buffer for cluster buffer head.
2003-07-30 17:36:40 +04:00
bp->b_flags = B_BUSY | B_CALL;
bp->b_dev = NODEV;
bp->b_blkno = bp->b_lblkno = addr;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
bp->b_iodone = lfs_cluster_callback;
use b_private rather than b_saveaddr. XXX LFS_USE_B_INVAL
2003-12-04 17:57:47 +03:00
bp->b_private = cl;
using normal bufcache buffer for cluster buffer head.
2003-07-30 17:36:40 +04:00
bp->b_vp = vp;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
return bp;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writeseg(struct lfs *fs, struct segment *sp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
struct buf **bpp, *bp, *cbp, *newbp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
SEGUSE *sup;
SEGSUM *ssp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
int i, s;
int do_again, nblocks, byteoffset;
lfs_writeseg(): make el_size a size_t (cosmetic only, no functional change)
2001-07-27 00:20:15 +04:00
size_t el_size;
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
struct lfs_cluster *cl;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
u_short ninos;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
struct vnode *devvp;
Fix uninitialized variable warnings.
2003-10-25 22:26:46 +04:00
char *p = NULL;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
struct vnode *vp;
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
int32_t *daddrp; /* XXX ondisk32 */
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
int changed;
calculate data checksum inline.
2004-03-09 10:43:49 +03:00
u_int32_t sum;
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
#ifdef DEBUG
FINFO *fip;
int findex;
#endif
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Note partial segments that are written by the cleaner, to help out the roll-forward agent.
2006-07-21 03:16:50 +04:00
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
ssp = (SEGSUM *)sp->segsum;
Note partial segments that are written by the cleaner, to help out the roll-forward agent.
2006-07-21 03:16:50 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
* If there are no buffers other than the segment summary to write,
* don't do anything. If we are the end of a dirop sequence, however,
* write the empty segment summary anyway, to help out the
* roll-forward agent.
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
if ((nblocks = sp->cbpp - sp->bpp) == 1) {
if ((ssp->ss_flags & (SS_DIROP | SS_CONT)) != SS_DIROP)
return 0;
}
/* Note if partial segment is being written by the cleaner */
if (sp->seg_flags & SEGM_CLEAN)
ssp->ss_flags |= SS_CLEAN;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
devvp = VTOI(fs->lfs_ivnode)->i_devvp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Update the segment usage information. */
LFS_SEGENTRY(sup, fs, sp->seg_number, bp);
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
/* Loop through all blocks, except the segment summary. */
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
for (bpp = sp->bpp; ++bpp < sp->cbpp; ) {
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if ((*bpp)->b_vp != devvp) {
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
sup->su_nbytes += (*bpp)->b_bcount;
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SU, "seg %" PRIu32 " += %ld for ino %d"
" lbn %" PRId64 " db 0x%" PRIx64 "\n",
sp->seg_number, (*bpp)->b_bcount,
VTOI((*bpp)->b_vp)->i_number, (*bpp)->b_lblkno,
(*bpp)->b_blkno));
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
}
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
#ifdef DEBUG
Fix a bug in which FINFOs were written with a version number of zero. Add assertions and add this to the DEBUG fip test in lfs_writeseg.
2006-05-20 05:10:18 +04:00
/* Check for zero-length and zero-version FINFO entries. */
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
fip = (struct finfo *)((char *)ssp + SEGSUM_SIZE(fs));
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
for (findex = 0; findex < ssp->ss_nfinfo; findex++) {
KDASSERT(fip->fi_nblocks > 0);
Fix a bug in which FINFOs were written with a version number of zero. Add assertions and add this to the DEBUG fip test in lfs_writeseg.
2006-05-20 05:10:18 +04:00
KDASSERT(fip->fi_version > 0);
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
fip = (FINFO *)((char *)fip + FINFOSIZE +
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
sizeof(int32_t) * fip->fi_nblocks);
}
#endif /* DEBUG */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
ninos = (ssp->ss_ninos + INOPB(fs) - 1) / INOPB(fs);
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SU, "seg %d += %d for %d inodes\n",
sp->seg_number, ssp->ss_ninos * sizeof (struct ufs1_dinode),
ssp->ss_ninos));
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
sup->su_nbytes += ssp->ss_ninos * sizeof (struct ufs1_dinode);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
/* sup->su_nbytes += fs->lfs_sumsize; */
if (fs->lfs_version == 1)
merge FreeBSD timecounters from branch simonb-timecounters - struct timeval time is gone time.tv_sec -> time_second - struct timeval mono_time is gone mono_time.tv_sec -> time_uptime - access to time via {get,}{micro,nano,bin}time() get* versions are fast but less precise - support NTP nanokernel implementation (NTP API 4) - further reading: Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
2006-06-08 02:33:33 +04:00
sup->su_olastmod = time_second;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
else
merge FreeBSD timecounters from branch simonb-timecounters - struct timeval time is gone time.tv_sec -> time_second - struct timeval mono_time is gone mono_time.tv_sec -> time_uptime - access to time via {get,}{micro,nano,bin}time() get* versions are fast but less precise - support NTP nanokernel implementation (NTP API 4) - further reading: Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
2006-06-08 02:33:33 +04:00
sup->su_lastmod = time_second;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sup->su_ninos += ninos;
++sup->su_nsums;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_avail -= btofsb(fs, fs->lfs_sumsize);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
do_again = !(bp->b_flags & B_GATHERED);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
LFS_WRITESEGENTRY(sup, fs, sp->seg_number, bp); /* Ifile */
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
/*
* Mark blocks B_BUSY, to prevent then from being changed between
* the checksum computation and the actual write.
*
* If we are cleaning, check indirect blocks for UNWRITTEN, and if
* there are any, replace them with copies that have UNASSIGNED
* instead.
*/
for (bpp = sp->bpp, i = nblocks - 1; i--;) {
++bpp;
bp = *bpp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (bp->b_flags & B_CALL) { /* UBC or malloced buffer */
bp->b_flags |= B_BUSY;
continue;
}
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&bp->b_interlock);
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
s = splbio();
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
while (bp->b_flags & B_BUSY) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "lfs_writeseg: avoiding potential"
" data summary corruption for ino %d, lbn %"
PRId64 "\n",
VTOI(bp->b_vp)->i_number, bp->b_lblkno));
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
bp->b_flags |= B_WANTED;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ltsleep(bp, (PRIBIO + 1), "lfs_writeseg", 0,
&bp->b_interlock);
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
splx(s);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
s = splbio();
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
}
bp->b_flags |= B_BUSY;
splx(s);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&bp->b_interlock);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/*
* Check and replace indirect block UNWRITTEN bogosity.
* XXX See comment in lfs_writefile.
*/
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (bp->b_lblkno < 0 && bp->b_vp != devvp && bp->b_vp &&
Add support for UFS2. UFS2 is an enhanced FFS, adding support for 64 bit block pointers, extended attribute storage, and a few other things. This commit does not yet include the code to manipulate the extended storage (for e.g. ACLs), this will be done later. Originally written by Kirk McKusick and Network Associates Laboratories for FreeBSD.
2003-04-02 14:39:19 +04:00
VTOI(bp->b_vp)->i_ffs1_blocks !=
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
VTOI(bp->b_vp)->i_lfs_effnblks) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_VNODE, "lfs_writeseg: cleansing ino %d (%d != %d)\n",
VTOI(bp->b_vp)->i_number,
VTOI(bp->b_vp)->i_lfs_effnblks,
VTOI(bp->b_vp)->i_ffs1_blocks));
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
/* Make a copy we'll make changes to */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
newbp = lfs_newbuf(fs, bp->b_vp, bp->b_lblkno,
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
bp->b_bcount, LFS_NB_IBLOCK);
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
newbp->b_blkno = bp->b_blkno;
memcpy(newbp->b_data, bp->b_data,
newbp->b_bcount);
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
changed = 0;
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
/* XXX ondisk32 */
for (daddrp = (int32_t *)(newbp->b_data);
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
daddrp < (int32_t *)((char *)newbp->b_data +
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
newbp->b_bcount); daddrp++) {
if (*daddrp == UNWRITTEN) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
++changed;
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
*daddrp = 0;
}
}
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
/*
* Get rid of the old buffer. Don't mark it clean,
* though, if it still has dirty data on it.
*/
if (changed) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "lfs_writeseg: replacing UNWRITTEN(%d):"
" bp = %p newbp = %p\n", changed, bp,
newbp));
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
*bpp = newbp;
Make LFS work better (though still not "well") as an NFS-exported filesystem (and other things that needed to be fixed before the tests would complete), to wit: * Include the fs ident in the filehandle; improve stale filehandle checks. * Change definition of blksize() to use the on-dinode size instead of the inode's i_size, so that fsck_lfs will work properly again. * Use b_interlock in lfs_vtruncbuf. * Postpone dirop reclamation until after the seglock has been released, so that lfs_truncate is not called with the segment lock held. * Don't loop in lfs_fsync(), just write everything and wait. * Be more careful about the interlock/uobjlock in lfs_putpages: when we lose this lock, we have to resynchronize dirtiness of pages in each block. * Be sure to always write indirect blocks and update metadata in lfs_putpages; fixes a bug that caused blocks to be accounted to the wrong segment.
2003-04-23 11:20:37 +04:00
bp->b_flags &= ~(B_ERROR | B_GATHERED);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (bp->b_flags & B_CALL) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "lfs_writeseg: "
"indir bp should not be B_CALL\n"));
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
s = splbio();
biodone(bp);
splx(s);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
bp = NULL;
} else {
Fix a buffer-cache corrupting bug in lfs_writeseg, where brelse could be improperly used on an already-queued buffer.
2000-09-09 08:13:43 +04:00
/* Still on free list, leave it there */
s = splbio();
bp->b_flags &= ~B_BUSY;
if (bp->b_flags & B_WANTED)
wakeup(bp);
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
splx(s);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
/*
* We have to re-decrement lfs_avail
* since this block is going to come
* back around to us in the next
* segment.
*/
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
fs->lfs_avail -=
btofsb(fs, bp->b_bcount);
Fix a buffer-cache corrupting bug in lfs_writeseg, where brelse could be improperly used on an already-queued buffer.
2000-09-09 08:13:43 +04:00
}
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
} else {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_freebuf(fs, newbp);
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
}
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
}
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Compute checksum across data and then across summary; the first
* block (the summary block) is skipped. Set the create time here
* so that it's guaranteed to be later than the inode mod times.
*/
calculate data checksum inline.
2004-03-09 10:43:49 +03:00
sum = 0;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (fs->lfs_version == 1)
el_size = sizeof(u_long);
else
el_size = sizeof(u_int32_t);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
for (bpp = sp->bpp, i = nblocks - 1; i--; ) {
++bpp;
/* Loop through gop_write cluster blocks */
for (byteoffset = 0; byteoffset < (*bpp)->b_bcount;
byteoffset += fs->lfs_bsize) {
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
#ifdef LFS_USE_B_INVAL
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (((*bpp)->b_flags & (B_CALL | B_INVAL)) ==
(B_CALL | B_INVAL)) {
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
if (copyin((void *)(*bpp)->b_saveaddr +
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
byteoffset, dp, el_size)) {
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
panic("lfs_writeseg: copyin failed [1]:"
" ino %d blk %" PRId64,
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
VTOI((*bpp)->b_vp)->i_number,
(*bpp)->b_lblkno);
}
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
} else
#endif /* LFS_USE_B_INVAL */
{
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
sum = lfs_cksum_part((char *)
calculate data checksum inline.
2004-03-09 10:43:49 +03:00
(*bpp)->b_data + byteoffset, el_size, sum);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
}
}
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
}
if (fs->lfs_version == 1)
merge FreeBSD timecounters from branch simonb-timecounters - struct timeval time is gone time.tv_sec -> time_second - struct timeval mono_time is gone mono_time.tv_sec -> time_uptime - access to time via {get,}{micro,nano,bin}time() get* versions are fast but less precise - support NTP nanokernel implementation (NTP API 4) - further reading: Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
2006-06-08 02:33:33 +04:00
ssp->ss_ocreate = time_second;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
else {
merge FreeBSD timecounters from branch simonb-timecounters - struct timeval time is gone time.tv_sec -> time_second - struct timeval mono_time is gone mono_time.tv_sec -> time_uptime - access to time via {get,}{micro,nano,bin}time() get* versions are fast but less precise - support NTP nanokernel implementation (NTP API 4) - further reading: Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
2006-06-08 02:33:33 +04:00
ssp->ss_create = time_second;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
ssp->ss_serial = ++fs->lfs_serial;
ssp->ss_ident = fs->lfs_ident;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
calculate data checksum inline.
2004-03-09 10:43:49 +03:00
ssp->ss_datasum = lfs_cksum_fold(sum);
ssp->ss_sumsum = cksum(&ssp->ss_datasum,
fs->lfs_sumsize - sizeof(ssp->ss_sumsum));
Various minor LFS improvements: * Note when lfs_putpages(9) thinks it is not going to be writing any pages before calling genfs_putpages(9). This prevents a situation in which blocks can be queued for writing without a segment header. * Correct computation of NRESERVE(), though it is still a gross overestimate in most cases. Note that if NRESERVE() is too high, it may be impossible to create files on the filesystem. We catch this case on filesystem mount and refuse to mount r/w. * Allow filesystems to be mounted whose block size is == MAXBSIZE. * Somewhere along the line, ufs_bmaparray(9) started mangling UNWRITTEN entries in indirect blocks again, triggering a failed assertion "daddr <= LFS_MAX_DADDR". Explicitly convert to and from int32_t to correct this. * Add a high-water mark for the number of dirty pages any given LFS can hold before triggering a flush. This is settable by sysctl, but off (zero) by default. * Be more careful about the MAX_BYTES and MAX_BUFS computations so we shouldn't see "please increase to at least zero" messages. * Note that VBLK and VCHR vnodes can have nonzero values in di_db[0] even though their v_size == 0. Don't panic when we see this. * Change lfs_bfree to a signed quantity. The manner in which it is processed before being passed to the cleaner means that sometimes it may drop below zero, and the cleaner must be aware of this. * Never report bfree < 0 (or higher than lfs_dsize) through lfs_statvfs(9). This prevents df(1) from ever telling us that our full filesystems have 16TB free. * Account space allocated through lfs_balloc(9) that does not have associated buffer headers, so that the pagedaemon doesn't run us out of segments. * Return ENOSPC from lfs_balloc(9) when bfree drops to zero. * Address a deadlock in lfs_bmapv/lfs_markv when the filesystem is being unmounted. Because vfs_busy() is a shared lock, and lfs_bmapv/lfs_markv mark the filesystem vfs_busy(), the cleaner can be holding the lock that umount() is blocking on, then try to vfs_busy() again in getnewvnode().
2005-02-26 08:40:42 +03:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&fs->lfs_interlock);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_bfree -= (btofsb(fs, ninos * fs->lfs_ibsize) +
btofsb(fs, fs->lfs_sumsize));
From Konrad Schroeder, in response to strange df output on anoncvs.netbsd.org: We were returning the wrong value for free space. Now we're not.
2006-03-18 02:21:01 +03:00
fs->lfs_dmeta += (btofsb(fs, ninos * fs->lfs_ibsize) +
btofsb(fs, fs->lfs_sumsize));
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* When we simply write the blocks we lose a rotation for every block
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
* written. To avoid this problem, we cluster the buffers into a
* chunk and write the chunk. MAXPHYS is the largest size I/O
* devices can handle, use that for the size of the chunks.
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
*
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
* Blocks that are already clusters (from GOP_WRITE), however, we
* don't bother to copy into other clusters.
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#define CHUNKSIZE MAXPHYS
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (devvp == NULL)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
panic("devvp is NULL");
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
for (bpp = sp->bpp, i = nblocks; i;) {
cbp = lfs_newclusterbuf(fs, devvp, (*bpp)->b_blkno, i);
use b_private rather than b_saveaddr. XXX LFS_USE_B_INVAL
2003-12-04 17:57:47 +03:00
cl = cbp->b_private;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
cbp->b_flags |= B_ASYNC | B_BUSY;
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
cbp->b_bcount = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#if defined(DEBUG) && defined(DIAGNOSTIC)
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
if (bpp - sp->bpp > (fs->lfs_sumsize - SEGSUM_SIZE(fs))
/ sizeof(int32_t)) {
panic("lfs_writeseg: real bpp overwrite");
}
use correct segment size. this fixes memory corruption when using lfsv1.
2004-03-09 09:43:18 +03:00
if (bpp - sp->bpp > segsize(fs) / fs->lfs_fsize) {
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
panic("lfs_writeseg: theoretical bpp overwrite");
}
Change lfs_sb_cksum to use offsetof() instead of an inlined version. Fix lfs_vref/lfs_vunredf to ignore VXLOCKed vnodes that are also being flushed. Improve the debugging messages somewhat.
1999-03-26 00:54:10 +03:00
#endif
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/*
* Construct the cluster.
*/
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&fs->lfs_interlock);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
++fs->lfs_iocount;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
KNF (space after keywords).
2003-03-21 09:26:36 +03:00
while (i && cbp->b_bcount < CHUNKSIZE) {
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
bp = *bpp;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
if (bp->b_bcount > (CHUNKSIZE - cbp->b_bcount))
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
break;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
if (cbp->b_bcount > 0 && !(cl->flags & LFS_CL_MALLOC))
break;
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
/* Clusters from GOP_WRITE are expedited */
if (bp->b_bcount > fs->lfs_bsize) {
if (cbp->b_bcount > 0)
/* Put in its own buffer */
break;
else {
cbp->b_data = bp->b_data;
}
} else if (cbp->b_bcount == 0) {
p = cbp->b_data = lfs_malloc(fs, CHUNKSIZE,
LFS_NB_CLUSTER);
cl->flags |= LFS_CL_MALLOC;
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#ifdef DIAGNOSTIC
if (dtosn(fs, dbtofsb(fs, bp->b_blkno +
btodb(bp->b_bcount - 1))) !=
sp->seg_number) {
change some members of struct buf from long to int. ride on 2.0H.
2004-09-18 20:40:11 +04:00
printf("blk size %d daddr %" PRIx64
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
" not in seg %d\n",
bp->b_bcount, bp->b_blkno,
sp->seg_number);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
panic("segment overwrite");
}
#endif
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
#ifdef LFS_USE_B_INVAL
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Fake buffers from the cleaner are marked as B_INVAL.
* We need to copy the data from user space rather than
* from the buffer indicated.
* XXX == what do I do on an error?
*/
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
if ((bp->b_flags & (B_CALL|B_INVAL)) ==
(B_CALL|B_INVAL)) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (copyin(bp->b_saveaddr, p, bp->b_bcount))
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
panic("lfs_writeseg: "
"copyin failed [2]");
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
} else
#endif /* LFS_USE_B_INVAL */
if (cl->flags & LFS_CL_MALLOC) {
- a comment. - bcopy -> memcpy - increase 'p' only when needed.
2003-10-08 19:07:25 +04:00
/* copy data into our cluster. */
memcpy(p, bp->b_data, bp->b_bcount);
p += bp->b_bcount;
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
}
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
cbp->b_bcount += bp->b_bcount;
cl->bufsize += bp->b_bcount;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
bp->b_flags &= ~(B_ERROR | B_READ | B_DELWRI | B_DONE);
cl->bpp[cl->bufcount++] = bp;
vp = bp->b_vp;
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
s = splbio();
reassignbuf() when lfs_writeseg() takes away B_DELWRI.
2003-10-03 19:35:03 +04:00
reassignbuf(bp, vp);
Make the buffer cache code MP-safe.
2003-02-06 00:38:38 +03:00
V_INCR_NUMOUTPUT(vp);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
splx(s);
Make sure that the wakeup occurs for vnodes that lfs_update might be sleeping on (nodes which are not marked IN_MODIFIED/IN_CLEANING, but which have dirty buffers), by marking them with the appropriate flag if dirtybuffers were added while the write was in progress.
1999-04-12 04:25:13 +04:00
bpp++;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
i--;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
store a i/o priority hint in struct buf for buffer queue discipline.
2004-01-10 17:39:50 +03:00
if (fs->lfs_sp->seg_flags & SEGM_SYNC)
BIO_SETPRIO(cbp, BPRIO_TIMECRITICAL);
else
BIO_SETPRIO(cbp, BPRIO_TIMELIMITED);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
s = splbio();
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
V_INCR_NUMOUTPUT(devvp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
splx(s);
eliminate tricky usages of VOP_STRATEGY which are (no longer?) necessary.
2004-01-28 13:53:12 +03:00
VOP_STRATEGY(devvp, cbp);
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
curproc->p_stats->p_ru.ru_oublock++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (lfs_dostats) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.psegwrites;
lfs_stats.blocktot += nblocks - 1;
if (fs->lfs_sp->seg_flags & SEGM_SYNC)
++lfs_stats.psyncwrites;
if (fs->lfs_sp->seg_flags & SEGM_CLEAN) {
++lfs_stats.pcleanwrites;
lfs_stats.cleanblocks += nblocks - 1;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Install a new sysctl, vfs.lfs.ignore_lazy_sync, which causes LFS to ignore the "smooth" syncer, as if vfs.sync.*delay = 0, but only for LFS. The default is "on", i.e., ignore lazy sync. Reduce the amount of polling/busy-waiting done by lfs_putpages(). To accomplish this, copied genfs_putpages() and modified it to indicate which page it was that caused it to return with EDEADLK. fsync()/fdatasync() should no longer ever fail with EAGAIN, and should not consume huge quantities of cpu. Also, try to make dirops less likely to be written as the result of a VOP_PUTPAGES(), while ensuring that they are written regularly.
2007-04-17 05:16:46 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
return (lfs_initseg(fs) || do_again);
}
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writesuper(struct lfs *fs, daddr_t daddr)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct buf *bp;
int s;
eliminate tricky usages of VOP_STRATEGY which are (no longer?) necessary.
2004-01-28 13:53:12 +03:00
struct vnode *devvp = VTOI(fs->lfs_ivnode)->i_devvp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_MAYBE_SEGLOCK(fs);
Various minor LFS improvements: * Note when lfs_putpages(9) thinks it is not going to be writing any pages before calling genfs_putpages(9). This prevents a situation in which blocks can be queued for writing without a segment header. * Correct computation of NRESERVE(), though it is still a gross overestimate in most cases. Note that if NRESERVE() is too high, it may be impossible to create files on the filesystem. We catch this case on filesystem mount and refuse to mount r/w. * Allow filesystems to be mounted whose block size is == MAXBSIZE. * Somewhere along the line, ufs_bmaparray(9) started mangling UNWRITTEN entries in indirect blocks again, triggering a failed assertion "daddr <= LFS_MAX_DADDR". Explicitly convert to and from int32_t to correct this. * Add a high-water mark for the number of dirty pages any given LFS can hold before triggering a flush. This is settable by sysctl, but off (zero) by default. * Be more careful about the MAX_BYTES and MAX_BUFS computations so we shouldn't see "please increase to at least zero" messages. * Note that VBLK and VCHR vnodes can have nonzero values in di_db[0] even though their v_size == 0. Don't panic when we see this. * Change lfs_bfree to a signed quantity. The manner in which it is processed before being passed to the cleaner means that sometimes it may drop below zero, and the cleaner must be aware of this. * Never report bfree < 0 (or higher than lfs_dsize) through lfs_statvfs(9). This prevents df(1) from ever telling us that our full filesystems have 16TB free. * Account space allocated through lfs_balloc(9) that does not have associated buffer headers, so that the pagedaemon doesn't run us out of segments. * Return ENOSPC from lfs_balloc(9) when bfree drops to zero. * Address a deadlock in lfs_bmapv/lfs_markv when the filesystem is being unmounted. Because vfs_busy() is a shared lock, and lfs_bmapv/lfs_markv mark the filesystem vfs_busy(), the cleaner can be holding the lock that umount() is blocking on, then try to vfs_busy() again in getnewvnode().
2005-02-26 08:40:42 +03:00
#ifdef DIAGNOSTIC
KASSERT(fs->lfs_magic == LFS_MAGIC);
#endif
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
* If we can write one superblock while another is in
* progress, we risk not having a complete checkpoint if we crash.
* So, block here if a superblock write is in progress.
*/
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&fs->lfs_interlock);
Fix spllevel problem with superblock exclusion and with segment write throttle. May address PR#8383.
1999-11-17 23:08:30 +03:00
s = splbio();
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
while (fs->lfs_sbactive) {
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ltsleep(&fs->lfs_sbactive, PRIBIO+1, "lfs sb", 0,
&fs->lfs_interlock);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
fs->lfs_sbactive = daddr;
Fix spllevel problem with superblock exclusion and with segment write throttle. May address PR#8383.
1999-11-17 23:08:30 +03:00
splx(s);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/* Set timestamp of this version of the superblock */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (fs->lfs_version == 1)
merge FreeBSD timecounters from branch simonb-timecounters - struct timeval time is gone time.tv_sec -> time_second - struct timeval mono_time is gone mono_time.tv_sec -> time_uptime - access to time via {get,}{micro,nano,bin}time() get* versions are fast but less precise - support NTP nanokernel implementation (NTP API 4) - further reading: Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
2006-06-08 02:33:33 +04:00
fs->lfs_otstamp = time_second;
fs->lfs_tstamp = time_second;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Checksum the superblock and copy it into a buffer. */
PR#6032: define fixed sized on-disk superblock structure.
1998-09-12 01:27:12 +04:00
fs->lfs_cksum = lfs_sb_cksum(&(fs->lfs_dlfs));
eliminate tricky usages of VOP_STRATEGY which are (no longer?) necessary.
2004-01-28 13:53:12 +03:00
bp = lfs_newbuf(fs, devvp,
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
fsbtodb(fs, daddr), LFS_SBPAD, LFS_NB_SBLOCK);
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
memset((char *)bp->b_data + sizeof(struct dlfs), 0,
- wrap long lines. - remove a mysterious blank line.
2003-07-12 20:17:52 +04:00
LFS_SBPAD - sizeof(struct dlfs));
PR#6032: define fixed sized on-disk superblock structure.
1998-09-12 01:27:12 +04:00
*(struct dlfs *)bp->b_data = fs->lfs_dlfs;
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
bp->b_flags |= B_BUSY | B_CALL | B_ASYNC;
bp->b_flags &= ~(B_DONE | B_ERROR | B_READ | B_DELWRI);
bp->b_iodone = lfs_supercallback;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
store a i/o priority hint in struct buf for buffer queue discipline.
2004-01-10 17:39:50 +03:00
if (fs->lfs_sp != NULL && fs->lfs_sp->seg_flags & SEGM_SYNC)
BIO_SETPRIO(bp, BPRIO_TIMECRITICAL);
else
BIO_SETPRIO(bp, BPRIO_TIMELIMITED);
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
curproc->p_stats->p_ru.ru_oublock++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
s = splbio();
Make the buffer cache code MP-safe.
2003-02-06 00:38:38 +03:00
V_INCR_NUMOUTPUT(bp->b_vp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
splx(s);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&fs->lfs_interlock);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
++fs->lfs_iocount;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
eliminate tricky usages of VOP_STRATEGY which are (no longer?) necessary.
2004-01-28 13:53:12 +03:00
VOP_STRATEGY(devvp, bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
/*
* Logical block number match routines used when traversing the dirty block
* chain.
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_fake(struct lfs *fs, struct buf *bp)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
{
KNF.
2003-07-23 17:53:51 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
return LFS_IS_MALLOC_BUF(bp);
}
#if 0
int
lfs_match_real(struct lfs *fs, struct buf *bp)
{
KNF.
2003-07-23 17:53:51 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
return (lfs_match_data(fs, bp) && !lfs_match_fake(fs, bp));
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#endif
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_data(struct lfs *fs, struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
KNF.
2003-07-23 17:53:51 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
return (bp->b_lblkno >= 0);
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_indir(struct lfs *fs, struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t lbn;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lbn = bp->b_lblkno;
return (lbn < 0 && (-lbn - NDADDR) % NINDIR(fs) == 0);
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_dindir(struct lfs *fs, struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t lbn;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lbn = bp->b_lblkno;
return (lbn < 0 && (-lbn - NDADDR) % NINDIR(fs) == 1);
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_tindir(struct lfs *fs, struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t lbn;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(fs);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lbn = bp->b_lblkno;
return (lbn < 0 && (-lbn - NDADDR) % NINDIR(fs) == 2);
}
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
static void
lfs_free_aiodone(struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
struct lfs *fs;
use b_private rather than b_saveaddr. XXX LFS_USE_B_INVAL
2003-12-04 17:57:47 +03:00
fs = bp->b_private;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_NO_SEGLOCK(fs);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_freebuf(fs, bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
static void
lfs_super_aiodone(struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
struct lfs *fs;
use b_private rather than b_saveaddr. XXX LFS_USE_B_INVAL
2003-12-04 17:57:47 +03:00
fs = bp->b_private;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_NO_SEGLOCK(fs);
simple_lock(&fs->lfs_interlock);
NULL != 0
2000-05-19 08:34:39 +04:00
fs->lfs_sbactive = 0;
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
if (--fs->lfs_iocount <= 1)
Fixes associated with filling an LFS: Change the space computation to appear to change the size of the *disk* rather than the *bytes used* when more segment summaries and inode blocks are written. Try to estimate the amount of space that these will take up when more files are written, so the disk size doesn't change too much. Regularize error returns from lfs_valloc, lfs_balloc, lfs_truncate: they now fail entirely, rather than succeeding half-way and leaving the fs in an inconsistent state. Rewrite lfs_truncate, mostly stealing from ffs_truncate. The old lfs_truncate had difficulty truncating a large file to a non-zero size (indirect blocks were not handled appropriately). Unmark VDIROP on fvp after ufs_remove, ufs_rmdir, so these can be reclaimed immediately: this vnode would not be written to disk again anyway if the removal succeeded, and if it failed, no directory operation occurred. ufs_makeinode and ufs_mkdir now remove IN_ADIROP on error.
2000-06-28 00:57:11 +04:00
wakeup(&fs->lfs_iocount);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
wakeup(&fs->lfs_sbactive);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_freebuf(fs, bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
static void
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
lfs_cluster_aiodone(struct buf *bp)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
{
struct lfs_cluster *cl;
struct lfs *fs;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
struct buf *tbp, *fbp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
struct vnode *vp, *devvp;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
struct inode *ip;
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
int s, error=0;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
KNF (space after keywords).
2003-03-21 09:26:36 +03:00
if (bp->b_flags & B_ERROR)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
error = bp->b_error;
use b_private rather than b_saveaddr. XXX LFS_USE_B_INVAL
2003-12-04 17:57:47 +03:00
cl = bp->b_private;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
fs = cl->fs;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
devvp = VTOI(fs->lfs_ivnode)->i_devvp;
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_NO_SEGLOCK(fs);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/* Put the pages back, and release the buffer */
KNF (space after keywords).
2003-03-21 09:26:36 +03:00
while (cl->bufcount--) {
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
tbp = cl->bpp[cl->bufcount];
lfs_cluster_aiodone: turn an invariant condition into an assertion.
2004-05-19 15:29:32 +04:00
KASSERT(tbp->b_flags & B_BUSY);
KNF (space after keywords).
2003-03-21 09:26:36 +03:00
if (error) {
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
tbp->b_flags |= B_ERROR;
tbp->b_error = error;
}
/*
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* We're done with tbp. If it has not been re-dirtied since
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
* the cluster was written, free it. Otherwise, keep it on
* the locked list to be written again.
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
vp = tbp->b_vp;
Use VONWORKLST as a heuristic for vnode emptiness, rather than exhaustively checking the memq. Take greater care not to dirty the Ifile vnode when unmounting the filesystem. This should fix a "(vp->v_flag & VONWORKLST) == 0" assertion panic in vgonel that could occur when unmounting. Do not allow the Ifile to be mapped for writing.
2003-03-21 09:16:53 +03:00
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
tbp->b_flags &= ~B_GATHERED;
LFS_BCLEAN_LOG(fs, tbp);
KNF (space after keywords).
2003-03-21 09:26:36 +03:00
if (!(tbp->b_flags & B_CALL)) {
assertions.
2003-10-03 19:35:54 +04:00
KASSERT(tbp->b_flags & B_LOCKED);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
s = splbio();
- buffer cache MP locks. - avoid changing buffer state on the free queue.
2003-09-07 15:47:07 +04:00
simple_lock(&bqueue_slock);
bremfree(tbp);
simple_unlock(&bqueue_slock);
KNF (space after keywords).
2003-03-21 09:26:36 +03:00
if (vp)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
reassignbuf(tbp, vp);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
splx(s);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
tbp->b_flags |= B_ASYNC; /* for biodone */
}
- buffer cache MP locks. - avoid changing buffer state on the free queue.
2003-09-07 15:47:07 +04:00
if ((tbp->b_flags & (B_LOCKED | B_DELWRI)) == B_LOCKED)
LFS_UNLOCK_BUF(tbp);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if (tbp->b_flags & B_DONE) {
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "blk %d biodone already (flags %lx)\n",
cl->bufcount, (long)tbp->b_flags));
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
lfs_cluster_aiodone: turn an invariant condition into an assertion.
2004-05-19 15:29:32 +04:00
if ((tbp->b_flags & B_CALL) && !LFS_IS_MALLOC_BUF(tbp)) {
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
/*
lfs_cluster_aiodone: turn an invariant condition into an assertion.
2004-05-19 15:29:32 +04:00
* A buffer from the page daemon.
* We use the same iodone as it does,
* so we must manually disassociate its
* buffers from the vp.
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
*/
lfs_cluster_aiodone: turn an invariant condition into an assertion.
2004-05-19 15:29:32 +04:00
if (tbp->b_vp) {
/* This is just silly */
s = splbio();
brelvp(tbp);
tbp->b_vp = vp;
splx(s);
}
/* Put it back the way it was */
tbp->b_flags |= B_ASYNC;
/* Master buffers have B_AGE */
if (tbp->b_private == tbp)
tbp->b_flags |= B_AGE;
}
s = splbio();
biodone(tbp);
/*
* If this is the last block for this vnode, but
* there are other blocks on its dirty list,
* set IN_MODIFIED/IN_CLEANING depending on what
* sort of block. Only do this for our mount point,
* not for, e.g., inode blocks that are attached to
* the devvp.
* XXX KS - Shouldn't we set *both* if both types
* of blocks are present (traverse the dirty list?)
*/
simple_lock(&global_v_numoutput_slock);
if (vp != devvp && vp->v_numoutput == 0 &&
(fbp = LIST_FIRST(&vp->v_dirtyblkhd)) != NULL) {
ip = VTOI(vp);
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
DLOG((DLOG_SEG, "lfs_cluster_aiodone: mark ino %d\n",
ip->i_number));
lfs_cluster_aiodone: turn an invariant condition into an assertion.
2004-05-19 15:29:32 +04:00
if (LFS_IS_MALLOC_BUF(fbp))
LFS_SET_UINO(ip, IN_CLEANING);
else
LFS_SET_UINO(ip, IN_MODIFIED);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
lfs_cluster_aiodone: turn an invariant condition into an assertion.
2004-05-19 15:29:32 +04:00
simple_unlock(&global_v_numoutput_slock);
splx(s);
wakeup(vp);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
/* Fix up the cluster buffer, and release it */
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
if (cl->flags & LFS_CL_MALLOC)
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_free(fs, bp->b_data, LFS_NB_CLUSTER);
- add simple functions to allocate/free a buffer for i/o. - make bufpool static.
2006-01-04 13:13:05 +03:00
putiobuf(bp);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
/* Note i/o done */
if (cl->flags & LFS_CL_SYNC) {
nuke trailing whitespace
2005-02-27 01:31:44 +03:00
if (--cl->seg->seg_iocount == 0)
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
wakeup(&cl->seg->seg_iocount);
}
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_lock(&fs->lfs_interlock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#ifdef DIAGNOSTIC
if (fs->lfs_iocount == 0)
remove trailing \n in panic(). approved perry.
2002-09-27 19:35:29 +04:00
panic("lfs_cluster_aiodone: zero iocount");
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#endif
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
if (--fs->lfs_iocount <= 1)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
wakeup(&fs->lfs_iocount);
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
simple_unlock(&fs->lfs_interlock);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
pool_put(&fs->lfs_bpppool, cl->bpp);
cl->bpp = NULL;
pool_put(&fs->lfs_clpool, cl);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
}
static void
lfs_generic_callback(struct buf *bp, void (*aiodone)(struct buf *))
{
/* reset b_iodone for when this is a single-buf i/o. */
bp->b_iodone = aiodone;
merge yamt-splraiseipl branch. - finish implementing splraiseipl (and makeiplcookie). http://mail-index.NetBSD.org/tech-kern/2006/07/01/0000.html - complete workqueue(9) and fix its ipl problem, which is reported to cause audio skipping. - fix netbt (at least compilation problems) for some ports. - fix PR/33218.
2006-12-21 18:55:21 +03:00
workqueue_enqueue(uvm.aiodone_queue, &bp->b_work);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
}
static void
lfs_cluster_callback(struct buf *bp)
{
KNF.
2003-07-23 17:53:51 +04:00
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
lfs_generic_callback(bp, lfs_cluster_aiodone);
}
void
lfs_supercallback(struct buf *bp)
{
KNF.
2003-07-23 17:53:51 +04:00
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
lfs_generic_callback(bp, lfs_super_aiodone);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
Changes to help the roll-forward agent, to wit: * Mark being-deleted files in the Ifile so we can finish deleting them at fs mount time. * Flag the Ifile with "cleaner must clean" when writers are waiting for the cleaner, rather than relying solely on the cleaner's estimation of whether it should clean or not. * Note partial segments written by a user agent (in particular, fsck_lfs) so that repeated rolls forward don't interfere with one another. * Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once, for better testing of the validity of checkpoints. * Keep track of the on-disk nlink count when cleaning, so that we don't partially complete directory operations while cleaning. * Ensure that every single Ifile inode write represents a consistent view of the filesystem. In particular, the accounting for the segment we are writing the inode into must be correct, and the accounting for the segment that inode used to reside in must be correct. Rather than just rewriting the inode if we wrote it wrong, rewrite the necessary ifile blocks before writing the inode so we never write it wrong. * Don't unmark any VDIROP vnodes if we haven't written them to disk, avoiding yet another problem with the "wait for the cleaner" error return from lfs_putpages(). Also, move the last callback to an aiodone call, so we no longer do any memory management from interrupt context.
2006-09-01 23:41:28 +04:00
/*
* The only buffers that are going to hit these functions are the
* segment write blocks, or the segment summaries, or the superblocks.
*
* All of the above are created by lfs_newbuf, and so do not need to be
* released via brelse.
*/
void
lfs_callback(struct buf *bp)
{
lfs_generic_callback(bp, lfs_free_aiodone);
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Shellsort (diminishing increment sort) from Data Structures and
* Algorithms, Aho, Hopcraft and Ullman, 1983 Edition, page 290;
* see also Knuth Vol. 3, page 84. The increments are selected from
* formula (8), page 95. Roughly O(N^3/2).
*/
/*
* This is our own private copy of shellsort because we want to sort
* two parallel arrays (the array of buffer pointers and the array of
* logical block numbers) simultaneously. Note that we cast the array
* of logical block numbers to a unsigned in this routine so that the
* negative block numbers (meta data blocks) sort AFTER the data blocks.
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
void
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
lfs_shellsort(struct buf **bp_array, int32_t *lb_array, int nmemb, int size)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
static int __rsshell_increments[] = { 4, 1, 0 };
Remove register declarations.
2000-03-30 16:41:09 +04:00
int incr, *incrp, t1, t2;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
struct buf *bp_temp;
add assertions and a debug check.
2003-04-01 18:58:43 +04:00
#ifdef DEBUG
incr = 0;
for (t1 = 0; t1 < nmemb; t1++) {
for (t2 = 0; t2 * size < bp_array[t1]->b_bcount; t2++) {
if (lb_array[incr++] != bp_array[t1]->b_lblkno + t2) {
/* dump before panic */
printf("lfs_shellsort: nmemb=%d, size=%d\n",
nmemb, size);
incr = 0;
for (t1 = 0; t1 < nmemb; t1++) {
const struct buf *bp = bp_array[t1];
printf("bp[%d]: lbn=%" PRIu64 ", size=%"
PRIu64 "\n", t1,
(uint64_t)bp->b_bcount,
(uint64_t)bp->b_lblkno);
printf("lbns:");
for (t2 = 0; t2 * size < bp->b_bcount;
t2++) {
printf(" %" PRId32,
lb_array[incr++]);
}
printf("\n");
}
panic("lfs_shellsort: inconsistent input");
}
}
}
#endif
lfs prototypes
1996-02-10 01:28:45 +03:00
for (incrp = __rsshell_increments; (incr = *incrp++) != 0;)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
for (t1 = incr; t1 < nmemb; ++t1)
for (t2 = t1 - incr; t2 >= 0;)
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
if ((u_int32_t)bp_array[t2]->b_lblkno >
(u_int32_t)bp_array[t2 + incr]->b_lblkno) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
bp_temp = bp_array[t2];
bp_array[t2] = bp_array[t2 + incr];
bp_array[t2 + incr] = bp_temp;
t2 -= incr;
} else
break;
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
/* Reform the list of logical blocks */
incr = 0;
for (t1 = 0; t1 < nmemb; t1++) {
for (t2 = 0; t2 * size < bp_array[t1]->b_bcount; t2++) {
lb_array[incr++] = bp_array[t1]->b_lblkno + t2;
}
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
/*
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
* Call vget with LK_NOWAIT. If we are the one who holds VXLOCK/VFREEING,
* however, we must press on. Just fake success in that case.
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
lfs prototypes
1996-02-10 01:28:45 +03:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_vref(struct vnode *vp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
int error;
struct lfs *fs;
fs = VTOI(vp)->i_lfs;
ASSERT_MAYBE_SEGLOCK(fs);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
* If we return 1 here during a flush, we risk vinvalbuf() not
* being able to flush all of the pages from this vnode, which
* will cause it to panic. So, return 0 if a flush is in progress.
*/
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
error = vget(vp, LK_NOWAIT);
if (error == EBUSY && IS_FLUSHING(VTOI(vp)->i_lfs, vp)) {
++fs->lfs_flushvp_fakevref;
return 0;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
return error;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
/*
* This is vrele except that we do not want to VOP_INACTIVE this vnode. We
* inline vrele here to avoid the vn_lock and VOP_INACTIVE call at the end.
*/
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_vunref(struct vnode *vp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
struct lfs *fs;
fs = VTOI(vp)->i_lfs;
ASSERT_MAYBE_SEGLOCK(fs);
Change lfs_sb_cksum to use offsetof() instead of an inlined version. Fix lfs_vref/lfs_vunredf to ignore VXLOCKed vnodes that are also being flushed. Improve the debugging messages somewhat.
1999-03-26 00:54:10 +03:00
/*
* Analogous to lfs_vref, if the node is flushing, fake it.
*/
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING explicitly (especially since we didn't know about VFREEING at all before), but notice the EBUSY return from vget() instead. Fix some more MP locking protocol issues, most of which were pointed out by Christian Ehrhardt this morning on tech-kern.
2006-04-14 03:46:28 +04:00
if (IS_FLUSHING(fs, vp) && fs->lfs_flushvp_fakevref) {
--fs->lfs_flushvp_fakevref;
Change lfs_sb_cksum to use offsetof() instead of an inlined version. Fix lfs_vref/lfs_vunredf to ignore VXLOCKed vnodes that are also being flushed. Improve the debugging messages somewhat.
1999-03-26 00:54:10 +03:00
return;
}
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
simple_lock(&vp->v_interlock);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#ifdef DIAGNOSTIC
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_usecount <= 0) {
64 bit inode changes.
2005-08-19 06:04:03 +04:00
printf("lfs_vunref: inum is %llu\n", (unsigned long long)
VTOI(vp)->i_number);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
printf("lfs_vunref: flags are 0x%lx\n", (u_long)vp->v_flag);
printf("lfs_vunref: usecount = %ld\n", (long)vp->v_usecount);
Straighten out the maze of ifdefs. Instead, consolidate all the debugging stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS statistics in sysctl, while I'm there. A bit of a rototill.
2005-03-08 03:18:19 +03:00
panic("lfs_vunref: v_usecount < 0");
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
#endif
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
vp->v_usecount--;
if (vp->v_usecount > 0) {
simple_unlock(&vp->v_interlock);
return;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
* insert at tail of LRU list
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
simple_lock(&vnode_free_list_slock);
Changes to stabilize LFS. The first two of these should also apply to the 1.4 branch. * Use a separate per-fs lock, instead of ufs_hashlock, to protect the Inode free list. This seems to prevent the "lockmgr: %d, not exclusive lock holder %d, unlocking" message I was mis-attributing last night to an unlocked vnode being passed to vrele. * Change calling semantics of lfs_ifind, to give better error reporting: If fed a struct buf, it can report the block number of the offending inode block as well as the inode number. * Back out rev 1.10 of lfs_subr.c, since the replacement code was slightly uglier while being functionally identical. * Make lfs_vunref use the same free list convention as vrele/vput, so that vget does not remove vnodes from a hash list they are not on.
2000-01-19 03:03:04 +03:00
if (vp->v_holdcnt > 0)
TAILQ_INSERT_TAIL(&vnode_hold_list, vp, v_freelist);
else
TAILQ_INSERT_TAIL(&vnode_free_list, vp, v_freelist);
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
simple_unlock(&vnode_free_list_slock);
simple_unlock(&vp->v_interlock);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
* We use this when we have vnodes that were loaded in solely for cleaning.
* There is no reason to believe that these vnodes will be referenced again
* soon, since the cleaning process is unrelated to normal filesystem
* activity. Putting cleaned vnodes at the tail of the list has the effect
* of flushing the vnode LRU. So, put vnodes that were loaded only for
* cleaning at the head of the list, instead.
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_vunref_head(struct vnode *vp)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
{
KNF.
2003-07-23 17:53:51 +04:00
Protect various per-fs structures with fs->lfs_interlock simple_lock, to improve behavior in the multiprocessor case. Add debugging segment-lock assertion statements.
2005-04-02 01:59:46 +04:00
ASSERT_SEGLOCK(VTOI(vp)->i_lfs);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
simple_lock(&vp->v_interlock);
#ifdef DIAGNOSTIC
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_usecount == 0) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
panic("lfs_vunref: v_usecount<0");
}
#endif
vp->v_usecount--;
if (vp->v_usecount > 0) {
simple_unlock(&vp->v_interlock);
return;
}
/*
* insert at head of LRU list
*/
simple_lock(&vnode_free_list_slock);
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
if (vp->v_holdcnt > 0)
TAILQ_INSERT_TAIL(&vnode_hold_list, vp, v_freelist);
else
TAILQ_INSERT_HEAD(&vnode_free_list, vp, v_freelist);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
simple_unlock(&vnode_free_list_slock);
simple_unlock(&vp->v_interlock);
}
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
/*
* Set up an FINFO entry for a new file. The fip pointer is assumed to
* point at uninitialized space.
*/
void
lfs_acquire_finfo(struct lfs *fs, ino_t ino, int vers)
{
struct segment *sp = fs->lfs_sp;
Fix a bug in which FINFOs were written with a version number of zero. Add assertions and add this to the DEBUG fip test in lfs_writeseg.
2006-05-20 05:10:18 +04:00
KASSERT(vers > 0);
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
if (sp->seg_bytes_left < fs->lfs_bsize ||
sp->sum_bytes_left < sizeof(struct finfo))
(void) lfs_writeseg(fs, fs->lfs_sp);
sp->sum_bytes_left -= FINFOSIZE;
++((SEGSUM *)(sp->segsum))->ss_nfinfo;
sp->fip->fi_nblocks = 0;
sp->fip->fi_ino = ino;
sp->fip->fi_version = vers;
}
/*
* Release the FINFO entry, either clearing out an unused entry or
* advancing us to the next available entry.
*/
void
lfs_release_finfo(struct lfs *fs)
{
struct segment *sp = fs->lfs_sp;
if (sp->fip->fi_nblocks != 0) {
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
2007-03-04 08:59:00 +03:00
sp->fip = (FINFO*)((char *)sp->fip + FINFOSIZE +
Break out the finfo array manipulation code into two new functions, lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check for zero-length finfo arrays in the segment summary to avoid future regressions.
2006-05-19 03:15:09 +04:00
sizeof(int32_t) * sp->fip->fi_nblocks);
sp->start_lbp = &sp->fip->fi_blocks[0];
} else {
sp->sum_bytes_left += FINFOSIZE;
--((SEGSUM *)(sp->segsum))->ss_nfinfo;
}
}
Reference in New Issue Copy Permalink