Aren/NetBSD
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2bdc2add9c
NetBSD/sys/ufs/lfs/lfs_segment.c

2455 lines
67 KiB
C
Raw Normal View History

Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
/* $NetBSD: lfs_segment.c,v 1.112 2003/03/20 06:51:17 perseant Exp $ */
New RCS ID's, take two. they're more aesthecially pleasant, and use 'NetBSD'
1994-06-29 10:39:25 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*-
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* Copyright (c) 1999, 2000, 2001, 2002, 2003 The NetBSD Foundation, Inc.
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
* All rights reserved.
*
* This code is derived from software contributed to The NetBSD Foundation
* by Konrad E. Schroder <perseant@hhhh.org>.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* This product includes software developed by the NetBSD
* Foundation, Inc. and its contributors.
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
* 4. Neither the name of The NetBSD Foundation nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Copyright (c) 1991, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
* This product includes software developed by the University of
* California, Berkeley and its contributors.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
* @(#)lfs_segment.c 8.10 (Berkeley) 6/10/95
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
add RCSID
2001-11-08 05:39:06 +03:00
#include <sys/cdefs.h>
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
__KERNEL_RCSID(0, "$NetBSD: lfs_segment.c,v 1.112 2003/03/20 06:51:17 perseant Exp $");
add RCSID
2001-11-08 05:39:06 +03:00
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
#define ivndebug(vp,str) printf("ino %d: %s\n",VTOI(vp)->i_number,(str))
use _KERNEL_OPT
2001-05-30 15:57:16 +04:00
#if defined(_KERNEL_OPT)
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
#include "opt_ddb.h"
only include opt_ddb.h for !LKM
2000-11-30 18:59:47 +03:00
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#include <sys/param.h>
#include <sys/systm.h>
#include <sys/namei.h>
#include <sys/kernel.h>
#include <sys/resourcevar.h>
#include <sys/file.h>
#include <sys/stat.h>
#include <sys/buf.h>
#include <sys/proc.h>
#include <sys/vnode.h>
#include <sys/mount.h>
#include <miscfs/specfs/specdev.h>
#include <miscfs/fifofs/fifo.h>
#include <ufs/ufs/inode.h>
#include <ufs/ufs/dir.h>
#include <ufs/ufs/ufsmount.h>
#include <ufs/ufs/ufs_extern.h>
#include <ufs/lfs/lfs.h>
#include <ufs/lfs/lfs_extern.h>
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
#include <uvm/uvm.h>
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#include <uvm/uvm_extern.h>
Add extensible malloc types, adapted from FreeBSD. This turns malloc types into a structure, a pointer to which is passed around, instead of an int constant. Allow the limit to be adjusted when the malloc type is defined, or with a function call, as suggested by Jonathan Stone.
2003-02-01 09:23:35 +03:00
MALLOC_DEFINE(M_SEGMENT, "LFS segment", "Segment for LFS");
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
extern int count_lock_queue(void);
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
extern struct simplelock vnode_free_list_slock; /* XXX */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
static void lfs_generic_callback(struct buf *, void (*)(struct buf *));
static void lfs_super_aiodone(struct buf *);
static void lfs_cluster_aiodone(struct buf *);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
static void lfs_cluster_callback(struct buf *);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Determine if it's OK to start a partial in this segment, or if we need
* to go on to a new segment.
*/
#define LFS_PARTIAL_FITS(fs) \
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
((fs)->lfs_fsbpseg - ((fs)->lfs_offset - (fs)->lfs_curseg) > \
fragstofsb((fs), (fs)->lfs_frag))
void lfs_callback(struct buf *);
int lfs_gather(struct lfs *, struct segment *,
struct vnode *, int (*)(struct lfs *, struct buf *));
int lfs_gatherblock(struct segment *, struct buf *, int *);
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
void lfs_iset(struct inode *, daddr_t, time_t);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
int lfs_match_fake(struct lfs *, struct buf *);
int lfs_match_data(struct lfs *, struct buf *);
int lfs_match_dindir(struct lfs *, struct buf *);
int lfs_match_indir(struct lfs *, struct buf *);
int lfs_match_tindir(struct lfs *, struct buf *);
void lfs_newseg(struct lfs *);
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
/* XXX ondisk32 */
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
void lfs_shellsort(struct buf **, int32_t *, int, int);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
void lfs_supercallback(struct buf *);
void lfs_updatemeta(struct segment *);
int lfs_vref(struct vnode *);
void lfs_vunref(struct vnode *);
void lfs_writefile(struct lfs *, struct segment *, struct vnode *);
int lfs_writeinode(struct lfs *, struct segment *, struct inode *);
int lfs_writeseg(struct lfs *, struct segment *);
void lfs_writesuper(struct lfs *, daddr_t);
int lfs_writevnodes(struct lfs *fs, struct mount *mp,
struct segment *sp, int dirops);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int lfs_allclean_wakeup; /* Cleaner wakeup address. */
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
int lfs_writeindir = 1; /* whether to flush indir on non-ckp */
Better checking for held inode locks in lfs_fastvget, for a number of error conditions. Also change the default setting of lfs_clean_vnhead to 0, which seems to make the locking problems go away (although this is difficult to test as I can't reliably reproduce them).
1999-04-12 04:11:01 +04:00
int lfs_clean_vnhead = 0; /* Allow freeing to head of vn list */
Address ufs_hashlock/ufs_ihashins protocol bug, discovered while doing a post-mortem of a production machine. Also, take the active dirop count off of the fs and make it global (since it is measuring a global resource) and tie the threshold value LFS_MAXDIROP to desiredvnodes.
1999-11-06 23:33:05 +03:00
int lfs_dirvcount = 0; /* # active dirops */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Statistics Counters */
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int lfs_dostats = 1;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
struct lfs_stats lfs_stats;
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
extern int locked_queue_count;
extern long locked_queue_bytes;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* op values to lfs_writevnodes */
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
#define VN_REG 0
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#define VN_DIROP 1
#define VN_EMPTY 2
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
#define VN_CLEAN 3
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
#define LFS_MAX_ACTIVE 10
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
* XXX KS - Set modification time on the Ifile, so the cleaner can
* read the fs mod time off of it. We don't set IN_UPDATE here,
* since we don't really need this to be flushed to disk (and in any
* case that wouldn't happen to the Ifile until we checkpoint).
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_imtime(struct lfs *fs)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
{
struct timespec ts;
struct inode *ip;
TIMEVAL_TO_TIMESPEC(&time, &ts);
ip = VTOI(fs->lfs_ivnode);
ip->i_ffs_mtime = ts.tv_sec;
ip->i_ffs_mtimensec = ts.tv_nsec;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Ifile and meta data blocks are not marked busy, so segment writes MUST be
* single threaded. Currently, there are two paths into lfs_segwrite, sync()
* and getnewbuf(). They both mark the file system busy. Lfs_vflush()
* explicitly marks the file system busy. So lfs_segwrite is safe. I think.
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#define SET_FLUSHING(fs,vp) (fs)->lfs_flushvp = (vp)
#define IS_FLUSHING(fs,vp) ((fs)->lfs_flushvp == (vp))
#define CLR_FLUSHING(fs,vp) (fs)->lfs_flushvp = NULL
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_vflush(struct vnode *vp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct inode *ip;
struct lfs *fs;
struct segment *sp;
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
struct buf *bp, *nbp, *tbp, *tnbp;
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
int error, s;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
int flushed;
#if 0
int redo;
#endif
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
Move variable initialization to the top of lfs_vflush
1999-03-30 20:03:16 +04:00
ip = VTOI(vp);
fs = VFSTOUFS(vp->v_mount)->um_lfs;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ip->i_flag & IN_CLEANING) {
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
#ifdef DEBUG_LFS
ivndebug(vp,"vflush/in_cleaning");
#endif
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
LFS_CLR_UINO(ip, IN_CLEANING);
LFS_SET_UINO(ip, IN_MODIFIED);
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
/*
* Toss any cleaning buffers that have real counterparts
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* to avoid losing new data.
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
*/
s = splbio();
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
for (bp = LIST_FIRST(&vp->v_dirtyblkhd); bp; bp = nbp) {
nbp = LIST_NEXT(bp, b_vnbufs);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (!LFS_IS_MALLOC_BUF(bp))
continue;
/*
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
* Look for pages matching the range covered
* by cleaning blocks. It's okay if more dirty
* pages appear, so long as none disappear out
* from under us.
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
*/
if (bp->b_lblkno > 0 && vp->v_type == VREG &&
vp != fs->lfs_ivnode) {
struct vm_page *pg;
voff_t off;
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
simple_lock(&vp->v_interlock);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
for (off = lblktosize(fs, bp->b_lblkno);
off < lblktosize(fs, bp->b_lblkno + 1);
off += PAGE_SIZE) {
pg = uvm_pagelookup(&vp->v_uobj, off);
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
if (pg == NULL)
continue;
if ((pg->flags & PG_CLEAN) == 0 ||
pmap_is_modified(pg)) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
fs->lfs_avail += btofsb(fs,
bp->b_bcount);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
wakeup(&fs->lfs_avail);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_freebuf(fs, bp);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
bp = NULL;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
goto nextbp;
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
}
}
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
simple_unlock(&vp->v_interlock);
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
for (tbp = LIST_FIRST(&vp->v_dirtyblkhd); tbp;
tbp = tnbp)
{
tnbp = LIST_NEXT(tbp, b_vnbufs);
if (tbp->b_vp == bp->b_vp
&& tbp->b_lblkno == bp->b_lblkno
&& tbp != bp)
{
fs->lfs_avail += btofsb(fs,
bp->b_bcount);
wakeup(&fs->lfs_avail);
lfs_freebuf(fs, bp);
bp = NULL;
break;
}
}
nextbp:
SO C requires a statement after a label.
2003-03-15 05:27:18 +03:00
;
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
}
splx(s);
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
}
/* If the node is being written, wait until that is done */
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
s = splbio();
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (WRITEINPROG(vp)) {
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
#ifdef DEBUG_LFS
ivndebug(vp,"vflush/writeinprog");
#endif
tsleep(vp, PRIBIO+1, "lfs_vw", 0);
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
splx(s);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/* Protect against VXLOCK deadlock in vinvalbuf() */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_seglock(fs, SEGM_SYNC);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
/* If we're supposed to flush a freed inode, just toss it */
/* XXX - seglock, so these buffers can't be gathered, right? */
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ip->i_ffs_mode == 0) {
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
printf("lfs_vflush: ino %d is freed, not flushing\n",
ip->i_number);
s = splbio();
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
for (bp = LIST_FIRST(&vp->v_dirtyblkhd); bp; bp = nbp) {
nbp = LIST_NEXT(bp, b_vnbufs);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
if (bp->b_flags & B_DELWRI) { /* XXX always true? */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_avail += btofsb(fs, bp->b_bcount);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
wakeup(&fs->lfs_avail);
}
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
/* Copied from lfs_writeseg */
if (bp->b_flags & B_CALL) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
biodone(bp);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
} else {
bremfree(bp);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
LFS_UNLOCK_BUF(bp);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
bp->b_flags &= ~(B_ERROR | B_READ | B_DELWRI |
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
B_GATHERED);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
bp->b_flags |= B_DONE;
reassignbuf(bp, vp);
brelse(bp);
}
}
splx(s);
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
LFS_CLR_UINO(ip, IN_CLEANING);
LFS_CLR_UINO(ip, IN_MODIFIED | IN_ACCESSED);
update for IN_ACCESSED changes
2000-05-31 05:40:01 +04:00
ip->i_flag &= ~IN_ALLMOD;
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
printf("lfs_vflush: done not flushing ino %d\n",
ip->i_number);
lfs_segunlock(fs);
return 0;
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
SET_FLUSHING(fs,vp);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
if (fs->lfs_nactive > LFS_MAX_ACTIVE ||
(fs->lfs_sp->seg_flags & SEGM_CKP)) {
error = lfs_segwrite(vp->v_mount, SEGM_CKP | SEGM_SYNC);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
CLR_FLUSHING(fs,vp);
lfs_segunlock(fs);
return error;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp = fs->lfs_sp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
flushed = 0;
if (VPISEMPTY(vp)) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_writevnodes(fs, vp->v_mount, sp, VN_EMPTY);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
++flushed;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
} else if ((ip->i_flag & IN_CLEANING) &&
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
(fs->lfs_sp->seg_flags & SEGM_CLEAN)) {
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
#ifdef DEBUG_LFS
ivndebug(vp,"vflush/clean");
#endif
lfs_writevnodes(fs, vp->v_mount, sp, VN_CLEAN);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
++flushed;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
} else if (lfs_dostats) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (!VPISEMPTY(vp) || (VTOI(vp)->i_flag & IN_ALLMOD))
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.vflush_invoked;
#ifdef DEBUG_LFS
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
ivndebug(vp,"vflush");
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#endif
}
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
#ifdef DIAGNOSTIC
lfs_truncate calls vinvalbuf to invalidate all currently-hald buffers, which in turn forces a flush of the vnode, whether or not it is involved in a dirop. (This can happen during a remove or rmdir, when the directory is shrunk.) Because of the nature of dirops, however, flushing a vnode involved in a dirop is disallowed (and was marked with a panic). This patch has lfs_truncate call a specialized vinvalbuf that only invalidates buffers following the new end-of-file, and thus does not require a flush. Also the panic is demoted, in case I missed any other path to lfs_vflush.
1999-03-30 01:51:38 +04:00
/* XXX KS This actually can happen right now, though it shouldn't(?) */
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_flag & VDIROP) {
lfs_truncate calls vinvalbuf to invalidate all currently-hald buffers, which in turn forces a flush of the vnode, whether or not it is involved in a dirop. (This can happen during a remove or rmdir, when the directory is shrunk.) Because of the nature of dirops, however, flushing a vnode involved in a dirop is disallowed (and was marked with a panic). This patch has lfs_truncate call a specialized vinvalbuf that only invalidates buffers following the new end-of-file, and thus does not require a flush. Also the panic is demoted, in case I missed any other path to lfs_vflush.
1999-03-30 01:51:38 +04:00
printf("lfs_vflush: flushing VDIROP, this shouldn\'t be\n");
/* panic("VDIROP being flushed...this can\'t happen"); */
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_usecount < 0) {
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
printf("usecount=%ld\n", (long)vp->v_usecount);
Fixes to make dirops and lfs_vflush play together well. In particular, if we are short on vnodes, lfs_vflush from another process can grab a vnode that lfs_markv has already processed but not yet written; but lfs_markv holds the seglock. When lfs_vflush gets around to writing it, the context for copyin is gone. So, now lfs_markv calls copyin itself, rather than having lfs_writeseg do it.
1999-03-26 01:26:52 +03:00
panic("lfs_vflush: usecount<0");
}
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
#if 1
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
do {
do {
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
if (LIST_FIRST(&vp->v_dirtyblkhd) != NULL)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_writefile(fs, sp, vp);
} while (lfs_writeinode(fs, sp, ip));
} while (lfs_writeseg(fs, sp) && ip->i_number == LFS_IFILE_INUM);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#else
if (flushed && vp != fs->lfs_ivnode)
lfs_writeseg(fs, sp);
else do {
fs->lfs_flags &= ~LFS_IFDIRTY;
lfs_writefile(fs, sp, vp);
redo = lfs_writeinode(fs, sp, ip);
redo += lfs_writeseg(fs, sp);
redo += (fs->lfs_flags & LFS_IFDIRTY);
} while (redo && vp == fs->lfs_ivnode);
#endif
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (lfs_dostats) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.nwrites;
if (sp->seg_flags & SEGM_SYNC)
++lfs_stats.nsync_writes;
if (sp->seg_flags & SEGM_CKP)
++lfs_stats.ncheckpoints;
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/*
* If we were called from somewhere that has already held the seglock
* (e.g., lfs_markv()), the lfs_segunlock will not wait for
* the write to complete because we are still locked.
* Since lfs_vflush() must return the vnode with no dirty buffers,
* we must explicitly wait, if that is the case.
*
* We compare the iocount against 1, not 0, because it is
* artificially incremented by lfs_seglock().
*/
Add simple_lock protection for lfs_seglock and lfs_subsys_pages; these will be expanded to cover other per-fs and subsystem-wide data as well. Fix a case of IN_MODIFIED being set without updating lfs_uinodes, resulting in a "lfs_uinodes < 0" panic. Fix a deadlock in lfs_putpages arising from the need to busy all pages in a block; unbusy any that had already been busied before starting over.
2003-03-15 09:58:49 +03:00
simple_lock(&fs->lfs_interlock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if (fs->lfs_seglock > 1) {
Add simple_lock protection for lfs_seglock and lfs_subsys_pages; these will be expanded to cover other per-fs and subsystem-wide data as well. Fix a case of IN_MODIFIED being set without updating lfs_uinodes, resulting in a "lfs_uinodes < 0" panic. Fix a deadlock in lfs_putpages arising from the need to busy all pages in a block; unbusy any that had already been busied before starting over.
2003-03-15 09:58:49 +03:00
simple_unlock(&fs->lfs_interlock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
while (fs->lfs_iocount > 1)
(void)tsleep(&fs->lfs_iocount, PRIBIO + 1,
"lfs_vflush", 0);
Add simple_lock protection for lfs_seglock and lfs_subsys_pages; these will be expanded to cover other per-fs and subsystem-wide data as well. Fix a case of IN_MODIFIED being set without updating lfs_uinodes, resulting in a "lfs_uinodes < 0" panic. Fix a deadlock in lfs_putpages arising from the need to busy all pages in a block; unbusy any that had already been busied before starting over.
2003-03-15 09:58:49 +03:00
} else
simple_unlock(&fs->lfs_interlock);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_segunlock(fs);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
CLR_FLUSHING(fs,vp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
return (0);
}
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
#ifdef DEBUG_LFS_VERBOSE
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
# define vndebug(vp,str) if (VTOI(vp)->i_flag & IN_CLEANING) printf("not writing ino %d because %s (op %d)\n",VTOI(vp)->i_number,(str),op)
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
#else
# define vndebug(vp,str)
#endif
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writevnodes(struct lfs *fs, struct mount *mp, struct segment *sp, int op)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct inode *ip;
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
struct vnode *vp, *nvp;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
int inodes_written = 0, only_cleaning;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#ifndef LFS_NO_BACKVP_HACK
/* BEGIN HACK */
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
#define VN_OFFSET (((caddr_t)&LIST_NEXT(vp, v_mntvnodes)) - (caddr_t)vp)
#define BACK_VP(VP) ((struct vnode *)(((caddr_t)(VP)->v_mntvnodes.le_prev) - VN_OFFSET))
#define BEG_OF_VLIST ((struct vnode *)(((caddr_t)&(LIST_FIRST(&mp->mnt_vnodelist))) - VN_OFFSET))
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/* Find last vnode. */
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
loop: for (vp = LIST_FIRST(&mp->mnt_vnodelist);
vp && LIST_NEXT(vp, v_mntvnodes) != NULL;
vp = LIST_NEXT(vp, v_mntvnodes));
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
for (; vp && vp != BEG_OF_VLIST; vp = nvp) {
nvp = BACK_VP(vp);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#else
loop:
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
for (vp = LIST_FIRST(&mp->mnt_vnodelist); vp; vp = nvp) {
nvp = LIST_NEXT(vp, v_mntvnodes);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If the vnode that we are about to sync is no longer
* associated with this mount point, start over.
*/
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
if (vp->v_mount != mp) {
printf("lfs_writevnodes: starting over\n");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
goto loop;
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
}
in lfs_writefile, check v_type==VNON earlier. to avoid null dereference with DEBUG_LFS_VERBOSE.
2002-12-14 14:54:47 +03:00
if (vp->v_type == VNON) {
continue;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
ip = VTOI(vp);
if ((op == VN_DIROP && !(vp->v_flag & VDIROP)) ||
(op != VN_DIROP && op != VN_CLEAN && (vp->v_flag & VDIROP))) {
vndebug(vp,"dirop");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (op == VN_EMPTY && !VPISEMPTY(vp)) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
vndebug(vp,"empty");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (op == VN_CLEAN && ip->i_number != LFS_IFILE_INUM
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
&& vp != fs->lfs_flushvp
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
&& !(ip->i_flag & IN_CLEANING)) {
vndebug(vp,"cleaning");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
if (lfs_vref(vp)) {
vndebug(vp,"vref");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Add initialization to quell compiler warning (only on some platforms?)
1999-03-30 20:11:43 +04:00
only_cleaning = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
* Write the inode/file if dirty and it's not the IFILE.
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if ((ip->i_flag & IN_ALLMOD) || !VPISEMPTY(vp)) {
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
only_cleaning = ((ip->i_flag & IN_ALLMOD) == IN_CLEANING);
Make sysctl variable lfs_clean_vnhead do what it was supposed to do, namely, toggle whether vnodes loaded only for cleaning (as opposed to normal filesystem use) are freed to the *head* of the vnode free list, rather than the tail. This should avoid a possible cache flushing effect, if the cleaner cleans a segment containing a large number of live inodes.
1999-03-26 01:38:28 +03:00
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (ip->i_number != LFS_IFILE_INUM)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_writefile(fs, sp, vp);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (!VPISEMPTY(vp)) {
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (WRITEINPROG(vp)) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#ifdef DEBUG_LFS
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
ivndebug(vp,"writevnodes/write2");
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#endif
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
} else if (!(ip->i_flag & IN_ALLMOD)) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#ifdef DEBUG_LFS
printf("<%d>",ip->i_number);
#endif
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
LFS_SET_UINO(ip, IN_MODIFIED);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
(void) lfs_writeinode(fs, sp, ip);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
inodes_written++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Fixes associated with filling an LFS: Change the space computation to appear to change the size of the *disk* rather than the *bytes used* when more segment summaries and inode blocks are written. Try to estimate the amount of space that these will take up when more files are written, so the disk size doesn't change too much. Regularize error returns from lfs_valloc, lfs_balloc, lfs_truncate: they now fail entirely, rather than succeeding half-way and leaving the fs in an inconsistent state. Rewrite lfs_truncate, mostly stealing from ffs_truncate. The old lfs_truncate had difficulty truncating a large file to a non-zero size (indirect blocks were not handled appropriately). Unmark VDIROP on fvp after ufs_remove, ufs_rmdir, so these can be reclaimed immediately: this vnode would not be written to disk again anyway if the removal succeeded, and if it failed, no directory operation occurred. ufs_makeinode and ufs_mkdir now remove IN_ADIROP on error.
2000-06-28 00:57:11 +04:00
if (lfs_clean_vnhead && only_cleaning)
Make sysctl variable lfs_clean_vnhead do what it was supposed to do, namely, toggle whether vnodes loaded only for cleaning (as opposed to normal filesystem use) are freed to the *head* of the vnode free list, rather than the tail. This should avoid a possible cache flushing effect, if the cleaner cleans a segment containing a large number of live inodes.
1999-03-26 01:38:28 +03:00
lfs_vunref_head(vp);
else
lfs_vunref(vp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
return inodes_written;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
/*
* Do a checkpoint.
*/
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_segwrite(struct mount *mp, int flags)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct buf *bp;
struct inode *ip;
struct lfs *fs;
struct segment *sp;
struct vnode *vp;
SEGUSE *segusep;
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t ibno;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
int do_ckp, did_ckp, error, i;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int writer_set = 0;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
int dirty;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
int redo;
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
int sn;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
fs = VFSTOUFS(mp)->um_lfs;
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
if (fs->lfs_ronly)
return EROFS;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
lfs_imtime(fs);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Allocate a segment structure and enough space to hold pointers to
* the maximum possible number of buffers which can be described in a
* single summary block.
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
do_ckp = (flags & SEGM_CKP) || fs->lfs_nactive > LFS_MAX_ACTIVE;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_seglock(fs, flags | (do_ckp ? SEGM_CKP : 0));
sp = fs->lfs_sp;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
* If lfs_flushvp is non-NULL, we are called from lfs_vflush,
* in which case we have to flush *all* buffers off of this vnode.
Handle the case of a vnode flush while dirops are active correctly in lfs_segwrite. Also, make sure a flush is called in SET_DIROP before sleeping on its results. Addresses PR #8863.
1999-12-04 00:47:44 +03:00
* We don't care about other nodes, but write any non-dirop nodes
* anyway in anticipation of another getnewvnode().
*
* If we're cleaning we only write cleaning and ifile blocks, and
* no dirops, since otherwise we'd risk corruption in a crash.
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
*/
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (sp->seg_flags & SEGM_CLEAN)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
lfs_writevnodes(fs, mp, sp, VN_CLEAN);
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
else if (!(sp->seg_flags & SEGM_FORCE_CKP)) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
lfs_writevnodes(fs, mp, sp, VN_REG);
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (!fs->lfs_dirops || !fs->lfs_flushvp) {
while (fs->lfs_dirops)
if ((error = tsleep(&fs->lfs_writer, PRIBIO + 1,
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
"lfs writer", 0)))
{
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
printf("segwrite mysterious error\n");
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
/* XXX why not segunlock? */
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
pool_put(&fs->lfs_bpppool, sp->bpp);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->bpp = NULL;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
pool_put(&fs->lfs_segpool, sp);
sp = fs->lfs_sp = NULL;
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
return (error);
}
fs->lfs_writer++;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
writer_set = 1;
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
lfs_writevnodes(fs, mp, sp, VN_DIROP);
((SEGSUM *)(sp->segsum))->ss_flags &= ~(SS_CONT);
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If we are doing a checkpoint, mark everything since the
* last checkpoint as no longer ACTIVE.
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
if (do_ckp) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
for (ibno = fs->lfs_cleansz + fs->lfs_segtabsz;
--ibno >= fs->lfs_cleansz; ) {
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
dirty = 0;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
if (bread(fs->lfs_ivnode, ibno, fs->lfs_bsize, NOCRED, &bp))
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
panic("lfs_segwrite: ifile read");
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
segusep = (SEGUSE *)bp->b_data;
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
for (i = fs->lfs_sepb; i > 0; i--) {
sn = (ibno - fs->lfs_cleansz) * fs->lfs_sepb +
fs->lfs_sepb - i;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
if (segusep->su_flags & SEGUSE_ACTIVE) {
segusep->su_flags &= ~SEGUSE_ACTIVE;
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
--fs->lfs_nactive;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
++dirty;
}
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
fs->lfs_suflags[fs->lfs_activesb][sn] =
segusep->su_flags;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (fs->lfs_version > 1)
++segusep;
else
segusep = (SEGUSE *)
((SEGUSE_V1 *)segusep + 1);
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/* But the current segment is still ACTIVE */
From John Evans <jevans@cray.com>: use datosn() to convert to segment number, when remarking the current segment ACTIVE. See PR #10463.
2000-06-28 00:00:03 +04:00
segusep = (SEGUSE *)bp->b_data;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (dtosn(fs, fs->lfs_curseg) / fs->lfs_sepb ==
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
(ibno-fs->lfs_cleansz)) {
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
sn = dtosn(fs, fs->lfs_curseg);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (fs->lfs_version > 1)
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
segusep[sn % fs->lfs_sepb].su_flags |=
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
SEGUSE_ACTIVE;
else
((SEGUSE *)
((SEGUSE_V1 *)(bp->b_data) +
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
(sn % fs->lfs_sepb)))->su_flags
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
|= SEGUSE_ACTIVE;
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
fs->lfs_suflags[fs->lfs_activesb][sn] |=
SEGUSE_ACTIVE;
++fs->lfs_nactive;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
--dirty;
}
if (dirty)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
error = LFS_BWRITE_LOG(bp); /* Ifile */
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
else
brelse(bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
did_ckp = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (do_ckp || fs->lfs_doifile) {
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
do {
vp = fs->lfs_ivnode;
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#ifdef DEBUG
LFS_ENTER_LOG("pretend", __FILE__, __LINE__, 0, 0);
#endif
fs->lfs_flags &= ~LFS_IFDIRTY;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
ip = VTOI(vp);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
write ifile only when it has dirty buffers.
2002-12-31 17:54:32 +03:00
if (LIST_FIRST(&vp->v_dirtyblkhd) != NULL)
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
lfs_writefile(fs, sp, vp);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
if (ip->i_flag & IN_ALLMOD)
++did_ckp;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
redo = lfs_writeinode(fs, sp, ip);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
redo += lfs_writeseg(fs, sp);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
redo += (fs->lfs_flags & LFS_IFDIRTY);
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
} while (redo && do_ckp);
/*
* Unless we are unmounting, the Ifile may continue to have
* dirty blocks even after a checkpoint, due to changes to
* inodes' atime. If we're checkpointing, it's "impossible"
* for other parts of the Ifile to be dirty after the loop
* above, since we hold the segment lock.
*/
if (LIST_EMPTY(&vp->v_dirtyblkhd)) {
LFS_CLR_UINO(ip, IN_ALLMOD);
}
#ifdef DIAGNOSTIC
else if (do_ckp) {
LIST_FOREACH(bp, &vp->v_dirtyblkhd, b_vnbufs) {
if (bp->b_lblkno < fs->lfs_cleansz +
fs->lfs_segtabsz &&
!(bp->b_flags & B_GATHERED)) {
panic("dirty blocks");
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
}
}
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
#endif
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
} else {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
(void) lfs_writeseg(fs, sp);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Don't break out of Ifile-writing loop in lfs_segwrite until nothing is left. Note however that blocks can be added to the Ifile even when the segment block is held because of inodes' atime. Do not panic with "dirty blocks" if these blocks are present.
2003-03-20 09:51:17 +03:00
/* Note Ifile no longer needs to be written */
fs->lfs_doifile = 0;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (writer_set && --fs->lfs_writer == 0)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
wakeup(&fs->lfs_dirops);
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
/*
* If we didn't write the Ifile, we didn't really do anything.
* That means that (1) there is a checkpoint on disk and (2)
* nothing has changed since it was written.
*
* Take the flags off of the segment so that lfs_segunlock
* doesn't have to write the superblock either.
*/
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
if (do_ckp && !did_ckp) {
sp->seg_flags &= ~SEGM_CKP;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
}
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (lfs_dostats) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.nwrites;
if (sp->seg_flags & SEGM_SYNC)
++lfs_stats.nsync_writes;
if (sp->seg_flags & SEGM_CKP)
++lfs_stats.ncheckpoints;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_segunlock(fs);
return (0);
}
/*
* Write the dirty blocks associated with a vnode.
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writefile(struct lfs *fs, struct segment *sp, struct vnode *vp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct buf *bp;
struct finfo *fip;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
struct inode *ip;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
IFILE *ifp;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
int i, frag;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
ip = VTOI(vp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (sp->seg_bytes_left < fs->lfs_bsize ||
sp->sum_bytes_left < sizeof(struct finfo))
(void) lfs_writeseg(fs, sp);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
sp->sum_bytes_left -= FINFOSIZE;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
++((SEGSUM *)(sp->segsum))->ss_nfinfo;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_flag & VDIROP)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
((SEGSUM *)(sp->segsum))->ss_flags |= (SS_DIROP|SS_CONT);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
fip = sp->fip;
fip->fi_nblocks = 0;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
fip->fi_ino = ip->i_number;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
LFS_IENTRY(ifp, fs, fip->fi_ino, bp);
fip->fi_version = ifp->if_version;
brelse(bp);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if (sp->seg_flags & SEGM_CLEAN) {
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
lfs_gather(fs, sp, vp, lfs_match_fake);
/*
* For a file being flushed, we need to write *all* blocks.
* This means writing the cleaning blocks first, and then
* immediately following with any non-cleaning blocks.
* The same is true of the Ifile since checkpoints assume
* that all valid Ifile blocks are written.
*/
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
if (IS_FLUSHING(fs,vp) || vp == fs->lfs_ivnode) {
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
lfs_gather(fs, sp, vp, lfs_match_data);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/*
* Don't call VOP_PUTPAGES: if we're flushing,
* we've already done it, and the Ifile doesn't
* use the page cache.
*/
}
} else {
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
lfs_gather(fs, sp, vp, lfs_match_data);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/*
* If we're flushing, we've already called VOP_PUTPAGES
* so don't do it again. Otherwise, we want to write
* everything we've got.
*/
if (!IS_FLUSHING(fs, vp)) {
acquire v_interlock before calling VOP_PUTPAGES.
2003-02-19 15:02:38 +03:00
simple_lock(&vp->v_interlock);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
VOP_PUTPAGES(vp, 0, 0,
PGO_CLEANIT | PGO_ALLPAGES | PGO_LOCKED |
PGO_BUSYFAIL);
}
}
Better handling of various combinations of cleaning, vnode flushing, and dirop writing. In particular, lfs_writevnodes now writes all buffers from a flushed vnode whether cleaning or not, and the same with the Ifile; and lfs_segwrite does not attempt to write data from other non-cleaning vnodes, even if a vnode is being flushed.
2000-01-15 00:38:46 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* It may not be necessary to write the meta-data blocks at this point,
* as the roll-forward recovery code should be able to reconstruct the
* list.
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
*
* We have to write them anyway, though, under two conditions: (1) the
* vnode is being flushed (for reuse by vinvalbuf); or (2) we are
* checkpointing.
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
*
* BUT if we are cleaning, we might have indirect blocks that refer to
* new blocks not being written yet, in addition to fragments being
* moved out of a cleaned segment. If that is the case, don't
* write the indirect blocks, or the finfo will have a small block
* in the middle of it!
* XXX in this case isn't the inode size wrong too?
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
frag = 0;
if (sp->seg_flags & SEGM_CLEAN) {
for (i = 0; i < NDADDR; i++)
if (ip->i_lfs_fragsize[i] > 0 &&
ip->i_lfs_fragsize[i] < fs->lfs_bsize)
++frag;
}
#ifdef DIAGNOSTIC
if (frag > 1)
panic("lfs_writefile: more than one fragment!");
#endif
if (IS_FLUSHING(fs, vp) ||
(frag == 0 && (lfs_writeindir || (sp->seg_flags & SEGM_CKP)))) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
lfs_gather(fs, sp, vp, lfs_match_indir);
lfs_gather(fs, sp, vp, lfs_match_dindir);
lfs_gather(fs, sp, vp, lfs_match_tindir);
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
fip = sp->fip;
if (fip->fi_nblocks != 0) {
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
sp->fip = (FINFO*)((caddr_t)fip + FINFOSIZE +
sizeof(int32_t) * (fip->fi_nblocks));
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->start_lbp = &sp->fip->fi_blocks[0];
} else {
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
sp->sum_bytes_left += FINFOSIZE;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
--((SEGSUM *)(sp->segsum))->ss_nfinfo;
}
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writeinode(struct lfs *fs, struct segment *sp, struct inode *ip)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct buf *bp, *ibp;
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
struct dinode *cdp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
IFILE *ifp;
SEGUSE *sup;
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t daddr;
int32_t *daddrp; /* XXX ondisk32 */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
ino_t ino;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
int error, i, ndx, fsb = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int redo_ifile = 0;
Add a set of generic file system operations that most file systems use. Also, fix some time stamp bogosities.
1996-09-02 03:47:48 +04:00
struct timespec ts;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
int gotblk = 0;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
update for IN_ACCESSED changes
2000-05-31 05:40:01 +04:00
if (!(ip->i_flag & IN_ALLMOD))
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (0);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Allocate a new inode block if necessary. */
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if ((ip->i_number != LFS_IFILE_INUM || sp->idp == NULL) && sp->ibp == NULL) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Allocate a new segment if necessary. */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (sp->seg_bytes_left < fs->lfs_ibsize ||
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
sp->sum_bytes_left < sizeof(int32_t))
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
(void) lfs_writeseg(fs, sp);
/* Get next inode block. */
daddr = fs->lfs_offset;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_offset += btofsb(fs, fs->lfs_ibsize);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->ibp = *sp->cbpp++ =
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
getblk(VTOI(fs->lfs_ivnode)->i_devvp, fsbtodb(fs, daddr),
fs->lfs_ibsize, 0, 0);
Fix "lfs_ifind: dinode xxx not found" panic. When inodes were freed, then immediately reloaded, their dinodes were located in an inode block which was not on disk at the advertized location, nor in the cache (although it would be flushed to disk next segment write). Fix this by using getblk() instead of lfs_newbuf() for inode blocks.
1999-04-12 04:04:21 +04:00
gotblk++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Zero out inode numbers */
for (i = 0; i < INOPB(fs); ++i)
((struct dinode *)sp->ibp->b_data)[i].di_inumber = 0;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
++sp->start_bpp;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_avail -= btofsb(fs, fs->lfs_ibsize);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Set remaining space counters. */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_bytes_left -= fs->lfs_ibsize;
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
sp->sum_bytes_left -= sizeof(int32_t);
ndx = fs->lfs_sumsize / sizeof(int32_t) -
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
sp->ninodes / INOPB(fs) - 1;
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
((int32_t *)(sp->segsum))[ndx] = daddr;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Update the inode times and copy the inode onto the inode page. */
TIMESPEC_TO_TIMEVAL => TIMEVAL_TO_TIMESPEC
1997-06-13 12:59:51 +04:00
TIMEVAL_TO_TIMESPEC(&time, &ts);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/* XXX kludge --- don't redirty the ifile just to put times on it */
if (ip->i_number != LFS_IFILE_INUM)
LFS_ITIMES(ip, &ts, &ts, &ts);
clean up unused/required #ifdefs
1999-03-26 00:39:18 +03:00
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
/*
* If this is the Ifile, and we've already written the Ifile in this
* partial segment, just overwrite it (it's not on disk yet) and
* continue.
*
* XXX we know that the bp that we get the second time around has
* already been gathered.
*/
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ip->i_number == LFS_IFILE_INUM && sp->idp) {
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
*(sp->idp) = ip->i_din.ffs_din;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
ip->i_lfs_osize = ip->i_ffs_size;
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
return 0;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
bp = sp->ibp;
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
cdp = ((struct dinode *)bp->b_data) + (sp->ninodes % INOPB(fs));
*cdp = ip->i_din.ffs_din;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
#ifdef LFS_IFILE_FRAG_ADDRESSING
if (fs->lfs_version > 1)
fsb = (sp->ninodes % INOPB(fs)) / INOPF(fs);
#endif
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
/*
* If we are cleaning, ensure that we don't write UNWRITTEN disk
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
* addresses to disk; possibly revert the inode size.
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* XXX By not writing these blocks, we are making the lfs_avail
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* XXX count on disk wrong by the same amount. We should be
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* XXX able to "borrow" from lfs_avail and return it after the
* XXX Ifile is written. See also in lfs_writeseg.
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
*/
if (ip->i_lfs_effnblks != ip->i_ffs_blocks) {
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
cdp->di_size = ip->i_lfs_osize;
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
#ifdef DEBUG_LFS
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
printf("lfs_writeinode: cleansing ino %d (%d != %d)\n",
ip->i_number, ip->i_lfs_effnblks, ip->i_ffs_blocks);
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
#endif
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
for (daddrp = cdp->di_db; daddrp < cdp->di_ib + NIADDR;
daddrp++) {
if (*daddrp == UNWRITTEN) {
i_lfs_effnblks fixes. Put debugging printfs under #ifdef DEBUG_LFS.
2000-07-03 12:20:58 +04:00
#ifdef DEBUG_LFS
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
printf("lfs_writeinode: wiping UNWRITTEN\n");
#endif
*daddrp = 0;
}
}
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
} else {
/* If all blocks are goig to disk, update the "size on disk" */
ip->i_lfs_osize = ip->i_ffs_size;
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
}
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ip->i_flag & IN_CLEANING)
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
LFS_CLR_UINO(ip, IN_CLEANING);
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
else {
Clean up accounting of lfs_uinodes (dirty but unwritten inodes). Make lfs_uinodes a signed quantity for debugging purposes, and set it to zero as fs mount time. Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED, IN_CLEANING) in macros, and use those macros everywhere. Make LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-06 02:25:43 +04:00
/* XXX IN_ALLMOD */
LFS_CLR_UINO(ip, IN_ACCESSED | IN_ACCESS | IN_CHANGE |
IN_UPDATE);
if (ip->i_lfs_effnblks == ip->i_ffs_blocks)
LFS_CLR_UINO(ip, IN_MODIFIED);
If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint on mount, through the newer checkpoint and on through any newer partial-segments that may have been written but not checkpointed because of an intervening crash. LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 06:33:57 +03:00
#ifdef DEBUG_LFS
else
printf("lfs_writeinode: ino %d: real blks=%d, "
"eff=%d\n", ip->i_number, ip->i_ffs_blocks,
ip->i_lfs_effnblks);
#endif
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
}
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ip->i_number == LFS_IFILE_INUM) /* We know sp->idp == NULL */
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
sp->idp = ((struct dinode *)bp->b_data) +
(sp->ninodes % INOPB(fs));
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (gotblk) {
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
LFS_LOCK_BUF(bp);
Fix "lfs_ifind: dinode xxx not found" panic. When inodes were freed, then immediately reloaded, their dinodes were located in an inode block which was not on disk at the advertized location, nor in the cache (although it would be flushed to disk next segment write). Fix this by using getblk() instead of lfs_newbuf() for inode blocks.
1999-04-12 04:04:21 +04:00
brelse(bp);
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Increment inode count in segment summary block. */
++((SEGSUM *)(sp->segsum))->ss_ninos;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* If this page is full, set flag to allocate a new page. */
if (++sp->ninodes % INOPB(fs) == 0)
sp->ibp = NULL;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If updating the ifile, update the super-block. Update the disk
* address and access times for this inode in the ifile.
*/
ino = ip->i_number;
if (ino == LFS_IFILE_INUM) {
daddr = fs->lfs_idaddr;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_idaddr = dbtofsb(fs, bp->b_blkno);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
} else {
LFS_IENTRY(ifp, fs, ino, ibp);
daddr = ifp->if_daddr;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
ifp->if_daddr = dbtofsb(fs, bp->b_blkno) + fsb;
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
#ifdef LFS_DEBUG_NEXTFREE
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (ino > 3 && ifp->if_nextfree) {
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
vprint("lfs_writeinode",ITOV(ip));
printf("lfs_writeinode: updating free ino %d\n",
ip->i_number);
}
#endif
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
error = LFS_BWRITE_LOG(ibp); /* Ifile */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Fix obsolete comments in lfs_writeinode since rev. 1.27. New comments are mostly from perseant, with my additions.
2000-11-12 05:13:51 +03:00
/*
* The inode's last address should not be in the current partial
* segment, except under exceptional circumstances (lfs_writevnodes
* had to start over, and in the meantime more blocks were written
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* to a vnode). Both inodes will be accounted to this segment
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
* in lfs_writeseg so we need to subtract the earlier version
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* here anyway. The segment count can temporarily dip below
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
* zero here; keep track of how many duplicates we have in
* "dupino" so we don't panic below.
Fix obsolete comments in lfs_writeinode since rev. 1.27. New comments are mostly from perseant, with my additions.
2000-11-12 05:13:51 +03:00
*/
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
if (daddr >= fs->lfs_lastpseg && daddr <= dbtofsb(fs, bp->b_blkno)) {
++sp->ndupino;
Protect inode free list with seglock, instead of separate lock, so that the head of the inode free list (on the superblock) always matches the rest of the free list (in the ifile). Protect lfs_fragextend with seglock, to prevent the segment byte count fudging from making its way to disk. Don't try to inactivate dirop vnodes that are still in the middle of their dirop (may address PR#10285).
2000-06-07 00:19:14 +04:00
printf("lfs_writeinode: last inode addr in current pseg "
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
"(ino %d daddr 0x%llx) ndupino=%d\n", ino,
(long long)daddr, sp->ndupino);
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
}
/*
* Account the inode: it no longer belongs to its former segment,
* though it will not belong to the new segment until that segment
* is actually written.
*/
Protect inode free list with seglock, instead of separate lock, so that the head of the inode free list (on the superblock) always matches the rest of the free list (in the ifile). Protect lfs_fragextend with seglock, to prevent the segment byte count fudging from making its way to disk. Don't try to inactivate dirop vnodes that are still in the middle of their dirop (may address PR#10285).
2000-06-07 00:19:14 +04:00
if (daddr != LFS_UNUSED_DADDR) {
correct DIAGNOSTIC code for duplicated inodes in a segment and su_nbytes.
2002-12-12 15:28:13 +03:00
u_int32_t oldsn = dtosn(fs, daddr);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#ifdef DIAGNOSTIC
correct DIAGNOSTIC code for duplicated inodes in a segment and su_nbytes.
2002-12-12 15:28:13 +03:00
int ndupino = (sp->seg_number == oldsn) ? sp->ndupino : 0;
#endif
LFS_SEGENTRY(sup, fs, oldsn, bp);
#ifdef DIAGNOSTIC
if (sup->su_nbytes + DINODE_SIZE * ndupino < DINODE_SIZE) {
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
printf("lfs_writeinode: negative bytes "
make these compilable with lfs debug options. (follow daddr_t change) XXX maybe segment number should be 64bit.
2003-01-28 02:17:56 +03:00
"(segment %" PRIu32 " short by %d, "
"oldsn=%" PRIu32 ", cursn=%" PRIu32
", daddr=%" PRId64 ", su_nbytes=%u, "
correct DIAGNOSTIC code for duplicated inodes in a segment and su_nbytes.
2002-12-12 15:28:13 +03:00
"ndupino=%d)\n",
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
dtosn(fs, daddr),
correct DIAGNOSTIC code for duplicated inodes in a segment and su_nbytes.
2002-12-12 15:28:13 +03:00
(int)DINODE_SIZE * (1 - sp->ndupino)
- sup->su_nbytes,
make these compilable with lfs debug options. (follow daddr_t change) XXX maybe segment number should be 64bit.
2003-01-28 02:17:56 +03:00
oldsn, sp->seg_number, daddr,
correct DIAGNOSTIC code for duplicated inodes in a segment and su_nbytes.
2002-12-12 15:28:13 +03:00
(unsigned int)sup->su_nbytes,
sp->ndupino);
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
panic("lfs_writeinode: negative bytes");
sup->su_nbytes = DINODE_SIZE;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
#endif
#ifdef DEBUG_SU_NBYTES
printf("seg %d -= %d for ino %d inode\n",
dtosn(fs, daddr), DINODE_SIZE, ino);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#endif
Use DINODE_SIZE rather than sizeof(struct dinode).
1998-10-23 04:32:35 +04:00
sup->su_nbytes -= DINODE_SIZE;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
redo_ifile =
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
(ino == LFS_IFILE_INUM && !(bp->b_flags & B_GATHERED));
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if (redo_ifile)
fs->lfs_flags |= LFS_IFDIRTY;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
LFS_WRITESEGENTRY(sup, fs, oldsn, bp); /* Ifile */
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
return (redo_ifile);
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_gatherblock(struct segment *sp, struct buf *bp, int *sptr)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct lfs *fs;
int version;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
int j, blksinblk;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If full, finish this segment. We may be doing I/O, so
* release and reacquire the splbio().
*/
#ifdef DIAGNOSTIC
if (sp->vp == NULL)
panic ("lfs_gatherblock: Null vp in segment");
#endif
fs = sp->fs;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
blksinblk = howmany(bp->b_bcount, fs->lfs_bsize);
if (sp->sum_bytes_left < sizeof(int32_t) * blksinblk ||
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
sp->seg_bytes_left < bp->b_bcount) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (sptr)
splx(*sptr);
lfs_updatemeta(sp);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
version = sp->fip->fi_version;
(void) lfs_writeseg(fs, sp);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->fip->fi_version = version;
sp->fip->fi_ino = VTOI(sp->vp)->i_number;
/* Add the current file to the segment summary. */
++((SEGSUM *)(sp->segsum))->ss_nfinfo;
don't use daddr_t for segment summary since it's an on-disk structure.
2003-01-29 16:14:33 +03:00
sp->sum_bytes_left -= FINFOSIZE;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (sptr)
*sptr = splbio();
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (1);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#ifdef DEBUG
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (bp->b_flags & B_GATHERED) {
Fix further printf format warnings for DEBUG, in the wake of daddr_t having changed.
2003-01-26 02:00:09 +03:00
printf("lfs_gatherblock: already gathered! Ino %d,"
" lbn %" PRId64 "\n",
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
sp->fip->fi_ino, bp->b_lblkno);
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (0);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Insert into the buffer list, update the FINFO block. */
bp->b_flags |= B_GATHERED;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
bp->b_flags &= ~B_DONE;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*sp->cbpp++ = bp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
for (j = 0; j < blksinblk; j++)
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
sp->fip->fi_blocks[sp->fip->fi_nblocks++] = bp->b_lblkno + j;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
sp->sum_bytes_left -= sizeof(int32_t) * blksinblk;
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
sp->seg_bytes_left -= bp->b_bcount;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (0);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_gather(struct lfs *fs, struct segment *sp, struct vnode *vp, int (*match)(struct lfs *, struct buf *))
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
struct buf *bp, *nbp;
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
int s, count = 0;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->vp = vp;
s = splbio();
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#ifndef LFS_NO_BACKBUF_HACK
/* This is a hack to see if ordering the blocks in LFS makes a difference. */
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
# define BUF_OFFSET (((caddr_t)&LIST_NEXT(bp, b_vnbufs)) - (caddr_t)bp)
# define BACK_BUF(BP) ((struct buf *)(((caddr_t)(BP)->b_vnbufs.le_prev) - BUF_OFFSET))
# define BEG_OF_LIST ((struct buf *)(((caddr_t)&LIST_FIRST(&vp->v_dirtyblkhd)) - BUF_OFFSET))
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
/* Find last buffer. */
use macros from <sys/queue.h>
2002-05-18 01:42:38 +04:00
loop: for (bp = LIST_FIRST(&vp->v_dirtyblkhd); bp && LIST_NEXT(bp, b_vnbufs) != NULL;
bp = LIST_NEXT(bp, b_vnbufs));
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
for (; bp && bp != BEG_OF_LIST; bp = nbp) {
nbp = BACK_BUF(bp);
#else /* LFS_NO_BACKBUF_HACK */
loop: for (bp = LIST_FIRST(&vp->v_dirtyblkhd); bp; bp = nbp) {
nbp = LIST_NEXT(bp, b_vnbufs);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#endif /* LFS_NO_BACKBUF_HACK */
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if ((bp->b_flags & (B_BUSY|B_GATHERED)) || !match(fs, bp)) {
#ifdef DEBUG_LFS
if (vp == fs->lfs_ivnode && (bp->b_flags & (B_BUSY|B_GATHERED)) == B_BUSY)
make these compilable with lfs debug options. (follow daddr_t change) XXX maybe segment number should be 64bit.
2003-01-28 02:17:56 +03:00
printf("(%" PRId64 ":%lx)", bp->b_lblkno, bp->b_flags);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
continue;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_type == VBLK) {
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
/* For block devices, just write the blocks. */
/* XXX Do we really need to even do this? */
#ifdef DEBUG_LFS
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (count == 0)
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
printf("BLK(");
printf(".");
#endif
/* Get the block before bwrite, so we don't corrupt the free list */
bp->b_flags |= B_BUSY;
bremfree(bp);
bwrite(bp);
} else {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#ifdef DIAGNOSTIC
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
# ifdef LFS_USE_B_INVAL
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if ((bp->b_flags & (B_CALL|B_INVAL)) == B_INVAL) {
Use PRId64 instead of hard coding "%lld" to fix build problems under LP64 ports.
2003-01-25 21:12:31 +03:00
printf("lfs_gather: lbn %" PRId64 " is "
"B_INVAL\n", bp->b_lblkno);
Change the way LFS does block accounting, from trying to infer from the buffer cache flags, to marking the inode and/or indirect blocks with a special disk address UNWRITTEN==-2 when a block is accounted for. (This address is never written to disk, but only used in-core. This is essentially the same method of block accounting as on the UBC branch, where the buffer headers don't exist.) Make sure that truncation is handled properly, especially in the case of holey files. Fixes PR#9994.
2000-05-06 00:59:20 +04:00
VOP_PRINT(bp->b_vp);
}
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
# endif /* LFS_USE_B_INVAL */
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
if (!(bp->b_flags & B_DELWRI))
panic("lfs_gather: bp not B_DELWRI");
if (!(bp->b_flags & B_LOCKED)) {
Use PRId64 instead of hard coding "%lld" to fix build problems under LP64 ports.
2003-01-25 21:12:31 +03:00
printf("lfs_gather: lbn %" PRId64 " blk "
"%" PRId64 " not B_LOCKED\n",
bp->b_lblkno,
dbtofsb(fs, bp->b_blkno));
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
VOP_PRINT(bp->b_vp);
panic("lfs_gather: bp not B_LOCKED");
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
#endif
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
if (lfs_gatherblock(sp, bp, &s)) {
goto loop;
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
count++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
splx(s);
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
#ifdef DEBUG_LFS
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_type == VBLK && count)
Make changes that will allow an LFS filesystem to be used as the root filesystem. In particular, - Fix mknod deadlock, described in PR 8172. - Enable lfs_mountroot. - Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly, by flushing that device rather than trying to add blocks to the device inode. This, in combination with lfs boot blocks, will allow operation of an all-lfs system.
1999-09-04 02:48:51 +04:00
printf(")\n");
#endif
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_updatemeta(sp);
sp->vp = NULL;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
return count;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#if DEBUG
# define DEBUG_OOFF(n) do { \
if (ooff == 0) { \
printf("lfs_updatemeta[%d]: warning: writing " \
Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning actually happens. Add a new fcntl call that will write the minimum necessary to checkpoint (i.e., for on-disk directory structure to be consistent, not including updates to file data) so that the cleaner can clean segments more quickly without sacrificing three-way commit for cleaning.
2003-03-02 07:34:30 +03:00
"ino %d lbn %" PRId64 " at 0x%" PRIx32 \
", was 0x0 (or %" PRId64 ")\n", \
(n), ip->i_number, lbn, ndaddr, daddr); \
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
} \
} while(0)
#else
# define DEBUG_OOFF(n)
#endif
/*
* Change the given block's address to ndaddr, finding its previous
* location using ufs_bmaparray().
*
* Account for this change in the segment table.
*/
void
lfs_update_single(struct lfs *fs, struct segment *sp, daddr_t lbn,
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
int32_t ndaddr, int size)
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
{
SEGUSE *sup;
struct buf *bp;
struct indir a[NIADDR + 2], *ap;
struct inode *ip;
struct vnode *vp;
daddr_t daddr, ooff;
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
int num, error;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
int bb, osize, obb;
vp = sp->vp;
ip = VTOI(vp);
error = ufs_bmaparray(vp, lbn, &daddr, a, &num, NULL);
if (error)
panic("lfs_updatemeta: ufs_bmaparray returned %d", error);
if (daddr > 0)
daddr = dbtofsb(fs, daddr);
bb = fragstofsb(fs, numfrags(fs, size));
switch (num) {
case 0:
ooff = ip->i_ffs_db[lbn];
DEBUG_OOFF(0);
if (ooff == UNWRITTEN)
ip->i_ffs_blocks += bb;
else {
/* possible fragment truncation or extension */
obb = btofsb(fs, ip->i_lfs_fragsize[lbn]);
ip->i_ffs_blocks += (bb - obb);
}
ip->i_ffs_db[lbn] = ndaddr;
break;
case 1:
ooff = ip->i_ffs_ib[a[0].in_off];
DEBUG_OOFF(1);
if (ooff == UNWRITTEN)
ip->i_ffs_blocks += bb;
ip->i_ffs_ib[a[0].in_off] = ndaddr;
break;
default:
ap = &a[num - 1];
if (bread(vp, ap->in_lbn, fs->lfs_bsize, NOCRED, &bp))
panic("lfs_updatemeta: bread bno %" PRId64,
ap->in_lbn);
/* XXX ondisk32 */
ooff = ((int32_t *)bp->b_data)[ap->in_off];
DEBUG_OOFF(num);
if (ooff == UNWRITTEN)
ip->i_ffs_blocks += bb;
/* XXX ondisk32 */
((int32_t *)bp->b_data)[ap->in_off] = ndaddr;
(void) VOP_BWRITE(bp);
}
Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush. Note that pages can become dirty without our knowing it, anyway; don't panic if that happens.
2003-03-04 22:19:43 +03:00
/*
* Though we'd rather it couldn't, this *can* happen right now
* if cleaning blocks and regular blocks coexist.
*/
/* KASSERT(daddr < fs->lfs_lastpseg || daddr > ndaddr); */
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/*
* Update segment usage information, based on old size
* and location.
*/
if (daddr > 0) {
u_int32_t oldsn = dtosn(fs, daddr);
#ifdef DIAGNOSTIC
int ndupino = (sp->seg_number == oldsn) ?
sp->ndupino : 0;
#endif
if (lbn >= 0 && lbn < NDADDR)
osize = ip->i_lfs_fragsize[lbn];
else
osize = fs->lfs_bsize;
LFS_SEGENTRY(sup, fs, oldsn, bp);
#ifdef DIAGNOSTIC
if (sup->su_nbytes + DINODE_SIZE * ndupino < osize) {
printf("lfs_updatemeta: negative bytes "
"(segment %" PRIu32 " short by %" PRId64
")\n", dtosn(fs, daddr),
(int64_t)osize -
(DINODE_SIZE * sp->ndupino +
sup->su_nbytes));
printf("lfs_updatemeta: ino %d, lbn %" PRId64
", addr = 0x%" PRIx64 "\n",
VTOI(sp->vp)->i_number, lbn, daddr);
printf("lfs_updatemeta: ndupino=%d\n", ndupino);
panic("lfs_updatemeta: negative bytes");
sup->su_nbytes = osize - DINODE_SIZE * sp->ndupino;
}
#endif
#ifdef DEBUG_SU_NBYTES
printf("seg %" PRIu32 " -= %d for ino %d lbn %" PRId64
" db 0x%" PRIx64 "\n",
dtosn(fs, daddr), osize,
VTOI(sp->vp)->i_number, lbn, daddr);
#endif
sup->su_nbytes -= osize;
if (!(bp->b_flags & B_GATHERED))
fs->lfs_flags |= LFS_IFDIRTY;
LFS_WRITESEGENTRY(sup, fs, oldsn, bp);
}
/*
* Now that this block has a new address, and its old
* segment no longer owns it, we can forget about its
* old size.
*/
if (lbn >= 0 && lbn < NDADDR)
ip->i_lfs_fragsize[lbn] = size;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Update the metadata that points to the blocks listed in the FINFO
* array.
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_updatemeta(struct segment *sp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
struct buf *sbp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
struct lfs *fs;
struct vnode *vp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
daddr_t lbn;
int i, nblocks, num;
int bb;
int bytesleft, size;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
vp = sp->vp;
nblocks = &sp->fip->fi_blocks[sp->fip->fi_nblocks] - sp->start_lbp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
KASSERT(nblocks >= 0);
if (vp == NULL || nblocks == 0)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
return;
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
/*
* This count may be high due to oversize blocks from lfs_gop_write.
* Correct for this. (XXX we should be able to keep track of these.)
*/
fs = sp->fs;
for (i = 0; i < nblocks; i++) {
if (sp->start_bpp[i] == NULL) {
printf("nblocks = %d, not %d\n", i, nblocks);
nblocks = i;
break;
}
num = howmany(sp->start_bpp[i]->b_bcount, fs->lfs_bsize);
nblocks -= num - 1;
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* Sort the blocks.
*
* We have to sort even if the blocks come from the
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
* cleaner, because there might be other pending blocks on the
* same inode...and if we don't sort, and there are fragments
* present, blocks may be written in the wrong place.
*/
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
lfs_shellsort(sp->start_bpp, sp->start_lbp, nblocks, fs->lfs_bsize);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
/*
* Record the length of the last block in case it's a fragment.
* If there are indirect blocks present, they sort last. An
* indirect block will be lfs_bsize and its presence indicates
* that you cannot have fragments.
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
*
* XXX This last is a lie. A cleaned fragment can coexist with
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* XXX a later indirect block. This will continue to be
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
* XXX true until lfs_markv is fixed to do everything with
* XXX fake blocks (including fake inodes and fake indirect blocks).
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
sp->fip->fi_lastlength = ((sp->start_bpp[nblocks - 1]->b_bcount - 1) &
fs->lfs_bmask) + 1;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Assign disk addresses, and update references to the logical
* block and the segment usage information.
*/
for (i = nblocks; i--; ++sp->start_bpp) {
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
sbp = *sp->start_bpp;
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
lbn = *sp->start_lbp;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
sbp->b_blkno = fsbtodb(fs, fs->lfs_offset);
/*
* If we write a frag in the wrong place, the cleaner won't
* be able to correctly identify its size later, and the
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* segment will be uncleanable. (Even worse, it will assume
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
* that the indirect block that actually ends the list
* is of a smaller size!)
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if ((sbp->b_bcount & fs->lfs_bmask) && i != 0)
remove trailing \n in panic(). approved perry.
2002-09-27 19:35:29 +04:00
panic("lfs_updatemeta: fragment is not last block");
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
/*
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
* For each subblock in this possibly oversized block,
* update its address on disk.
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
KASSERT(lbn >= 0 || sbp->b_bcount == fs->lfs_bsize);
for (bytesleft = sbp->b_bcount; bytesleft > 0;
bytesleft -= fs->lfs_bsize) {
size = MIN(bytesleft, fs->lfs_bsize);
bb = fragstofsb(fs, numfrags(fs, size));
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
lbn = *sp->start_lbp++;
lfs_update_single(fs, sp, lbn, fs->lfs_offset, size);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
fs->lfs_offset += bb;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
}
/*
* Start a new segment.
*/
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_initseg(struct lfs *fs)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct segment *sp;
SEGUSE *sup;
SEGSUM *ssp;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
struct buf *bp, *sbp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int repeat;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp = fs->lfs_sp;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
repeat = 0;
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Advance to the next segment. */
if (!LFS_PARTIAL_FITS(fs)) {
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
/* lfs_avail eats the remaining space */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_avail -= fs->lfs_fsbpseg - (fs->lfs_offset -
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
fs->lfs_curseg);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Wake up any cleaning procs waiting on this file system. */
wakeup(&lfs_allclean_wakeup);
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
wakeup(&fs->lfs_nextseg);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lfs_newseg(fs);
repeat = 1;
fs->lfs_offset = fs->lfs_curseg;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_number = dtosn(fs, fs->lfs_curseg);
sp->seg_bytes_left = fsbtob(fs, fs->lfs_fsbpseg);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If the segment contains a superblock, update the offset
* and summary address to skip over it.
*/
LFS_SEGENTRY(sup, fs, sp->seg_number, bp);
if (sup->su_flags & SEGUSE_SUPERBLOCK) {
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_offset += btofsb(fs, LFS_SBPAD);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->seg_bytes_left -= LFS_SBPAD;
}
brelse(bp);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
/* Segment zero could also contain the labelpad */
if (fs->lfs_version > 1 && sp->seg_number == 0 &&
fs->lfs_start < btofsb(fs, LFS_LABELPAD)) {
fs->lfs_offset += btofsb(fs, LFS_LABELPAD) - fs->lfs_start;
sp->seg_bytes_left -= LFS_LABELPAD - fsbtob(fs, fs->lfs_start);
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
} else {
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_number = dtosn(fs, fs->lfs_curseg);
sp->seg_bytes_left = fsbtob(fs, fs->lfs_fsbpseg -
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
(fs->lfs_offset - fs->lfs_curseg));
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
fs->lfs_lastpseg = fs->lfs_offset;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
/* Record first address of this partial segment */
if (sp->seg_flags & SEGM_CLEAN) {
fs->lfs_cleanint[fs->lfs_cleanind] = fs->lfs_offset;
if (++fs->lfs_cleanind >= LFS_MAX_CLEANIND) {
/* "1" is the artificial inc in lfs_seglock */
while (fs->lfs_iocount > 1) {
tsleep(&fs->lfs_iocount, PRIBIO + 1, "lfs_initseg", 0);
}
fs->lfs_cleanind = 0;
}
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->fs = fs;
sp->ibp = NULL;
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
sp->idp = NULL;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->ninodes = 0;
Deal with fragment size changes better. For each fragment that can exist on an on-disk inode, we keep a record of its size in struct inode, which is updated when we write the block to disk. The cleaner routines thus have ready access to what size is the correct size for this block, on disk. Fixed a related bug: if a file with fragments is being cleaned (fragments being cleaned) at the same time it is being extended beyond NDADDR blocks, we could write a bogus FINFO record that has a frag in the middle; when it was cleaned this would give back bogus file data. Don't write the indirect blocks in this case, since there is no need. lfs_fragextend and lfs_truncate no longer require the seglock, but instead take a shared lock, which the seglock locks exclusively.
2002-07-06 05:30:11 +04:00
sp->ndupino = 0;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Get a new buffer for SEGSUM and enter it into the buffer list. */
sp->cbpp = sp->bpp;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
sbp = *sp->cbpp = lfs_newbuf(fs, VTOI(fs->lfs_ivnode)->i_devvp,
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
fsbtodb(fs, fs->lfs_offset), fs->lfs_sumsize, LFS_NB_SUMMARY);
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
sp->segsum = (*sp->cbpp)->b_data;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
memset(sp->segsum, 0, fs->lfs_sumsize);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->start_bpp = ++sp->cbpp;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_offset += btofsb(fs, fs->lfs_sumsize);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Set point to SEGSUM, initialize it. */
ssp = sp->segsum;
ssp->ss_next = fs->lfs_nextseg;
ssp->ss_nfinfo = ssp->ss_ninos = 0;
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
ssp->ss_magic = SS_MAGIC;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Set pointer to first FINFO, initialize it. */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->fip = (struct finfo *)((caddr_t)sp->segsum + SEGSUM_SIZE(fs));
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sp->fip->fi_nblocks = 0;
sp->start_lbp = &sp->fip->fi_blocks[0];
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
sp->fip->fi_lastlength = 0;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_bytes_left -= fs->lfs_sumsize;
sp->sum_bytes_left = fs->lfs_sumsize - SEGSUM_SIZE(fs);
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (repeat);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
/*
* Return the next segment to write.
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_newseg(struct lfs *fs)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
CLEANERINFO *cip;
SEGUSE *sup;
struct buf *bp;
int curseg, isdirty, sn;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
LFS_SEGENTRY(sup, fs, dtosn(fs, fs->lfs_nextseg), bp);
#ifdef DEBUG_SU_NBYTES
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
printf("lfs_newseg: seg %d := 0 in newseg\n", /* XXXDEBUG */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
dtosn(fs, fs->lfs_nextseg)); /* XXXDEBUG */
#endif
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
sup->su_flags |= SEGUSE_DIRTY | SEGUSE_ACTIVE;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sup->su_nbytes = 0;
sup->su_nsums = 0;
sup->su_ninos = 0;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
LFS_WRITESEGENTRY(sup, fs, dtosn(fs, fs->lfs_nextseg), bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
LFS_CLEANERINFO(cip, fs, bp);
--cip->clean;
++cip->dirty;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
fs->lfs_nclean = cip->clean;
Do not needlessly dirty segment table blocks during lfs_segwrite, preventing needless disk activity when the filesystem is idle. (PR #10979.)
2000-11-12 10:58:36 +03:00
LFS_SYNC_CLEANERINFO(cip, fs, bp, 1);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
fs->lfs_lastseg = fs->lfs_curseg;
fs->lfs_curseg = fs->lfs_nextseg;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
for (sn = curseg = dtosn(fs, fs->lfs_curseg) + fs->lfs_interleave;;) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sn = (sn + 1) % fs->lfs_nseg;
if (sn == curseg)
panic("lfs_nextseg: no clean segments");
LFS_SEGENTRY(sup, fs, sn, bp);
isdirty = sup->su_flags & SEGUSE_DIRTY;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/* Check SEGUSE_EMPTY as we go along */
if (isdirty && sup->su_nbytes == 0 && !(sup->su_flags & SEGUSE_EMPTY))
LFS_WRITESEGENTRY(sup, fs, sn, bp);
else
brelse(bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (!isdirty)
break;
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
++fs->lfs_nactive;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_nextseg = sntod(fs, sn);
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (lfs_dostats) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.segsused;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#define BQUEUES 4 /* XXX */
#define BQ_EMPTY 3 /* XXX */
extern TAILQ_HEAD(bqueues, buf) bufqueues[BQUEUES];
Make the buffer cache code MP-safe.
2003-02-06 00:38:38 +03:00
extern struct simplelock bqueue_slock;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#define BUFHASH(dvp, lbn) \
(&bufhashtbl[((long)(dvp) / sizeof(*(dvp)) + (int)(lbn)) & bufhash])
extern LIST_HEAD(bufhashhdr, buf) invalhash;
/*
* Insq/Remq for the buffer hash lists.
*/
#define binshash(bp, dp) LIST_INSERT_HEAD(dp, bp, b_hash)
#define bremhash(bp) LIST_REMOVE(bp, b_hash)
static struct buf *
lfs_newclusterbuf(struct lfs *fs, struct vnode *vp, daddr_t addr, int n)
{
struct lfs_cluster *cl;
struct buf **bpp, *bp;
int s;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
cl = (struct lfs_cluster *)pool_get(&fs->lfs_clpool, PR_WAITOK);
bpp = (struct buf **)pool_get(&fs->lfs_bpppool, PR_WAITOK);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
memset(cl, 0, sizeof(*cl));
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
cl->fs = fs;
cl->bpp = bpp;
cl->bufcount = 0;
cl->bufsize = 0;
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
/* If this segment is being written synchronously, note that */
if (fs->lfs_sp->seg_flags & SEGM_SYNC) {
cl->flags |= LFS_CL_SYNC;
cl->seg = fs->lfs_sp;
++cl->seg->seg_iocount;
/* printf("+ %x => %d\n", cl->seg, cl->seg->seg_iocount); */
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/* Get an empty buffer header, or maybe one with something on it */
s = splbio();
Make the buffer cache code MP-safe.
2003-02-06 00:38:38 +03:00
simple_lock(&bqueue_slock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if((bp = bufqueues[BQ_EMPTY].tqh_first) != NULL) {
Make the buffer cache code MP-safe.
2003-02-06 00:38:38 +03:00
simple_lock(&bp->b_interlock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
bremfree(bp);
/* clear out various other fields */
bp->b_flags = B_BUSY;
bp->b_dev = NODEV;
bp->b_blkno = bp->b_lblkno = 0;
bp->b_error = 0;
bp->b_resid = 0;
bp->b_bcount = 0;
/* nuke any credentials we were holding */
/* XXXXXX */
bremhash(bp);
/* disassociate us from our vnode, if we had one... */
if (bp->b_vp)
brelvp(bp);
}
while (!bp)
bp = getnewbuf(0, 0);
bgetvp(vp, bp);
binshash(bp,&invalhash);
Make the buffer cache code MP-safe.
2003-02-06 00:38:38 +03:00
simple_unlock(&bp->b_interlock);
simple_unlock(&bqueue_slock);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
splx(s);
bp->b_bcount = 0;
bp->b_blkno = bp->b_lblkno = addr;
bp->b_flags |= B_CALL;
bp->b_iodone = lfs_cluster_callback;
cl->saveaddr = bp->b_saveaddr; /* XXX is this ever used? */
bp->b_saveaddr = (caddr_t)cl;
return bp;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writeseg(struct lfs *fs, struct segment *sp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
struct buf **bpp, *bp, *cbp, *newbp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
SEGUSE *sup;
SEGSUM *ssp;
dev_t i_dev;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
char *datap, *dp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
int i, s;
int do_again, nblocks, byteoffset;
lfs_writeseg(): make el_size a size_t (cosmetic only, no functional change)
2001-07-27 00:20:15 +04:00
size_t el_size;
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
struct lfs_cluster *cl;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
int (*strategy)(void *);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
struct vop_strategy_args vop_strategy_a;
u_short ninos;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
struct vnode *devvp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
char *p;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
struct vnode *vp;
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
int32_t *daddrp; /* XXX ondisk32 */
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
int changed;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#if defined(DEBUG) && defined(LFS_PROPELLER)
static int propeller;
char propstring[4] = "-\\|/";
printf("%c\b",propstring[propeller++]);
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (propeller == 4)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
propeller = 0;
#endif
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* If there are no buffers other than the segment summary to write
* and it is not a checkpoint, don't do anything. On a checkpoint,
* even if there aren't any buffers, you need to write the superblock.
*/
if ((nblocks = sp->cbpp - sp->bpp) == 1)
return (0);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
i_dev = VTOI(fs->lfs_ivnode)->i_dev;
devvp = VTOI(fs->lfs_ivnode)->i_devvp;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Update the segment usage information. */
LFS_SEGENTRY(sup, fs, sp->seg_number, bp);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
/* Loop through all blocks, except the segment summary. */
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
for (bpp = sp->bpp; ++bpp < sp->cbpp; ) {
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if ((*bpp)->b_vp != devvp) {
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
sup->su_nbytes += (*bpp)->b_bcount;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
#ifdef DEBUG_SU_NBYTES
make these compilable with lfs debug options. (follow daddr_t change) XXX maybe segment number should be 64bit.
2003-01-28 02:17:56 +03:00
printf("seg %" PRIu32 " += %ld for ino %d lbn %" PRId64
" db 0x%" PRIx64 "\n", sp->seg_number, (*bpp)->b_bcount,
VTOI((*bpp)->b_vp)->i_number, (*bpp)->b_lblkno,
(*bpp)->b_blkno);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
#endif
}
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
ssp = (SEGSUM *)sp->segsum;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
ninos = (ssp->ss_ninos + INOPB(fs) - 1) / INOPB(fs);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
#ifdef DEBUG_SU_NBYTES
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
printf("seg %d += %d for %d inodes\n", /* XXXDEBUG */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
sp->seg_number, ssp->ss_ninos * DINODE_SIZE,
ssp->ss_ninos);
#endif
Minor changes to the segment live bytes calculation. In particular, fixed a bug in fragment extension that could run the count negative. Also, don't overcount for inodes, and don't count segment summaries. Thus, for empty segments the live bytes count should now be exactly zero.
1999-06-16 02:25:41 +04:00
sup->su_nbytes += ssp->ss_ninos * DINODE_SIZE;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
/* sup->su_nbytes += fs->lfs_sumsize; */
if (fs->lfs_version == 1)
sup->su_olastmod = time.tv_sec;
else
sup->su_lastmod = time.tv_sec;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
sup->su_ninos += ninos;
++sup->su_nsums;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_dmeta += (btofsb(fs, fs->lfs_sumsize) + btofsb(fs, ninos *
fs->lfs_ibsize));
fs->lfs_avail -= btofsb(fs, fs->lfs_sumsize);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
do_again = !(bp->b_flags & B_GATHERED);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
LFS_WRITESEGENTRY(sup, fs, sp->seg_number, bp); /* Ifile */
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
/*
* Mark blocks B_BUSY, to prevent then from being changed between
* the checksum computation and the actual write.
*
* If we are cleaning, check indirect blocks for UNWRITTEN, and if
* there are any, replace them with copies that have UNASSIGNED
* instead.
*/
for (bpp = sp->bpp, i = nblocks - 1; i--;) {
++bpp;
bp = *bpp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (bp->b_flags & B_CALL) { /* UBC or malloced buffer */
bp->b_flags |= B_BUSY;
continue;
}
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
again:
s = splbio();
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (bp->b_flags & B_BUSY) {
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
#ifdef DEBUG
Use PRId64 instead of hard coding "%lld" to fix build problems under LP64 ports.
2003-01-25 21:12:31 +03:00
printf("lfs_writeseg: avoiding potential data summary "
"corruption for ino %d, lbn %" PRId64 "\n",
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
VTOI(bp->b_vp)->i_number, bp->b_lblkno);
#endif
bp->b_flags |= B_WANTED;
tsleep(bp, (PRIBIO + 1), "lfs_writeseg", 0);
splx(s);
goto again;
}
bp->b_flags |= B_BUSY;
splx(s);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
/*
* Check and replace indirect block UNWRITTEN bogosity.
* XXX See comment in lfs_writefile.
*/
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (bp->b_lblkno < 0 && bp->b_vp != devvp && bp->b_vp &&
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
VTOI(bp->b_vp)->i_ffs_blocks !=
VTOI(bp->b_vp)->i_lfs_effnblks) {
i_lfs_effnblks fixes. Put debugging printfs under #ifdef DEBUG_LFS.
2000-07-03 12:20:58 +04:00
#ifdef DEBUG_LFS
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
printf("lfs_writeseg: cleansing ino %d (%d != %d)\n",
VTOI(bp->b_vp)->i_number,
VTOI(bp->b_vp)->i_lfs_effnblks,
VTOI(bp->b_vp)->i_ffs_blocks);
i_lfs_effnblks fixes. Put debugging printfs under #ifdef DEBUG_LFS.
2000-07-03 12:20:58 +04:00
#endif
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
/* Make a copy we'll make changes to */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
newbp = lfs_newbuf(fs, bp->b_vp, bp->b_lblkno,
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
bp->b_bcount, LFS_NB_IBLOCK);
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
newbp->b_blkno = bp->b_blkno;
memcpy(newbp->b_data, bp->b_data,
newbp->b_bcount);
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
changed = 0;
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
/* XXX ondisk32 */
for (daddrp = (int32_t *)(newbp->b_data);
daddrp < (int32_t *)(newbp->b_data +
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
newbp->b_bcount); daddrp++) {
if (*daddrp == UNWRITTEN) {
i_lfs_effnblks fixes. Put debugging printfs under #ifdef DEBUG_LFS.
2000-07-03 12:20:58 +04:00
#ifdef DEBUG_LFS
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
off_t doff;
int32_t ioff;
ioff = daddrp - (int32_t *)(newbp->b_data);
doff = (-bp->b_lblkno + ioff) * fs->lfs_bsize;
printf("ino %d lbn %" PRId64 " entry %d off %" PRIx64 "\n",
VTOI(bp->b_vp)->i_number,
bp->b_lblkno, ioff, doff);
if (bp->b_vp->v_type == VREG) {
/*
* What is up with this page?
*/
struct vm_page *pg;
for (; doff / fs->lfs_bsize == (-bp->b_lblkno + ioff); doff += PAGE_SIZE) {
pg = uvm_pagelookup(&bp->b_vp->v_uobj, doff);
if (pg == NULL)
printf(" page at %" PRIx64 " is NULL\n", doff);
else
printf(" page at %" PRIx64 " flags 0x%x pqflags 0x%x\n", doff, pg->flags, pg->pqflags);
}
}
#endif /* DEBUG_LFS */
++changed;
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
*daddrp = 0;
}
}
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
/*
* Get rid of the old buffer. Don't mark it clean,
* though, if it still has dirty data on it.
*/
if (changed) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#ifdef DEBUG_LFS
printf("lfs_writeseg: replacing UNWRITTEN(%d):"
" bp = %p newbp = %p\n", changed, bp,
newbp);
#endif
*bpp = newbp;
bp->b_flags &= ~(B_ERROR | B_GATHERED | B_DONE);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (bp->b_flags & B_CALL) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
printf("lfs_writeseg: indir bp should not be B_CALL\n");
s = splbio();
biodone(bp);
splx(s);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
bp = NULL;
} else {
Fix a buffer-cache corrupting bug in lfs_writeseg, where brelse could be improperly used on an already-queued buffer.
2000-09-09 08:13:43 +04:00
/* Still on free list, leave it there */
s = splbio();
bp->b_flags &= ~B_BUSY;
if (bp->b_flags & B_WANTED)
wakeup(bp);
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
splx(s);
Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes. (PR #11468). In the case of fragment allocation, check to see if enough space is available before extending a fragment already scheduled for writing. The locked_queue_* variables indicate the number of buffer headers and bytes, respectively, that are unavailable to getnewbuf() because they are locked up waiting for LFS to flush them; make sure that that is actually what we're counting, i.e., never count malloced buffers, and always use b_bufsize instead of b_bcount. If DEBUG is defined, the periodic calls to lfs_countlocked will now complain if either counter is incorrect. (In the future lfs_countlocked will not need to be called at all if DEBUG is not defined.)
2000-11-17 22:14:41 +03:00
/*
* We have to re-decrement lfs_avail
* since this block is going to come
* back around to us in the next
* segment.
*/
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
fs->lfs_avail -= btofsb(fs, bp->b_bcount);
Fix a buffer-cache corrupting bug in lfs_writeseg, where brelse could be improperly used on an already-queued buffer.
2000-09-09 08:13:43 +04:00
}
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
} else {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_freebuf(fs, newbp);
Fix errors observed while trying to fill the filesystem with yesterday's fixes: - Write copies of bfree and avail in the CLEANERINFO block, so the cleaner doesn't have to guess which superblock has the current information (if indeed any do). - Tighten up accounting of lfs_avail (more needs to be done). - When cleansing indirect blocks of UNWRITTEN, make sure not to mark them clean, since they'll need to be rewritten later.
2000-07-05 02:30:37 +04:00
}
Allow the number of free segments reserved for the cleaner to be parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set to something more reasonable at newfs_lfs time. Note the number of blocks that have been scheduled for writing but which are not yet on disk in an inode extension, i_lfs_effnblks. Move i_ffs_effnlink out of the ffs extension and onto the main inode, since it's used all over the shared code and the lfs extension would clobber it. At inode write time, indirect blocks and inode-held blocks of inodes that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk addresses, so that these never make it to disk.
2000-07-03 05:45:46 +04:00
}
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Compute checksum across data and then across summary; the first
* block (the summary block) is skipped. Set the create time here
* so that it's guaranteed to be later than the inode mod times.
*
* XXX
* Fix this to do it inline, instead of malloc/copy.
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
datap = dp = pool_get(&fs->lfs_bpppool, PR_WAITOK);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (fs->lfs_version == 1)
el_size = sizeof(u_long);
else
el_size = sizeof(u_int32_t);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
for (bpp = sp->bpp, i = nblocks - 1; i--; ) {
++bpp;
/* Loop through gop_write cluster blocks */
for (byteoffset = 0; byteoffset < (*bpp)->b_bcount;
byteoffset += fs->lfs_bsize) {
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
#ifdef LFS_USE_B_INVAL
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if (((*bpp)->b_flags & (B_CALL | B_INVAL)) ==
(B_CALL | B_INVAL)) {
if (copyin((caddr_t)(*bpp)->b_saveaddr +
byteoffset, dp, el_size)) {
panic("lfs_writeseg: copyin failed [1]: "
"ino %d blk %" PRId64,
VTOI((*bpp)->b_vp)->i_number,
(*bpp)->b_lblkno);
}
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
} else
#endif /* LFS_USE_B_INVAL */
{
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
memcpy(dp, (*bpp)->b_data + byteoffset,
el_size);
}
dp += el_size;
}
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
}
if (fs->lfs_version == 1)
ssp->ss_ocreate = time.tv_sec;
else {
ssp->ss_create = time.tv_sec;
ssp->ss_serial = ++fs->lfs_serial;
ssp->ss_ident = fs->lfs_ident;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
ssp->ss_datasum = cksum(datap, dp - datap);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
ssp->ss_sumsum =
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
cksum(&ssp->ss_datasum, fs->lfs_sumsize - sizeof(ssp->ss_sumsum));
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
pool_put(&fs->lfs_bpppool, datap);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
datap = dp = NULL;
#ifdef DIAGNOSTIC
if (fs->lfs_bfree < btofsb(fs, ninos * fs->lfs_ibsize) + btofsb(fs, fs->lfs_sumsize))
panic("lfs_writeseg: No diskspace for summary");
#endif
fs->lfs_bfree -= (btofsb(fs, ninos * fs->lfs_ibsize) +
btofsb(fs, fs->lfs_sumsize));
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
strategy = devvp->v_op[VOFFSET(vop_strategy)];
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* When we simply write the blocks we lose a rotation for every block
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
* written. To avoid this problem, we cluster the buffers into a
* chunk and write the chunk. MAXPHYS is the largest size I/O
* devices can handle, use that for the size of the chunks.
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
*
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
* Blocks that are already clusters (from GOP_WRITE), however, we
* don't bother to copy into other clusters.
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#define CHUNKSIZE MAXPHYS
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (devvp == NULL)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
panic("devvp is NULL");
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
for (bpp = sp->bpp, i = nblocks; i;) {
cbp = lfs_newclusterbuf(fs, devvp, (*bpp)->b_blkno, i);
cl = (struct lfs_cluster *)cbp->b_saveaddr;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
cbp->b_dev = i_dev;
cbp->b_flags |= B_ASYNC | B_BUSY;
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
cbp->b_bcount = 0;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
cl->olddata = cbp->b_data;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#if defined(DEBUG) && defined(DIAGNOSTIC)
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
if (bpp - sp->bpp > (fs->lfs_sumsize - SEGSUM_SIZE(fs))
/ sizeof(int32_t)) {
panic("lfs_writeseg: real bpp overwrite");
}
if (bpp - sp->bpp > fs->lfs_ssize / fs->lfs_fsize) {
panic("lfs_writeseg: theoretical bpp overwrite");
}
Change lfs_sb_cksum to use offsetof() instead of an inlined version. Fix lfs_vref/lfs_vunredf to ignore VXLOCKed vnodes that are also being flushed. Improve the debugging messages somewhat.
1999-03-26 00:54:10 +03:00
#endif
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
/*
* Construct the cluster.
*/
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
++fs->lfs_iocount;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
while(i && cbp->b_bcount < CHUNKSIZE) {
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
bp = *bpp;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
if (bp->b_bcount > (CHUNKSIZE - cbp->b_bcount))
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
break;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
if (cbp->b_bcount > 0 && !(cl->flags & LFS_CL_MALLOC))
break;
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
/* Clusters from GOP_WRITE are expedited */
if (bp->b_bcount > fs->lfs_bsize) {
if (cbp->b_bcount > 0)
/* Put in its own buffer */
break;
else {
cbp->b_data = bp->b_data;
}
} else if (cbp->b_bcount == 0) {
p = cbp->b_data = lfs_malloc(fs, CHUNKSIZE,
LFS_NB_CLUSTER);
cl->flags |= LFS_CL_MALLOC;
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#ifdef DIAGNOSTIC
if (dtosn(fs, dbtofsb(fs, bp->b_blkno +
btodb(bp->b_bcount - 1))) !=
sp->seg_number) {
printf("blk size %ld daddr %" PRIx64 " not in seg %d\n",
bp->b_bcount, bp->b_blkno,
sp->seg_number);
panic("segment overwrite");
}
#endif
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
#ifdef LFS_USE_B_INVAL
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Fake buffers from the cleaner are marked as B_INVAL.
* We need to copy the data from user space rather than
* from the buffer indicated.
* XXX == what do I do on an error?
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
if ((bp->b_flags & (B_CALL|B_INVAL)) == (B_CALL|B_INVAL)) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
if (copyin(bp->b_saveaddr, p, bp->b_bcount))
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
panic("lfs_writeseg: copyin failed [2]");
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
} else
#endif /* LFS_USE_B_INVAL */
if (cl->flags & LFS_CL_MALLOC) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
bcopy(bp->b_data, p, bp->b_bcount);
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
p += bp->b_bcount;
cbp->b_bcount += bp->b_bcount;
cl->bufsize += bp->b_bcount;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
bp->b_flags &= ~(B_ERROR | B_READ | B_DELWRI | B_DONE);
cl->bpp[cl->bufcount++] = bp;
vp = bp->b_vp;
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
s = splbio();
Make the buffer cache code MP-safe.
2003-02-06 00:38:38 +03:00
V_INCR_NUMOUTPUT(vp);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
splx(s);
Make sure that the wakeup occurs for vnodes that lfs_update might be sleeping on (nodes which are not marked IN_MODIFIED/IN_CLEANING, but which have dirty buffers), by marking them with the appropriate flag if dirtybuffers were added while the write was in progress.
1999-04-12 04:25:13 +04:00
bpp++;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
i--;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
s = splbio();
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
V_INCR_NUMOUTPUT(devvp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
splx(s);
vop_strategy_a.a_desc = VDESC(vop_strategy);
vop_strategy_a.a_bp = cbp;
(strategy)(&vop_strategy_a);
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
curproc->p_stats->p_ru.ru_oublock++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (lfs_dostats) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
++lfs_stats.psegwrites;
lfs_stats.blocktot += nblocks - 1;
if (fs->lfs_sp->seg_flags & SEGM_SYNC)
++lfs_stats.psyncwrites;
if (fs->lfs_sp->seg_flags & SEGM_CLEAN) {
++lfs_stats.pcleanwrites;
lfs_stats.cleanblocks += nblocks - 1;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
return (lfs_initseg(fs) || do_again);
}
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_writesuper(struct lfs *fs, daddr_t daddr)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
struct buf *bp;
dev_t i_dev;
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
int (*strategy)(void *);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int s;
struct vop_strategy_args vop_strategy_a;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
* If we can write one superblock while another is in
* progress, we risk not having a complete checkpoint if we crash.
* So, block here if a superblock write is in progress.
*/
Fix spllevel problem with superblock exclusion and with segment write throttle. May address PR#8383.
1999-11-17 23:08:30 +03:00
s = splbio();
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
while (fs->lfs_sbactive) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
tsleep(&fs->lfs_sbactive, PRIBIO+1, "lfs sb", 0);
}
fs->lfs_sbactive = daddr;
Fix spllevel problem with superblock exclusion and with segment write throttle. May address PR#8383.
1999-11-17 23:08:30 +03:00
splx(s);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
i_dev = VTOI(fs->lfs_ivnode)->i_dev;
strategy = VTOI(fs->lfs_ivnode)->i_devvp->v_op[VOFFSET(vop_strategy)];
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/* Set timestamp of this version of the superblock */
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
if (fs->lfs_version == 1)
fs->lfs_otstamp = time.tv_sec;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
fs->lfs_tstamp = time.tv_sec;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/* Checksum the superblock and copy it into a buffer. */
PR#6032: define fixed sized on-disk superblock structure.
1998-09-12 01:27:12 +04:00
fs->lfs_cksum = lfs_sb_cksum(&(fs->lfs_dlfs));
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
bp = lfs_newbuf(fs, VTOI(fs->lfs_ivnode)->i_devvp, fsbtodb(fs, daddr), LFS_SBPAD, LFS_NB_SBLOCK);
memset(bp->b_data + sizeof(struct dlfs), 0, LFS_SBPAD - sizeof(struct dlfs));
PR#6032: define fixed sized on-disk superblock structure.
1998-09-12 01:27:12 +04:00
*(struct dlfs *)bp->b_data = fs->lfs_dlfs;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
bp->b_dev = i_dev;
bp->b_flags |= B_BUSY | B_CALL | B_ASYNC;
bp->b_flags &= ~(B_DONE | B_ERROR | B_READ | B_DELWRI);
bp->b_iodone = lfs_supercallback;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/* XXX KS - same nasty hack as above */
bp->b_saveaddr = (caddr_t)fs;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
vop_strategy_a.a_desc = VDESC(vop_strategy);
vop_strategy_a.a_bp = bp;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
curproc->p_stats->p_ru.ru_oublock++;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
s = splbio();
Make the buffer cache code MP-safe.
2003-02-06 00:38:38 +03:00
V_INCR_NUMOUTPUT(bp->b_vp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
splx(s);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
++fs->lfs_iocount;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
(strategy)(&vop_strategy_a);
}
/*
* Logical block number match routines used when traversing the dirty block
* chain.
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_fake(struct lfs *fs, struct buf *bp)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
{
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
return LFS_IS_MALLOC_BUF(bp);
}
#if 0
int
lfs_match_real(struct lfs *fs, struct buf *bp)
{
return (lfs_match_data(fs, bp) && !lfs_match_fake(fs, bp));
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#endif
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_data(struct lfs *fs, struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
return (bp->b_lblkno >= 0);
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_indir(struct lfs *fs, struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t lbn;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lbn = bp->b_lblkno;
return (lbn < 0 && (-lbn - NDADDR) % NINDIR(fs) == 0);
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_dindir(struct lfs *fs, struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t lbn;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lbn = bp->b_lblkno;
return (lbn < 0 && (-lbn - NDADDR) % NINDIR(fs) == 1);
}
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_match_tindir(struct lfs *fs, struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Bump daddr_t to 64 bits. Replace it with int32_t in all places where it was used on-disk, so that on-disk formats remain the same. Remove ufs_daddr_t and ufs_lbn_t for the time being.
2003-01-25 00:55:02 +03:00
daddr_t lbn;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
lbn = bp->b_lblkno;
return (lbn < 0 && (-lbn - NDADDR) % NINDIR(fs) == 2);
}
/*
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
* XXX - The only buffers that are going to hit these functions are the
* segment write blocks, or the segment summaries, or the superblocks.
*
* All of the above are created by lfs_newbuf, and so do not need to be
* released via brelse.
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_callback(struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
struct lfs *fs;
fs = (struct lfs *)bp->b_saveaddr;
lfs_freebuf(fs, bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
static void
lfs_super_aiodone(struct buf *bp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
struct lfs *fs;
fs = (struct lfs *)bp->b_saveaddr;
NULL != 0
2000-05-19 08:34:39 +04:00
fs->lfs_sbactive = 0;
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
wakeup(&fs->lfs_sbactive);
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
if (--fs->lfs_iocount <= 1)
Fixes associated with filling an LFS: Change the space computation to appear to change the size of the *disk* rather than the *bytes used* when more segment summaries and inode blocks are written. Try to estimate the amount of space that these will take up when more files are written, so the disk size doesn't change too much. Regularize error returns from lfs_valloc, lfs_balloc, lfs_truncate: they now fail entirely, rather than succeeding half-way and leaving the fs in an inconsistent state. Rewrite lfs_truncate, mostly stealing from ffs_truncate. The old lfs_truncate had difficulty truncating a large file to a non-zero size (indirect blocks were not handled appropriately). Unmark VDIROP on fvp after ufs_remove, ufs_rmdir, so these can be reclaimed immediately: this vnode would not be written to disk again anyway if the removal succeeded, and if it failed, no directory operation occurred. ufs_makeinode and ufs_mkdir now remove IN_ADIROP on error.
2000-06-28 00:57:11 +04:00
wakeup(&fs->lfs_iocount);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_freebuf(fs, bp);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
static void
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
lfs_cluster_aiodone(struct buf *bp)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
{
struct lfs_cluster *cl;
struct lfs *fs;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
struct buf *tbp, *fbp;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
struct vnode *vp, *devvp;
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
struct inode *ip;
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
int s, error=0;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
char *cp;
extern int locked_queue_count;
extern long locked_queue_bytes;
if(bp->b_flags & B_ERROR)
error = bp->b_error;
cl = (struct lfs_cluster *)bp->b_saveaddr;
fs = cl->fs;
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
devvp = VTOI(fs->lfs_ivnode)->i_devvp;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
bp->b_saveaddr = cl->saveaddr;
cp = (char *)bp->b_data + cl->bufsize;
/* Put the pages back, and release the buffer */
while(cl->bufcount--) {
tbp = cl->bpp[cl->bufcount];
if(error) {
tbp->b_flags |= B_ERROR;
tbp->b_error = error;
}
/*
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
* We're done with tbp. If it has not been re-dirtied since
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
* the cluster was written, free it. Otherwise, keep it on
* the locked list to be written again.
*/
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
vp = tbp->b_vp;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if ((tbp->b_flags & (B_LOCKED | B_DELWRI)) == B_LOCKED)
LFS_UNLOCK_BUF(tbp);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
#if 0
else if (vp != devvp)
printf("dirtied while busy?! bp %p, ino %d, lbn %d\n",
tbp, vp ? VTOI(vp)->i_number : -1,
tbp->b_lblkno);
#endif
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
tbp->b_flags &= ~B_GATHERED;
LFS_BCLEAN_LOG(fs, tbp);
if(!(tbp->b_flags & B_CALL)) {
bremfree(tbp);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
s = splbio();
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
if(vp)
reassignbuf(tbp, vp);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
splx(s);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
tbp->b_flags |= B_ASYNC; /* for biodone */
}
#ifdef DIAGNOSTIC
if (tbp->b_flags & B_DONE) {
printf("blk %d biodone already (flags %lx)\n",
cl->bufcount, (long)tbp->b_flags);
}
#endif
if (tbp->b_flags & (B_BUSY | B_CALL)) {
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
if ((tbp->b_flags & B_CALL) && !LFS_IS_MALLOC_BUF(tbp)) {
/* printf("flags 0x%lx\n", tbp->b_flags); */
/*
* A buffer from the page daemon.
* We use the same iodone as it does,
* so we must manually disassociate its
* buffers from the vp.
*/
if (tbp->b_vp) {
/* This is just silly */
s = splbio();
brelvp(tbp);
tbp->b_vp = vp;
splx(s);
}
/* Put it back the way it was */
tbp->b_flags |= B_ASYNC;
/* Master buffers have B_AGE */
if (tbp->b_private == tbp)
tbp->b_flags |= B_AGE;
}
s = splbio();
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
biodone(tbp);
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
/*
* If this is the last block for this vnode, but
* there are other blocks on its dirty list,
* set IN_MODIFIED/IN_CLEANING depending on what
* sort of block. Only do this for our mount point,
* not for, e.g., inode blocks that are attached to
* the devvp.
* XXX KS - Shouldn't we set *both* if both types
* of blocks are present (traverse the dirty list?)
*/
simple_lock(&global_v_numoutput_slock);
if (vp != devvp && vp->v_numoutput == 0 &&
(fbp = LIST_FIRST(&vp->v_dirtyblkhd)) != NULL) {
ip = VTOI(vp);
#ifdef DEBUG_LFS
printf("lfs_cluster_aiodone: marking ino %d\n",
ip->i_number);
#endif
if (LFS_IS_MALLOC_BUF(fbp))
LFS_SET_UINO(ip, IN_CLEANING);
else
LFS_SET_UINO(ip, IN_MODIFIED);
}
simple_unlock(&global_v_numoutput_slock);
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
splx(s);
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
wakeup(vp);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
}
/* Fix up the cluster buffer, and release it */
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
if (cl->flags & LFS_CL_MALLOC)
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
lfs_free(fs, bp->b_data, LFS_NB_CLUSTER);
- Get rid of unused #ifdefs LFS_NO_PAGEMOVE and LFS_MALLOC_SUMMARY (both always true) and accompanying dead code. - When constructing write clusters in lfs_writeseg, if the block we are about to add is itself a cluster from GOP_WRITE, don't put a cluster in a cluster, just write the GOP_WRITE cluster on its own. This seems to represent a slight performance gain on my test machine. - Charge someone's rusage for writes on LFSes. It's difficult to tell who the "right" process to charge is; just charge whoever triggered the write.
2003-03-11 05:47:39 +03:00
bp->b_data = cl->olddata;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
bp->b_bcount = 0;
bp->b_iodone = NULL;
bp->b_flags &= ~B_DELWRI;
bp->b_flags |= B_DONE;
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
s = splbio();
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
reassignbuf(bp, bp->b_vp);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
splx(s);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
brelse(bp);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
/* Note i/o done */
if (cl->flags & LFS_CL_SYNC) {
if (--cl->seg->seg_iocount == 0)
wakeup(&cl->seg->seg_iocount);
/* printf("- %x => %d\n", cl->seg, cl->seg->seg_iocount); */
}
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#ifdef DIAGNOSTIC
if (fs->lfs_iocount == 0)
remove trailing \n in panic(). approved perry.
2002-09-27 19:35:29 +04:00
panic("lfs_cluster_aiodone: zero iocount");
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
#endif
Add an lfs_strategy() that checks to make sure we're not trying to read where the cleaner is trying to write, instead of tying up the "live" buffers (or pages). Fix a bug in the LFS_UBC case where oversized buffers would not be checksummed correctly, causing uncleanable segments. Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1 as well as 0, since we wait in some places for it to drop to 1. Activate all pages that make it into lfs_gop_write without the segment lock held, since they must have been dirtied very recently, even if PG_DELWRI is not set.
2003-03-08 05:55:47 +03:00
if (--fs->lfs_iocount <= 1)
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
wakeup(&fs->lfs_iocount);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now (there are still some details to work out) but expect that to go away soon. To support these basic changes (creation of lfs_putpages, lfs_gop_write, mods to lfs_balloc) several other changes were made, to wit: * Create a writer daemon kernel thread whose purpose is to handle page writes for the pagedaemon, but which also takes over some of the functions of lfs_check(). This thread is started the first time an LFS is mounted. * Add a "flags" parameter to GOP_SIZE. Current values are GOP_SIZE_READ, meaning that the call should return the size of the in-core version of the file, and GOP_SIZE_WRITE, meaning that it should return the on-disk size. One of GOP_SIZE_READ or GOP_SIZE_WRITE must be specified. * Instead of using malloc(...M_WAITOK) for everything, reserve enough resources to get by and use malloc(...M_NOWAIT), using the reserves if necessary. Use the pool subsystem for structures small enough that this is feasible. This also obsoletes LFS_THROTTLE. And a few that are not strictly necessary: * Moves the LFS inode extensions off onto a separately allocated structure; getting closer to LFS as an LKM. "Welcome to 1.6O." * Unified GOP_ALLOC between FFS and LFS. * Update LFS copyright headers to correct values. * Actually cast to unsigned in lfs_shellsort, like the comment says. * Keep track of which segments were empty before the previous checkpoint; any segments that pass two checkpoints both dirty and empty can be summarily cleaned. Do this. Right now lfs_segclean still works, but this should be turned into an effectless compatibility syscall.
2003-02-18 02:48:08 +03:00
pool_put(&fs->lfs_bpppool, cl->bpp);
cl->bpp = NULL;
pool_put(&fs->lfs_clpool, cl);
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
}
static void
lfs_generic_callback(struct buf *bp, void (*aiodone)(struct buf *))
{
/* reset b_iodone for when this is a single-buf i/o. */
bp->b_iodone = aiodone;
Tabify, and fix some comment alignment problems.
2003-02-20 07:27:23 +03:00
simple_lock(&uvm.aiodoned_lock); /* locks uvm.aio_done */
For synchronous writes, keep separate i/o counters for each write, so processes don't have to wait for one another to finish (e.g., nfsd seems to be a little happier now, though I haven't measured the difference). Synchronous checkpoints, however, must always wait for all i/o to finish. Take the contents of the callback functions and have them run in thread context instead (aiodoned thread). lfs_iocount no longer has to be protected in splbio(), and quite a bit less of the segment construction loop needs to be in splbio() as well. If lfs_markv is handed a block that is not the correct size according to the inode, refuse to process it. (Formerly it was extended to the "correct" size.) This is possibly more prone to deadlock, but less prone to corruption. lfs_segclean now outright refuses to clean segments that appear to have live bytes in them. Again this may be more prone to deadlock but avoids corruption. Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means that no UFS functions need to know about LFS_ITIMES any more. Remove the reference from ufs/inode.h. Tested on i386, test-compiled on alpha.
2002-06-16 04:13:15 +04:00
TAILQ_INSERT_TAIL(&uvm.aio_done, bp, b_freelist);
wakeup(&uvm.aiodoned);
simple_unlock(&uvm.aiodoned_lock);
}
static void
lfs_cluster_callback(struct buf *bp)
{
lfs_generic_callback(bp, lfs_cluster_aiodone);
}
void
lfs_supercallback(struct buf *bp)
{
lfs_generic_callback(bp, lfs_super_aiodone);
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes I found while making sure there weren't any new ones. * Make the write clusters keep track of the buffers whose blocks they contain. This should make it possible to (1) write clusters using a page mapping instead of malloc, if desired, and (2) schedule blocks for rewriting (somewhere else) if a write error occurs. Code is present to use pagemove() to construct the clusters but that is untested and will go away anyway in favor of page mapping. * DEBUG now keeps a log of Ifile writes, so that any lingering instances of the "dirty bufs" problem can be properly debugged. * Keep track of whether the Ifile has been dirtied by various routines that can be called by lfs_segwrite, and loop on that until it is clean, for a checkpoint. Checkpoints need to be squeaky clean. * Warn the user (once) if the Ifile grows larger than is reasonable for their buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can grow. * If an inode is not found in a disk block, try rereading the block, under the assumption that the block was copied to a cluster and then freed. * Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
2002-05-15 00:03:53 +04:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
* Shellsort (diminishing increment sort) from Data Structures and
* Algorithms, Aho, Hopcraft and Ullman, 1983 Edition, page 290;
* see also Knuth Vol. 3, page 84. The increments are selected from
* formula (8), page 95. Roughly O(N^3/2).
*/
/*
* This is our own private copy of shellsort because we want to sort
* two parallel arrays (the array of buffer pointers and the array of
* logical block numbers) simultaneously. Note that we cast the array
* of logical block numbers to a unsigned in this routine so that the
* negative block numbers (meta data blocks) sort AFTER the data blocks.
*/
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
void
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
lfs_shellsort(struct buf **bp_array, int32_t *lb_array, int nmemb, int size)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
static int __rsshell_increments[] = { 4, 1, 0 };
Remove register declarations.
2000-03-30 16:41:09 +04:00
int incr, *incrp, t1, t2;
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
struct buf *bp_temp;
lfs prototypes
1996-02-10 01:28:45 +03:00
for (incrp = __rsshell_increments; (incr = *incrp++) != 0;)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
for (t1 = incr; t1 < nmemb; ++t1)
for (t2 = t1 - incr; t2 >= 0;)
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
if ((u_int32_t)bp_array[t2]->b_lblkno >
(u_int32_t)bp_array[t2 + incr]->b_lblkno) {
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
bp_temp = bp_array[t2];
bp_array[t2] = bp_array[t2 + incr];
bp_array[t2 + incr] = bp_temp;
t2 -= incr;
} else
break;
Fix a buffer overflow bug in the LFS_UBC case that manifested itself either as a mysterious UVM error or as "panic: dirty bufs". Verify maximum size in lfs_malloc. Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from lfs_gop_write. When unwiring pages in lfs_gop_write, deactivate them, under the theory that the pagedaemon wanted to free them last we knew.
2003-02-23 03:22:33 +03:00
/* Reform the list of logical blocks */
incr = 0;
for (t1 = 0; t1 < nmemb; t1++) {
for (t2 = 0; t2 * size < bp_array[t1]->b_bcount; t2++) {
lb_array[incr++] = bp_array[t1]->b_lblkno + t2;
}
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
/*
* Check VXLOCK. Return 1 if the vnode is locked. Otherwise, vget it.
*/
lfs prototypes
1996-02-10 01:28:45 +03:00
int
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_vref(struct vnode *vp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
* If we return 1 here during a flush, we risk vinvalbuf() not
* being able to flush all of the pages from this vnode, which
* will cause it to panic. So, return 0 if a flush is in progress.
*/
if (vp->v_flag & VXLOCK) {
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (IS_FLUSHING(VTOI(vp)->i_lfs,vp)) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
return 0;
}
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
return (1);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
return (vget(vp, 0));
}
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
/*
* This is vrele except that we do not want to VOP_INACTIVE this vnode. We
* inline vrele here to avoid the vn_lock and VOP_INACTIVE call at the end.
*/
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_vunref(struct vnode *vp)
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
{
Change lfs_sb_cksum to use offsetof() instead of an inlined version. Fix lfs_vref/lfs_vunredf to ignore VXLOCKed vnodes that are also being flushed. Improve the debugging messages somewhat.
1999-03-26 00:54:10 +03:00
/*
* Analogous to lfs_vref, if the node is flushing, fake it.
*/
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if ((vp->v_flag & VXLOCK) && IS_FLUSHING(VTOI(vp)->i_lfs,vp)) {
Change lfs_sb_cksum to use offsetof() instead of an inlined version. Fix lfs_vref/lfs_vunredf to ignore VXLOCKed vnodes that are also being flushed. Improve the debugging messages somewhat.
1999-03-26 00:54:10 +03:00
return;
}
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
simple_lock(&vp->v_interlock);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
#ifdef DIAGNOSTIC
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_usecount <= 0) {
Fixes associated with filling an LFS: Change the space computation to appear to change the size of the *disk* rather than the *bytes used* when more segment summaries and inode blocks are written. Try to estimate the amount of space that these will take up when more files are written, so the disk size doesn't change too much. Regularize error returns from lfs_valloc, lfs_balloc, lfs_truncate: they now fail entirely, rather than succeeding half-way and leaving the fs in an inconsistent state. Rewrite lfs_truncate, mostly stealing from ffs_truncate. The old lfs_truncate had difficulty truncating a large file to a non-zero size (indirect blocks were not handled appropriately). Unmark VDIROP on fvp after ufs_remove, ufs_rmdir, so these can be reclaimed immediately: this vnode would not be written to disk again anyway if the removal succeeded, and if it failed, no directory operation occurred. ufs_makeinode and ufs_mkdir now remove IN_ADIROP on error.
2000-06-28 00:57:11 +04:00
printf("lfs_vunref: inum is %d\n", VTOI(vp)->i_number);
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
printf("lfs_vunref: flags are 0x%lx\n", (u_long)vp->v_flag);
printf("lfs_vunref: usecount = %ld\n", (long)vp->v_usecount);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
panic("lfs_vunref: v_usecount<0");
}
#endif
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
vp->v_usecount--;
if (vp->v_usecount > 0) {
simple_unlock(&vp->v_interlock);
return;
}
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
/*
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
* insert at tail of LRU list
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
*/
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
simple_lock(&vnode_free_list_slock);
Changes to stabilize LFS. The first two of these should also apply to the 1.4 branch. * Use a separate per-fs lock, instead of ufs_hashlock, to protect the Inode free list. This seems to prevent the "lockmgr: %d, not exclusive lock holder %d, unlocking" message I was mis-attributing last night to an unlocked vnode being passed to vrele. * Change calling semantics of lfs_ifind, to give better error reporting: If fed a struct buf, it can report the block number of the offending inode block as well as the inode number. * Back out rev 1.10 of lfs_subr.c, since the replacement code was slightly uglier while being functionally identical. * Make lfs_vunref use the same free list convention as vrele/vput, so that vget does not remove vnodes from a hash list they are not on.
2000-01-19 03:03:04 +03:00
if (vp->v_holdcnt > 0)
TAILQ_INSERT_TAIL(&vnode_hold_list, vp, v_freelist);
else
TAILQ_INSERT_TAIL(&vnode_free_list, vp, v_freelist);
Merge with Lite2 + local changes
1998-03-01 05:20:01 +03:00
simple_unlock(&vnode_free_list_slock);
simple_unlock(&vp->v_interlock);
Update to 4.4-Lite fs code, with local changes.
1994-06-08 15:41:58 +04:00
}
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
/*
* We use this when we have vnodes that were loaded in solely for cleaning.
* There is no reason to believe that these vnodes will be referenced again
* soon, since the cleaning process is unrelated to normal filesystem
* activity. Putting cleaned vnodes at the tail of the list has the effect
* of flushing the vnode LRU. So, put vnodes that were loaded only for
* cleaning at the head of the list, instead.
*/
void
Merge the short-lived perseant-lfsv2 branch into the trunk. Kernels and tools understand both v1 and v2 filesystems; newfs_lfs generates v2 by default. Changes for the v2 layout include: - Segments of non-PO2 size and arbitrary block offset, so these can be matched to convenient physical characteristics of the partition (e.g., stripe or track size and offset). - Address by fragment instead of by disk sector, paving the way for non-512-byte-sector devices. In theory fragments can be as large as you like, though in reality they must be smaller than MAXBSIZE in size. - Use serial number and filesystem identifier to ensure that roll-forward doesn't get old data and think it's new. Roll-forward is enabled for v2 filesystems, though not for v1 filesystems by default. - The inode free list is now a tailq, paving the way for undelete (undelete is not yet implemented, but can be without further non-backwards-compatible changes to disk structures). - Inode atime information is kept in the Ifile, instead of on the inode; that is, the inode is never written *just* because atime was changed. Because of this the inodes remain near the file data on the disk, rather than wandering all over as the disk is read repeatedly. This speeds up repeated reads by a small but noticeable amount. Other changes of note include: - The ifile written by newfs_lfs can now be of arbitrary length, it is no longer restricted to a single indirect block. - Fixed an old bug where ctime was changed every time a vnode was created. I need to look more closely to make sure that the times are only updated during write(2) and friends, not after-the-fact during a segment write, and certainly not by the cleaner.
2001-07-14 00:30:18 +04:00
lfs_vunref_head(struct vnode *vp)
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
{
simple_lock(&vp->v_interlock);
#ifdef DIAGNOSTIC
add spaces for KNF. confirmed to produce identical objects.
2001-11-24 00:44:25 +03:00
if (vp->v_usecount == 0) {
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
panic("lfs_vunref: v_usecount<0");
}
#endif
vp->v_usecount--;
if (vp->v_usecount > 0) {
simple_unlock(&vp->v_interlock);
return;
}
/*
* insert at head of LRU list
*/
simple_lock(&vnode_free_list_slock);
Back out rev 1.174 of vfs_subr.c, because the splbio() wasn't protecting enough to be useful, and broadening it so that it did would have meant that operations possibly requiring synchronous disk activity would have to be done in splbio(). This clearly was not going to work. Worked around this in the LFS case by having lfs_cluster_callback put an extra hold on the vnode before calling biodone(), and taking the hold off without HOLDRELE's problematic list swapping. lfs_vunref() will take care of that---in thread context---on the next write if need be. Also, ensure that the list walking in lfs_{writevnodes,segunlock,gather} takes into account the possibility that the list may change underneath it (possibly because it itself deleted an element). Tested on i386, test-compiled on alpha.
2002-05-24 03:05:25 +04:00
if (vp->v_holdcnt > 0)
TAILQ_INSERT_TAIL(&vnode_hold_list, vp, v_freelist);
else
TAILQ_INSERT_HEAD(&vnode_free_list, vp, v_freelist);
New sources should leave the LFS in a more-or-less working state. Changes include: - DIROP segregation is enabled, and greater care is taken to make sure that a checkpoint completes. Fsck is not needed to remount the filesystem. - Several checks to make sure that the LFS subsystem does not overuse various resources (memory, in particular). - The cleaner routines, lfs_markv in particular, are completely rewritten. A buffer overflow is removed. Greater care is taken to ensure that inodes come from where lfs_cleanerd say they come from (so we know nothing has changed since lfs_bmapv was called). - Fragment allocation is fixed, so that writes beyond end-of-file do the right thing.
1999-03-10 03:20:00 +03:00
simple_unlock(&vnode_free_list_slock);
simple_unlock(&vp->v_interlock);
}
Reference in New Issue Copy Permalink