NetBSD/usr.sbin/makefs
chs f298a94b73 ufs: fixed signed/unsigned bugs affecting large file systems
Apply these commits from FreeBSD:

  commit e870d1e6f97cc73308c11c40684b775bcfa906a2
  Author: Kirk McKusick <mckusick@FreeBSD.org>
  Date:   Wed Feb 10 20:10:35 2010 +0000

    This fix corrects a problem in the file system that treats large
    inode numbers as negative rather than unsigned. For a default
    (16K block) file system, this bug began to show up at a file system
    size above about 16Tb.

    To fully handle this problem, newfs must be updated to ensure that
    it will never create a filesystem with more than 2^32 inodes. That
    patch will be forthcoming soon.

    Reported by: Scott Burns, John Kilburg, Bruce Evans
    Followup by: Jeff Roberson
    PR:          133980
    MFC after:   2 weeks

  commit 81479e688b0f643ffacd3f335b4b4bba460b769d
  Author: Kirk McKusick <mckusick@FreeBSD.org>
  Date:   Thu Feb 11 18:14:53 2010 +0000

    One last pass to get all the unsigned comparisons correct.


In additional to the changes from FreeBSD, this commit includes quite a few
related changes to appease -Wsign-compare.
2023-01-07 19:41:29 +00:00
..
cd9660 makefs(8): Nix trailing whitespace. 2022-04-09 10:05:35 +00:00
chfs makefs(8): Nix trailing whitespace. 2022-04-09 10:05:35 +00:00
ffs ufs: fixed signed/unsigned bugs affecting large file systems 2023-01-07 19:41:29 +00:00
msdos makefs(8): Nix trailing whitespace. 2022-04-09 10:05:35 +00:00
udf makefs(8): Fix tool build -- no libutil DPADD if HOSTPROG. 2022-04-10 09:22:35 +00:00
v7fs A component name is a counted string (cn_nameptr, cn_namelen), 2022-02-11 10:55:15 +00:00
Makefile ufs: fixed signed/unsigned bugs affecting large file systems 2023-01-07 19:41:29 +00:00
README fix various typos in comments and makefs README file. 2022-10-26 21:56:19 +00:00
TODO
cd9660.c s/filname/filename/ 2021-12-21 21:28:31 +00:00
cd9660.h little size_t/ssize_t... 2015-12-24 15:52:37 +00:00
chfs.c makefs(8): Nix trailing whitespace. 2022-04-09 10:05:35 +00:00
chfs_makefs.h - use emalloc and friends 2013-01-28 21:03:26 +00:00
ffs.c ufs: fixed signed/unsigned bugs affecting large file systems 2023-01-07 19:41:29 +00:00
ffs.h Restore backward compatibility of UFS2 with previous NetBSD releases by 2022-11-17 06:40:38 +00:00
makefs.8 Restore backward compatibility of UFS2 with previous NetBSD releases by 2022-11-17 06:40:38 +00:00
makefs.c makefs(8): Nix trailing whitespace. 2022-04-09 10:05:35 +00:00
makefs.h Revert, sorry this was not intended to be committed. 2022-05-08 21:29:20 +00:00
msdos.c makefs(8): Nix trailing whitespace. 2022-04-09 10:05:35 +00:00
msdos.h remove clause 3. 2015-10-16 16:40:02 +00:00
udf.c When creating disc images, keep the proposed size a multiple of the blockingnr 2022-05-07 08:54:02 +00:00
v7fs.c make everything use the generic options parser. 2013-01-29 15:52:25 +00:00
v7fs_makefs.h
walk.c makefs(8): Nix trailing whitespace. 2022-04-09 10:05:35 +00:00

README

$NetBSD: README,v 1.8 2022/10/26 21:56:19 andvar Exp $

makefs - build a file system image from a directory tree

NOTES:

    *   This tool uses modified local copies of source found in other
	parts of the tree.  This is intentional.

    *	makefs is a work in progress, and subject to change.


user overview:
--------------

makefs creates a file system image from a given directory tree.
the following file system types can be built:

	cd9660	ISO 9660 file system
	chfs	"Chip" file system, for flash devices
	ffs	BSD fast file system
	msdos	MS-DOS `FAT' file system (FAT12, FAT16, FAT32)
	udf	Universal Disk Format file system
	v7fs	7th edition(V7) file system

Support for the following file systems maybe be added in the future

	ext2fs	Linux EXT2 file system

Various file system independent parameters and constraints can be
specified, such as:

	- minimum file system size (in KB)
	- maximum file system size (in KB)
	- free inodes
	- free blocks (in KB)
	- mtree(8) specification file containing permissions and ownership
	  to use in image, overriding the settings in the directory tree
	- file containing list of files to specifically exclude or include
	- fnmatch(3) pattern of filenames to exclude or include
	- endianness of target file system

File system specific parameters can be given as well, with a command
line option such as "-o fsspeccific-options,comma-separated".
For example, ffs would allow tuning of:

	- block & fragment size
	- cylinder groups
	- number of blocks per inode
	- minimum free space

Other file systems might have controls on how to "munge" file names to
fit within the constraints of the target file system.

Exit codes:
	0	all ok
	1	fatal error
	2	some files couldn't be added during image creation
		(bad perms, missing file, etc). image will continue
		to be made


Implementation overview:
------------------------

The implementation must allow for easy addition of extra file systems
with minimal changes to the file system independent sections.

The main program will:
	- parse the options, including calling fs-specific routines to
	  validate fs-specific options
	- walk the tree, building up a data structure which represents
	  the tree to stuff into the image. The structure will
	  probably be a similar tree to what mtree(8) uses internally;
	  a linked list of entries per directory with a child pointer
	  to children of directories. ".." won't be stored in the list;
	  the fs-specific tree walker should add this if required by the fs. 
	  this builder have the smarts to handle hard links correctly.
	- (optionally) Change the permissions in the tree according to
	  the mtree(8) specfile
	- Call an fs-specific routine to build the image based on the
	  data structures.

Each fs-specific module should have the following external interfaces:

    prepare_options	optional file system specific defaults that need to be
			setup before parsing fs-specific options.

    parse_options	parse the string for fs-specific options, feeding
			errors back to the user as appropriate

    cleanup_options	optional file system specific data that need to be
			cleaned up when done with this filesystem.

    make_fs		take the data structures representing the
			directory tree and fs parameters,
			validate that the parameters are valid
			(e.g, the requested image will be large enough), 
			create the image, and
			populate the image

prepare_options and cleanup_options are optional and can be NULL.

NOTE: All file system specific options are referenced via the fs_specific
pointer from the fsinfo_t strucutre. It is up to the filesystem to allocate
and free any data needed for this via the prepare and cleanup callbacks.

Each fs-specific module will need to add its routines to the dispatch array
in makefs.c and add prototypes for these to makefs.h

All other implementation details should not need to change any of the
generic code.

ffs implementation
------------------

In the ffs case, we can leverage off sbin/newfs/mkfs.c to actually build
the image. When building and populating the image, the implementation
can be greatly simplified if some assumptions are made:
	- the total required size (in blocks and inodes) is determined
	  as part of the validation phase
	- a "file" (including a directory) has a known size, so
	  support for growing a file is not necessary

Two underlying primitives are provided:
	make_inode	create an inode, returning the inode number

	write_file	write file (from memory if DIR, file descriptor
			if FILE or SYMLINK), referencing given inode.
			it is smart enough to know if a short symlink
			can be stuffed into the inode, etc.

When creating a directory, the directory entries in the previously
built tree data structure is scanned and built in memory so it can
be written entirely as a single write_file() operation.