makefs - create a file system image from a directory tree.

It doesn't need any special privileges or kernel devices.

Only ffs image creation is supported at this time, although makefs has been
designed to allow the addition of other file system formats by writing new
back-ends.

This program was designed & implemented by Luke Mewburn of Wasabi Systems.
This commit is contained in:
lukem 2001-10-26 06:50:48 +00:00
parent 6d536163de
commit de8b3ad2c7
5 changed files with 744 additions and 0 deletions

21
usr.sbin/makefs/Makefile Normal file
View File

@ -0,0 +1,21 @@
# $NetBSD: Makefile,v 1.1.1.1 2001/10/26 06:56:13 lukem Exp $
#
PROG= makefs
SRCS= makefs.c walk.c \
ffs.c mkfs.c buf.c \
misc.c spec.c pack_dev.c stat_flags.c \
ffs_alloc.c ffs_balloc.c ffs_bswap.c ffs_subr.c ffs_tables.c ufs_bmap.c
LDADD+=-lutil
LSSRC= ${.CURDIR}/../../bin/ls
MKNODSRC= ${.CURDIR}/../../sbin/mknod
MTREESRC= ${.CURDIR}/../../usr.sbin/mtree
CPPFLAGS+= -I${.CURDIR} -I${LSSRC} -I${MKNODSRC} -I${MTREESRC}
.PATH: ${.CURDIR}/ffs ${.CURDIR}/../../sys/ufs/ffs \
${LSSRC} ${MKNODSRC} ${MTREESRC}
WARNS?= 2
.include <bsd.prog.mk>

111
usr.sbin/makefs/README Normal file
View File

@ -0,0 +1,111 @@
$NetBSD: README,v 1.1.1.1 2001/10/26 06:50:48 lukem Exp $
makefs - build a file system image from a directory tree
NOTES:
* This tool uses modified local copies of source found in other
parts of the tree. This is intentional.
* makefs is a work in progress, and subject to change.
user overview:
--------------
makefs creates a file system image from a given directory tree.
the following file system types can be built:
ffs BSD fast file system
Support for the following file systems maybe be added in the future
ext2fs Linux EXT2 file system
fat MS-DOS `FAT' file system (FAT12, FAT16, FAT32)
cd9660 ISO 9660 file system
Various file system independent parameters and contraints can be
specified, such as:
- minimum file system size (in KB)
- maximum file system size (in KB)
- free inodes
- free blocks (in KB)
- mtree(8) specification file containing permissions and ownership
to use in image, overridding the settings in the directory tree
- file containing list of files to specifically exclude or include
- fnmatch(3) pattern of filenames to exclude or include
- endianness of target file system
File system specific parameters can be given as well, with a command
line option such as "-o fsspeccific-options,comma-separated".
For example, ffs would allow tuning of:
- block & fragement size
- cylinder groups
- number of blocks per inode
- minimum free space
Other file systems might have controls on how to "munge" file names to
fit within the constraints of the target file system.
Exit codes:
0 all ok
1 fatal error
2 some files couldn't be added during image creation
(bad perms, missing file, etc). image will continue
to be made
Implementation overview:
------------------------
The implementation must allow for easy addition of extra file systems
with minimal changes to the file system independent sections.
The main program will:
- parse the options, including calling fs-specific routines to
validate fs-specific options
- walk the tree, building up a data structure which represents
the tree to stuff into the image. The structure will
probably be a similar tree to what mtree(8) uses internally;
a linked list of entries per directory with a child pointer
to children of directories. ".." won't be stored in the list;
the fs-specific tree walker should add this if required by the fs.
this builder have the smarts to handle hard links correctly.
- (optionally) Change the permissions in the tree according to
the mtree(8) specfile
- Call an fs-specific routine to build the image based on the
data structures.
Each fs-specific module should have the following external interfaces:
parse_options parse the string for fs-specific options, feeding
errors back to the user as appropriate
make_fs take the data structures representing the
directory tree and fs parameters,
validate that the parameters are valid
(e.g, the requested image will be large enough),
create the image, and
populate the image
ffs implementation
------------------
In the ffs case, we can leverage off sbin/newfs/mkfs.c to actually build
the image. When building and populating the image, the implementation
can be greatly simplified if some assumptions are made:
- the total required size (in blocks and inodes) is determined
as part of the validation phase
- a "file" (including a directory) has a known size, so
support for growing a file is not necessary
Two underlying primitives are provided:
make_inode create an inode, returning the inode number
write_file write file (from memory if DIR, file descriptor
if FILE or SYMLINK), referencing given inode.
it is smart enough to know if a short symlink
can be stuffed into the inode, etc.
When creating a directory, the directory entries in the previously
built tree data structure is scanned and built in memory so it can
be written entirely as a single write_file() operation.

26
usr.sbin/makefs/TODO Normal file
View File

@ -0,0 +1,26 @@
$NetBSD: TODO,v 1.1.1.1 2001/10/26 07:41:07 lukem Exp $
todo:
- finish makefs.8
- testing
- even more testing
outstanding bugs:
- size estimation is still out (need to take into account indirect blocks!)
- mkfs.c parameter checking when density is rather high or low (dumps core)
e.g, a large directory with a small number of files dumps core with a
floating exception i.e, density is too high such as 21886635 or too
low such as 30
discuss:
- consider replacing ffs_balloc() et al with own code that doesn't
need hacked-up buf.c code

257
usr.sbin/makefs/makefs.8 Normal file
View File

@ -0,0 +1,257 @@
.\" $NetBSD: makefs.8,v 1.1.1.1 2001/10/26 07:40:18 lukem Exp $
.\"
.\" Copyright 2001 Wasabi Systems, Inc.
.\" All rights reserved.
.\"
.\" Written by Luke Mewburn for Wasabi Systems, Inc.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" 3. All advertising materials mentioning features or use of this software
.\" must display the following acknowledgement:
.\" This product includes software developed for the NetBSD Project by
.\" Wasabi Systems, Inc.
.\" 4. The name of Wasabi Systems, Inc. may not be used to endorse
.\" or promote products derived from this software without specific prior
.\" written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY WASABI SYSTEMS, INC. ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
.\" PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WASABI SYSTEMS, INC
.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
.\" POSSIBILITY OF SUCH DAMAGE.
.\"
.Dd October 26, 2001
.Dt MAKEFS 8
.Os
.Sh NAME
.Nm makefs
.Nd create a file system image from a directory tree
.Sh SYNOPSIS
.Nm ""
.Bk -words
.Op Fl t Ar fs-type
.Ek
.Bk -words
.Op Fl o Ar fs-options
.Ek
.Bk -words
.Op Fl d Ar debug-mask
.Ek
.Bk -words
.Op Fl B Ar byte-order
.Ek
.Bk -words
.Op Fl S Ar sector-size
.Ek
.Bk -words
.Op Fl M Ar minimum-size
.Ek
.Bk -words
.Op Fl m Ar maximum-size
.Ek
.Bk -words
.Op Fl s Ar image-size
.Ek
.Bk -words
.Op Fl b Ar free-blocks
.Ek
.Bk -words
.Op Fl f Ar free-files
.Ek
.Bk -words
.Op Fl F Ar specfile
.Ek
.Ar image-file
.Ar directory
.Sh DESCRIPTION
The utility
.Nm
creates a file system image into
.Ar image-file
from the directory tree
.Ar directory .
No special devices or privileges are required to perform this task.
.Pp
The options are as follows:
.Bl -tag -width flag
.It Fl t Ar fs-type
Create an
.Ar fs-type
file system image.
The following file system types are supported:
.Bl -tag -width ffs -offset indent
.It Sy ffs
BSD fast file system (default).
.El
.It Fl o Ar fs-options
Set file system specific options.
.Ar fs-options
is a comma separated list of options.
Valid file system specific options are detailed below.
.It Fl d Ar debug-mask
Enable various levels of debugging, depending upon which bits are set
in
.Ar debug-mask .
XXX: document these
.It Fl B Ar byte-order
Set the byte order of the image to
.Ar byte-order .
Valid byte orders are
.Ql big
or
.Ql be
for big endian, and
.Ql little
or
.Ql le
for little endian.
Some file systems may have a fixed byte order; in those cases this
argument will be ignored.
.It Fl S Ar sector-size
Set the file system sector size to
.Ar sector-size .
Defaults to 512.
.It Fl M Ar minimum-size
Set the minimum size of the file system image to
.Ar minimum-size .
.It Fl m Ar maximum-size
Set the maximum size of the file system image to
.Ar maximum-size .
An error will be raised if the target file system needs to be larger
than this to accomodate the provided directory tree.
.It Fl s Ar image-size
Set the size of the file system image to
.Ar image-size .
.It Fl b Ar free-blocks
Ensure that a minimum of
.Ar free-blocks
free blocks exist in the image.
An optional
.Ql %
suffix may be provided to indicate that
.Ar free-blocks
indicates a percentage of the calculated image size
.It Fl f Ar free-files
Ensure that a minimum of
.Ar free-files
free files (inodes) exist in the image.
An optional
.Ql %
suffix may be provided to indicate that
.Ar free-blocks
indicates a percentage of the calculated image size
.It Fl F Ar specfile
Use
.Ar specfile
as an
.Xr mtree 8
.Sq specfile
specification.
.Pp
If a specfile entry exists in the underlying file system, its permissions and
modification time will be used unless specifically overridden by the specfile.
An error will be raised if the type of entry in the specfile conflicts
with that of an existing entry.
.Pp
Otherwise, it is necessary to specify at least the following parameters
in the specfile:
.Sy type ,
.Sy mode ,
.Sy gname
or
.Sy gid ,
and
.Sy uname
or
.Sy uid ,
.Sy device
(in the case of block or character devices), and
.Sy link
(in the case of symbolic links).
If
.Sy time
isn't provided, the current time will be used.
If
.Sy flags
isn't provided, the current file flags will be used.
.El
.Pp
An optional suffix may be provided for numeric size arguments,
which changes the intrepretation of the argument as follows:
.Bl -tag -width 3n -offset indent -compact
.It b
Causes no modification. (Default; optional)
.It k
Kilo; multiply the argument by 1024
.It m
Mega; multiply the argument by 1048576
.It g
Giga; multiply the argument by 1073741824
.El
.\"
.\"
.Ss FFS-specific options
.Pp
.Sy ffs
images have ffs-specific optional parameters that may be provided.
Each of the options consists of a keyword, an equals sign
.Pq Ql = ,
and a value.
The following keywords are supported:
.Pp
.Bl -tag -width optimization -offset indent -compact
.It Sy avgfilesize
Expected average file size
.It Sy avgfpdir
Expected number of files per directory
.It Sy bsize
Block size
.It Sy cpg
Cylinders per group
.It Sy density
Bytes per inode
.It Sy fsize
Fragment size
.It Sy maxbpg
Maximum blocks per cylinder group
.It Sy minfree
Minimum % free
.It Sy nsectors
Number of sectors
.It Sy ntracks
Number of tracks
.It Sy optimization
Optimization preference; one of
.Ql space
or
.Ql time .
.It Sy rotdelay
Rotational delay
.It Sy rpm
Revolutions per minute
.It Sy nrpos
Number of rotational positions
.El
.Sh SEE ALSO
.Xr mtree 8 ,
.Xr newfs 8
.Sh AUTHOR
Luke Mewburn <lukem@netbsd.org>
.Sh HISTORY
The
.Nm
utility appeared in
.Nx 1.6 .

329
usr.sbin/makefs/makefs.c Normal file
View File

@ -0,0 +1,329 @@
/* $NetBSD: makefs.c,v 1.1.1.1 2001/10/26 06:59:02 lukem Exp $ */
/*
* Copyright 2001 Wasabi Systems, Inc.
* All rights reserved.
*
* Written by Luke Mewburn for Wasabi Systems, Inc.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
* This product includes software developed for the NetBSD Project by
* Wasabi Systems, Inc.
* 4. The name of Wasabi Systems, Inc. may not be used to endorse
* or promote products derived from this software without specific prior
* written permission.
*
* THIS SOFTWARE IS PROVIDED BY WASABI SYSTEMS, INC. ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WASABI SYSTEMS, INC
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <assert.h>
#include <ctype.h>
#include <err.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include "makefs.h"
/*
* list of supported file systems and dispatch functions
*/
typedef struct {
const char *type;
int (*parse_options)(const char *, fsinfo_t *);
void (*make_fs)(const char *, const char *, fsnode *,
fsinfo_t *);
} fstype_t;
static fstype_t fstypes[] = {
{ "ffs", ffs_parse_opts, ffs_makefs },
{ NULL },
};
int debug;
struct timespec start_time;
static fstype_t *get_fstype(const char *);
static long long strsuftoll(const char *, const char *, long long, long long);
static void usage(void);
int main(int, char *[]);
int
main(int argc, char *argv[])
{
struct timeval start;
fstype_t *fstype;
fsinfo_t fsoptions;
fsnode *root;
int ch, len;
char *specfile;
debug = 0;
if ((fstype = get_fstype(DEFAULT_FSTYPE)) == NULL)
errx(1, "Unknown default fs type `%s'.", DEFAULT_FSTYPE);
(void)memset(&fsoptions, 0, sizeof(fsoptions));
fsoptions.fd = -1;
specfile = NULL;
if (gettimeofday(&start, NULL) == -1)
err(1, "Unable to get system time");
TIMEVAL_TO_TIMESPEC(&start, &start_time);
while ((ch = getopt(argc, argv, "B:b:d:f:F:M:m:o:s:S:t:")) != -1) {
switch (ch) {
case 'B':
if (strcmp(optarg, "be") == 0 ||
strcmp(optarg, "big") == 0) {
#if BYTE_ORDER == LITTLE_ENDIAN
fsoptions.needswap = 1;
#endif
} else if (strcmp(optarg, "le") == 0 ||
strcmp(optarg, "little") == 0) {
#if BYTE_ORDER == BIG_ENDIAN
fsoptions.needswap = 1;
#endif
} else {
warnx("Invalid endian `%s'.", optarg);
usage();
}
break;
case 'b':
len = strlen(optarg) - 1;
if (optarg[len] == '%') {
optarg[len] = '\0';
fsoptions.freeblockpc =
strsuftoll("free block percentage",
optarg, 0, LLONG_MAX);
} else {
fsoptions.freeblocks = strsuftoll("free blocks",
optarg, 0, LLONG_MAX);
}
break;
case 'd':
debug = (int)strsuftoll("debug mask", optarg,
0, LLONG_MAX);
break;
case 'f':
len = strlen(optarg) - 1;
if (optarg[len] == '%') {
optarg[len] = '\0';
fsoptions.freefilepc =
strsuftoll("free file percentage",
optarg, 0, LLONG_MAX);
} else {
fsoptions.freefiles = strsuftoll("free files",
optarg, 0, LLONG_MAX);
}
break;
case 'F':
specfile = optarg;
break;
case 'M':
fsoptions.minsize = strsuftoll("minimum size",
optarg, 1LL, LLONG_MAX);
break;
case 'm':
fsoptions.maxsize = strsuftoll("maximum size",
optarg, 1LL, LLONG_MAX);
break;
case 'o':
{
char *p;
while ((p = strsep(&optarg, ",")) != NULL) {
if (*p == '\0')
errx(1, "Empty option");
if (! fstype->parse_options(p, &fsoptions))
usage();
}
break;
}
case 's':
fsoptions.minsize = fsoptions.maxsize =
strsuftoll("size", optarg, 1LL, LLONG_MAX);
break;
case 'S':
fsoptions.sectorsize = strsuftoll("sector size",
optarg, 1LL, LLONG_MAX);
break;
case 't':
if ((fstype = get_fstype(optarg)) == NULL)
errx(1, "Unknown fs type `%s'.", optarg);
break;
case '?':
default:
usage();
/* NOTREACHED */
}
}
if (debug) {
printf("debug mask: 0x%08x\n", debug);
printf("start time: %ld.%ld, %s",
start_time.tv_sec, start_time.tv_nsec,
ctime(&start_time.tv_sec));
}
argc -= optind;
argv += optind;
if (argc != 2)
usage();
/* walk the tree */
TIMER_START(start);
root = walk_dir(argv[1], NULL);
TIMER_RESULTS(start, "walk_dir");
if (specfile) { /* apply a specfile */
TIMER_START(start);
apply_specfile(specfile, argv[1], root);
TIMER_RESULTS(start, "apply_specfile");
}
if (debug & DEBUG_DUMP_FSNODES) {
putchar('\n');
dump_fsnodes(argv[1], root);
putchar('\n');
}
/* build the file system */
TIMER_START(start);
fstype->make_fs(argv[0], argv[1], root, &fsoptions);
TIMER_RESULTS(start, "make_fs");
exit(0);
/* NOTREACHED */
}
int
set_option(option_t *options, const char *var, const char *val)
{
int i;
for (i = 0; options[i].name != NULL; i++) {
if (strcmp(options[i].name, var) != 0)
continue;
*options[i].value = (int)strsuftoll(options[i].desc, val,
options[i].minimum, options[i].maximum);
return (1);
}
warnx("Unknown option `%s'", var);
return (0);
}
static fstype_t *
get_fstype(const char *type)
{
int i;
for (i = 0; fstypes[i].type != NULL; i++)
if (strcmp(fstypes[i].type, type) == 0)
return (&fstypes[i]);
return (NULL);
}
static long long
strsuftoll(const char *desc, const char *arg, long long min, long long max)
{
long long result;
char *ep;
assert(desc != NULL);
assert(arg != NULL);
errno = 0;
result = strtoll(arg, &ep, 0);
if ((result == LLONG_MIN || result == LLONG_MAX) && errno == ERANGE) {
warn("%s `%s'", desc, arg);
usage();
}
if (ep[0] != '\0' && ep[1] != '\0') {
warnx("`%s' is not a valid number for %s.", optarg, desc);
usage();
}
switch (tolower((unsigned char)ep[0])) {
case '\0':
case 'b':
break;
case 'k':
result <<= 10;
break;
case 'm':
result <<= 20;
break;
case 'g':
result <<= 30;
break;
default:
warnx("`%s' is not a valid suffix for %s.", ep, desc);
usage();
}
if (result < min) {
warnx("%s `%s' (%lld) is less than %lld.",
desc, optarg, result, min);
usage();
}
if (result > max) {
warnx("%s `%s' (%lld) is greater than %lld.",
desc, optarg, result, max);
usage();
}
if (debug & DEBUG_STRSUFTOLL)
printf("strsuftoll: got %lld for %s %s\n",
result, desc, arg);
return (result);
}
static void
usage(void)
{
const char *prog;
prog = getprogname();
fprintf(stderr,
"Usage: %s [-t fs-type] [-o fs-options] [-d debug-mask] [-B endian]\n"
"\t[-S sector-size] [-M minimum-size] [-m maximum-size] [-s image-size]\n"
"\t[-b free-blocks] [-f free-files] [-F mtree-specfile]\n"
"\timage-file directory\n",
prog);
exit(1);
}