258 lines
7.3 KiB
Groff
258 lines
7.3 KiB
Groff
.\" $NetBSD: vndcompress.1,v 1.15 2020/01/24 20:20:35 pgoyette Exp $
|
|
.\"
|
|
.\" Copyright (c) 2013 The NetBSD Foundation, Inc.
|
|
.\" All rights reserved.
|
|
.\"
|
|
.\" This documentation is derived from text contributed to The NetBSD
|
|
.\" Foundation by Taylor R. Campbell.
|
|
.\"
|
|
.\" Redistribution and use in source and binary forms, with or without
|
|
.\" modification, are permitted provided that the following conditions
|
|
.\" are met:
|
|
.\" 1. Redistributions of source code must retain the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer.
|
|
.\" 2. Redistributions in binary form must reproduce the above copyright
|
|
.\" notice, this list of conditions and the following disclaimer in the
|
|
.\" documentation and/or other materials provided with the distribution.
|
|
.\"
|
|
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
|
|
.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
|
|
.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
|
.\" PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
|
|
.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
|
.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
|
.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
|
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
|
.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
|
.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
|
.\" POSSIBILITY OF SUCH DAMAGE.
|
|
.\"
|
|
.Dd January 24, 2020
|
|
.Dt VNDCOMPRESS 1
|
|
.Os
|
|
.Sh NAME
|
|
.Nm vndcompress ,
|
|
.Nm vnduncompress
|
|
.Nd compress and uncompress disk images in cloop2 format
|
|
.Sh SYNOPSIS
|
|
.Nm
|
|
.Op Fl c
|
|
.Op Fl rR
|
|
.Op Fl b Ar blocksize
|
|
.Op Fl k Ar checkpoint-blocks
|
|
.Op Fl l Ar length
|
|
.Op Fl p Ar partial-offset
|
|
.Op Fl w Ar winsize
|
|
.Ar image
|
|
.Ar compressed-image
|
|
.Op Ar blocksize
|
|
.Nm vnduncompress
|
|
.Op Fl d
|
|
.Op Fl w Ar winsize
|
|
.Ar compressed-image
|
|
.Ar image
|
|
.Sh DESCRIPTION
|
|
The
|
|
.Nm
|
|
utility compresses disk images in cloop2 format, which the kernel's
|
|
.Xr vnd 4
|
|
device can interpret as read-only disk devices using the
|
|
.Fl z
|
|
option to
|
|
.Xr vndconfig 8 .
|
|
.Pp
|
|
By default,
|
|
.Nm vndcompress
|
|
compresses an image, and
|
|
.Nm vnduncompress
|
|
uncompresses an image, but the
|
|
.Fl c
|
|
and
|
|
.Fl d
|
|
options can control whether either utility compresses or decompresses.
|
|
.Pp
|
|
The following options are available for the compression operation:
|
|
.Bl -tag -width indent
|
|
.It Fl b Ar blocksize
|
|
Set the compression block size to
|
|
.Ar blocksize ,
|
|
which must be a multiple of 512 and must be no more than 4294966784, or
|
|
0xfffffe00.
|
|
(On 32-bit systems, the limit may be smaller, limited by the available
|
|
virtual address space.)
|
|
.Pp
|
|
For compatibility with the old version of
|
|
.Nm ,
|
|
the compression block size may instead be specified at the end of the
|
|
command line.
|
|
.It Fl k Ar checkpoint-blocks
|
|
Write a checkpoint after every
|
|
.Ar checkpoint-blocks
|
|
blocks of output.
|
|
If interrupted,
|
|
.Nm
|
|
can restart at the last checkpoint with the
|
|
.Fl r
|
|
option.
|
|
You can also request a checkpoint at any time by sending
|
|
.Dv SIGUSR2
|
|
to the
|
|
.Nm
|
|
process.
|
|
.It Fl l Ar length
|
|
Specify the length of the input, so that the input may be a pipe,
|
|
socket, or FIFO.
|
|
Otherwise, the input must know its size, i.e. must have its size
|
|
reported in
|
|
.Fa st_size
|
|
by
|
|
.Xr fstat 2 .
|
|
.It Fl p Ar partial-blocks
|
|
Stop after writing
|
|
.Ar partial-blocks
|
|
blocks of output.
|
|
This option is mainly useful for automatic testing of restarts.
|
|
.It Fl R
|
|
With the
|
|
.Fl r
|
|
option, if restarting fails, then abort right now and don't touch the
|
|
output file.
|
|
.It Fl r
|
|
Try to restart a partial compression from the last checkpoint.
|
|
If restarting fails, and the
|
|
.Fl R
|
|
option is not specified, then truncate the output file and start
|
|
compression afresh.
|
|
Restarting may fail for various reasons: if there have been no
|
|
checkpoints, or if the output file has been corrupted in some easily
|
|
recognizable ways, or if the input file's size has changed, or if the
|
|
block size does not match, and so on.
|
|
.El
|
|
.Pp
|
|
The following option is available for both compression and
|
|
decompression:
|
|
.Bl -tag -width indent
|
|
.It Fl w Ar winsize
|
|
Use an in-memory window of
|
|
.Ar winsize
|
|
entries into the table of compressed block offsets.
|
|
If
|
|
.Ar winsize
|
|
is zero,
|
|
.Nm
|
|
will use memory proportional to the number of blocks in the
|
|
uncompressed image, namely 64 bits or 8 bytes per block.
|
|
If
|
|
.Ar winsize
|
|
is nonzero,
|
|
.Nm
|
|
will use memory proportional to
|
|
.Ar winsize ,
|
|
and independent of the size of the uncompressed image.
|
|
.Pp
|
|
A nonzero
|
|
.Ar winsize
|
|
requires the compressed image to be a seekable file, which compression
|
|
requires anyway, in order to record the offsets of compressed blocks
|
|
once they are compressed and written, but which is a limitation for
|
|
decompression.
|
|
Thus, decompressing from a pipe is incompatible with a nonzero
|
|
.Ar winsize .
|
|
.Pp
|
|
By default,
|
|
.Nm
|
|
uses a fixed window size, unless decompressing with nonseekable input.
|
|
.El
|
|
.Sh EXIT STATUS
|
|
.Ex -std
|
|
.Sh EXAMPLES
|
|
Compress an ISO 9660 CD-ROM image:
|
|
.Bd -literal -offset indent
|
|
# vndcompress cdrom.iso cdrom.izo
|
|
.Ed
|
|
.Pp
|
|
Send a 59 GB disk partition over a local network with netcat (don't do
|
|
this over the internet!):
|
|
.Bd -literal -offset indent
|
|
# nc 10.0.1.2 12345 < /dev/rcgd1h
|
|
.Ed
|
|
.Pp
|
|
Receive it and save it compressed on another host, showing a progress
|
|
bar interactively, restarting if possible, and taking a checkpoint
|
|
every 16 MB (i.e., every 256 compression blocks, which are 64 KB by
|
|
default):
|
|
.Bd -literal -offset indent
|
|
# nc -l 12345 | progress -l 59g \e
|
|
vndcompress -l 59g -k 256 -r /dev/stdin disk.cloop2
|
|
.Ed
|
|
.Pp
|
|
If the process is interrupted and you rewire your network and disks so
|
|
that the receiving host now has the disk you want to image, you can
|
|
start up where you left off, using the
|
|
.Fl R
|
|
option to keep
|
|
.Nm
|
|
from clobbering the partial transfer if anything goes wrong:
|
|
.Bd -literal -offset indent
|
|
# vndcompress -l 59g -k 256 -rR /dev/rcgd1h disk.cloop2
|
|
.Ed
|
|
.Pp
|
|
Mount the disk with
|
|
.Xr vnd 4 ,
|
|
assuming your kernel was built with the
|
|
.Dv VND_COMPRESSION
|
|
option enabled:
|
|
.Bd -literal -offset indent
|
|
# vndconfig -z vnd0 disk.cloop2
|
|
# mount /dev/vnd0d /mnt
|
|
.Ed
|
|
.Sh SIGNALS
|
|
.Nm
|
|
responds to the following signals:
|
|
.Bl -tag -width indent
|
|
.It Dv SIGINFO , Dv SIGUSR1
|
|
Report progress to standard error.
|
|
.It Dv SIGUSR2
|
|
Write a checkpoint to disk now.
|
|
.El
|
|
.Sh FORMAT
|
|
The cloop2 format consists of a header, an offset table, and a sequence
|
|
of compressed blocks.
|
|
The header is described by the following packed structure:
|
|
.Bd -literal -offset indent
|
|
struct cloop2_header {
|
|
char cl2h_magic[128];
|
|
uint32_t cl2h_blocksize;
|
|
uint32_t cl2h_n_blocks;
|
|
} __packed;
|
|
.Ed
|
|
.Pp
|
|
The
|
|
.Fa cl2h_magic
|
|
field is an arbitrary sequence of 128 bits, the
|
|
.Fa cl2h_blocksize
|
|
field is a big-endian integer number of bytes per compression block,
|
|
and the
|
|
.Fa cl2h_n_blocks
|
|
field is a big-endian integer number of the compression blocks in the
|
|
image.
|
|
.Pp
|
|
The offset table is a sequence of one more than
|
|
.Fa cl2h_n_blocks
|
|
big-endian 64-bit integers specifying the offset of each compression
|
|
block relative to the start of the file.
|
|
The extra offset specifies the end of the last compression block, which
|
|
may be truncated if the uncompressed image's size is not a multiple of
|
|
the compression block size.
|
|
.Sh SEE ALSO
|
|
.Xr vnd 4 ,
|
|
.Xr vndconfig 8
|
|
.Sh HISTORY
|
|
The
|
|
.Nm
|
|
command first appeared in
|
|
.Nx 3.0 .
|
|
It was rewritten to be more robust, to support restarting partial
|
|
transfers, and to support bounded memory usage in
|
|
.Nx 7.0 .
|