Aren/NetBSD
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Use more markup.

Browse Source
This commit is contained in:
wiz 2002-10-01 14:20:26 +00:00
parent 4a05361ffa
commit 1233505082
1 changed files with 150 additions and 115 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

265
sbin/raidctl/raidctl.8
View File

@ -1,6 +1,6 @@
.\" $NetBSD: raidctl.8,v 1.31 2002/10/01 13:40:49 wiz Exp $
.\" $NetBSD: raidctl.8,v 1.32 2002/10/01 14:20:26 wiz Exp $
.\"
.\" Copyright (c) 1998 The NetBSD Foundation, Inc.
.\" Copyright (c) 1998, 2002 The NetBSD Foundation, Inc.
.\" All rights reserved.
.\"
.\" This code is derived from software contributed to The NetBSD Foundation
@ -123,11 +123,11 @@
.Op Fl v
.Fl u Ar dev
.Sh DESCRIPTION
.Nm ""
.Nm
is the user-land control program for
.Xr raid 4 ,
the RAIDframe disk device.
.Nm ""
.Nm
is primarily used to dynamically configure and unconfigure RAIDframe disk
devices.
For more information about the RAIDframe disk device, see
@ -150,7 +150,9 @@ Make the RAID set auto-configurable.
The RAID set will be automatically configured at boot
.Ar before
the root file system is mounted.
Note that all components of the set must be of type RAID in the disklabel.
Note that all components of the set must be of type
.Dv RAID
in the disklabel.
.It Fl A Ic no Ar dev
Turn off auto-configuration for the RAID set.
.It Fl A Ic root Ar dev
@ -159,7 +161,9 @@ eligible to be the root partition.
A RAID set configured this way will
.Ar override
the use of the boot disk as the root device.
All components of the set must be of type RAID in the disklabel.
All components of the set must be of type
.Dv RAID
in the disklabel.
Note that the kernel being booted must currently reside on a non-RAID set.
.It Fl B Ar dev
Initiate a copyback of reconstructed data from a spare disk to
@ -176,14 +180,13 @@ A description of the contents of
is given later.
.It Fl C Ar config_file Ar dev
As for
.Ar -c ,
.Fl c ,
but forces the configuration to take place.
This is required the first time a RAID set is configured.
.It Fl f Ar component Ar dev
This marks the specified
.Ar component
as having failed, but does not initiate a reconstruction of that
component.
as having failed, but does not initiate a reconstruction of that component.
.It Fl F Ar component Ar dev
Fails the specified
.Ar component
@ -195,18 +198,18 @@ the reconstruction process if a component does have a hardware failure.
Get the component label for the specified component.
.It Fl G Ar dev
Generate the configuration of the RAIDframe device in a format suitable for
use with
.Nm
use with the
.Fl c
or
.Fl C .
.Fl C
options.
.It Fl i Ar dev
Initialize the RAID device.
In particular, (re-write) the parity on the selected device.
In particular, (re-)write the parity on the selected device.
This
.Ar MUST
.Em MUST
be done for
.Ar all
.Em all
RAID sets before the RAID device is labeled and before
file systems are created on the RAID device.
.It Fl I Ar serial_number Ar dev
@ -217,7 +220,7 @@ particular set of components belong to the same RAID set.
While not strictly enforced, different serial numbers should be used for
different RAID sets.
This step
.Ar MUST
.Em MUST
be performed when a new RAID set is created.
.It Fl p Ar dev
Check the status of the parity on the RAID set.
@ -261,9 +264,14 @@ The device used by
is specified by
.Ar dev .
.Ar dev
may be either the full name of the device, e.g. /dev/rraid0d,
for the i386 architecture, and /dev/rraid0c
for all others, or just simply raid0 (for /dev/rraid0d).
may be either the full name of the device, e.g.,
.Pa /dev/rraid0d ,
for the i386 architecture, or
.Pa /dev/rraid0c
for many others, or just simply
.Pa raid0
(for
.Pa /dev/rraid0[cd] ) .
.Ss Configuration file
The format of the configuration file is complex, and
only an abbreviated treatment is given here.
@ -289,7 +297,7 @@ START array
.Pp
indicates an array with 1 row, 3 columns, and 0 spare disks.
Note that although multi-dimensional arrays may be specified, they are
.Ar NOT
.Em NOT
supported in the driver.
.Pp
The second section, the
@ -309,7 +317,7 @@ configured, then they will be marked as
.Sq failed ,
and the system will operate in degraded mode.
Note that it is
.Ar imperative
.Em imperative
that the order of the components in the configuration file does not
change between configurations of a RAID device.
Changing the order of the components will result in data loss
@ -357,7 +365,7 @@ START layout
.Ed
.Pp
The sectors per stripe unit specifies, in blocks, the interleave
factor; i.e. the number of contiguous sectors to be written to each
factor; i.e., the number of contiguous sectors to be written to each
component for a single stripe.
Appropriate selection of this value (32 in this example)
is the subject of much research in RAID architectures.
@ -379,12 +387,10 @@ Mirroring.
The parity is the mirror.
.It 4
RAID level 4.
Striping across components, with parity stored on the
last component.
Striping across components, with parity stored on the last component.
.It 5
RAID level 5.
Striping across components, parity distributed across
all components.
Striping across components, parity distributed across all components.
.El
.Pp
There are other valid entries here, including those for Even-Odd
@ -429,8 +435,7 @@ It is highly recommended that before using the RAID driver for real
file systems that the system administrator(s) become quite familiar
with the use of
.Nm "" ,
and that they understand how the component reconstruction process
works.
and that they understand how the component reconstruction process works.
The examples in this section will focus on configuring a
number of different RAID sets of varying degrees of redundancy.
By working through these examples, administrators should be able to
@ -441,19 +446,18 @@ In the following examples
.Sq raid0
will be used to denote the RAID device.
Depending on the architecture,
.Sq /dev/rraid0c
.Pa /dev/rraid0c
or
.Sq /dev/rraid0d
.Pa /dev/rraid0d
may be used in place of
.Sq raid0 .
.Pa raid0 .
.Ss Initialization and Configuration
The initial step in configuring a RAID set is to identify the components
that will be used in the RAID set.
All components should be the same size.
Each component should have a disklabel type of
.Dv FS_RAID ,
and a typical disklabel entry for a RAID component
might look like:
and a typical disklabel entry for a RAID component might look like:
.Bd -literal -offset indent
f: 1800000 200495 RAID # (Cyl. 405*- 4041*)
.Ed
@ -471,8 +475,9 @@ A
.Sq component label
contains important information about the component, including a
user-specified serial number, the row and column of that component in
the RAID set, the redundancy level of the RAID set, a 'modification
counter', and whether the parity information (if any) on that
the RAID set, the redundancy level of the RAID set, a
.Sq modification counter ,
and whether the parity information (if any) on that
component is known to be correct.
Component labels are an integral part of the RAID set,
since they are used to ensure that components
@ -494,12 +499,11 @@ for more information about component labels.
.Pp
Once the components have been identified, and the disks have
appropriate labels,
.Nm ""
.Nm
is then used to configure the
.Xr raid 4
device.
To configure the device, a configuration file
which looks something like:
To configure the device, a configuration file which looks something like:
.Bd -literal -offset indent
START array
# numRow numCol numSpare
@ -523,8 +527,14 @@ fifo 100
.Pp
is created in a file.
The above configuration file specifies a RAID 5
set consisting of the components /dev/sd1e, /dev/sd2e, and /dev/sd3e,
with /dev/sd4e available as a
set consisting of the components
.Pa /dev/sd1e ,
.Pa /dev/sd2e ,
and
.Pa /dev/sd3e ,
with
.Pa /dev/sd4e
available as a
.Sq hot spare
in case one of the three main drives should fail.
A RAID 0 set would be specified in a similar way:
@ -547,7 +557,12 @@ START queue
fifo 100
.Ed
.Pp
In this case, devices /dev/sd10e, /dev/sd11e, /dev/sd12e, and /dev/sd13e
In this case, devices
.Pa /dev/sd10e ,
.Pa /dev/sd11e ,
.Pa /dev/sd12e ,
and
.Pa /dev/sd13e
are the components that make up this RAID set.
Note that there are no hot spares for a RAID 0 set,
since there is no way to recover data if any of the components fail.
@ -570,13 +585,15 @@ START queue
fifo 100
.Ed
.Pp
In this case, /dev/sd20e and /dev/sd21e are the two components of the
mirror set.
In this case,
.Pa /dev/sd20e
and
.Pa /dev/sd21e
are the two components of the mirror set.
While no hot spares have been specified in this
configuration, they easily could be, just as they were specified in
the RAID 5 case above.
Note as well that RAID 1 sets are currently
limited to only 2 components.
Note as well that RAID 1 sets are currently limited to only 2 components.
At present, n-way mirroring is not possible.
.Pp
The first time a RAID set is configured, the
@ -587,7 +604,7 @@ raidctl -C raid0.conf raid0
.Ed
.Pp
where
.Sq raid0.conf
.Pa raid0.conf
is the name of the RAID configuration file.
The
.Fl C
@ -619,10 +636,10 @@ where
.Sq 112341
is a user-specified serial number for the RAID set.
This initialization step is
.Ar required
.Em required
for all RAID sets.
As well, using different serial numbers between RAID sets is
.Ar strongly encouraged ,
.Em strongly encouraged ,
as using the same serial number for all RAID sets will only serve to
decrease the usefulness of the component label checking.
.Pp
@ -630,9 +647,9 @@ Initializing the RAID set is done via the
.Fl i
option.
This initialization
.Ar MUST
.Em MUST
be done for
.Ar all
.Em all
RAID sets, since among other things it verifies that the parity (if
any) on the RAID set is correct.
Since this initialization may be quite time-consuming, the
@ -658,13 +675,11 @@ to completion of the operation.
.Pp
Since it is the parity that provides the
.Sq redundancy
part of RAID, it is critical that the parity is correct
as much as possible.
part of RAID, it is critical that the parity is correct as much as possible.
If the parity is not correct, then there is no
guarantee that data will not be lost if a component fails.
.Pp
Once the parity is known to be correct,
it is then safe to perform
Once the parity is known to be correct, it is then safe to perform
.Xr disklabel 8 ,
.Xr newfs 8 ,
or
@ -672,13 +687,12 @@ or
on the device or its file systems, and then to mount the file systems
for use.
.Pp
Under certain circumstances (e.g. the additional component has not
Under certain circumstances (e.g., the additional component has not
arrived, or data is being migrated off of a disk destined to become a
component) it may be desirable to configure a RAID 1 set with only
a single component.
This can be achieved by configuring the set with
a physically existing component (as either the first or second
component) and with a
This can be achieved by configuring the set with a physically existing
component (as either the first or second component) and with a
.Sq fake
component.
In the following:
@ -699,10 +713,12 @@ START queue
fifo 100
.Ed
.Pp
/dev/sd0e is the real component, and will be the second disk of a RAID 1
set.
The component /dev/sd6e, which must exist, but have no physical
device associated with it, is simply used as a placeholder.
.Pa /dev/sd0e
is the real component, and will be the second disk of a RAID 1 set.
The component
.Pa /dev/sd6e ,
which must exist, but have no physical device associated with it,
is simply used as a placeholder.
Configuration (using
.Fl C
and
@ -728,12 +744,11 @@ raidctl -P raid0
.Pp
is used.
Note that re-writing the parity can be done while
other operations on the RAID set are taking place (e.g. while doing a
other operations on the RAID set are taking place (e.g., while doing a
.Xr fsck 8
on a file system on the RAID set).
However: for maximum effectiveness of the RAID set,
the parity should be known to be correct before any
data on the set is modified.
However: for maximum effectiveness of the RAID set, the parity should be
known to be correct before any data on the set is modified.
.Pp
To see how the RAID set is doing, the following command can be used to
show the RAID set's status:
@ -790,7 +805,9 @@ the individual components will not be
.Sq clean
but the set as a whole can still be clean.
.Pp
To check the component label of /dev/sd1e, the following is used:
To check the component label of
.Pa /dev/sd1e ,
the following is used:
.Bd -literal -offset indent
raidctl -g /dev/sd1e raid0
.Ed
@ -880,8 +897,9 @@ Copyback is 100% complete.
.Ed
.Pp
At this point there are at least two options.
First, if /dev/sd2e is known to be good (i.e. the failure
was either caused by
First, if
.Pa /dev/sd2e
is known to be good (i.e., the failure was either caused by
.Fl f
or
.Fl F ,
@ -890,7 +908,9 @@ be initiated with the
.Fl B
option.
In this example, this would copy the entire contents of
/dev/sd4e to /dev/sd2e.
.Pa /dev/sd4e
to
.Pa /dev/sd2e .
Once the copyback procedure is complete, the
status of the device would be (in part):
.Bd -literal -offset indent
@ -904,8 +924,11 @@ Spares:
.Pp
and the system is back to normal operation.
.Pp
The second option after the reconstruction is to simply use /dev/sd4e
in place of /dev/sd2e in the configuration file.
The second option after the reconstruction is to simply use
.Pa /dev/sd4e
in place of
.Pa /dev/sd2e
in the configuration file.
For example, the configuration file (in part) might now look like:
.Bd -literal -offset indent
START array
@ -917,14 +940,17 @@ START drives
/dev/sd3e
.Ed
.Pp
This can be done as /dev/sd4e is completely interchangeable with
/dev/sd2e at this point.
This can be done as
.Pa /dev/sd4e
is completely interchangeable with
.Pa /dev/sd2e
at this point.
Note that extreme care must be taken when
changing the order of the drives in a configuration.
This is one of the few instances where the devices and/or
their orderings can be changed without loss of data!
In general, the ordering of components in a configuration file should
.Ar never
.Em never
be changed.
.Pp
If a component fails and there are no hot spares
@ -957,14 +983,18 @@ Reconstruction could then take place using
.Fl F
as describe above.
.Pp
A second option is to rebuild directly onto /dev/sd2e.
Once the disk containing /dev/sd2e has been replaced,
one can simply use:
A second option is to rebuild directly onto
.Pa /dev/sd2e .
Once the disk containing
.Pa /dev/sd2e
has been replaced, one can simply use:
.Bd -literal -offset indent
raidctl -R /dev/sd2e raid0
.Ed
.Pp
to rebuild the /dev/sd2e component.
to rebuild the
.Pa /dev/sd2e
component.
As the rebuilding is in progress, the status will be:
.Bd -literal -offset indent
Components:
@ -1008,11 +1038,12 @@ raidctl -F component1 raid0
.Pp
at which point the data missing from
.Sq component1
would be reconstructed onto /dev/sd3e.
would be reconstructed onto
.Pa /dev/sd3e .
.Pp
When more than one component is marked as
.Sq failed
due to a non-component hardware failure (e.g. loss of power to two
due to a non-component hardware failure (e.g., loss of power to two
components, adapter problems, termination problems, or cabling issues) it
is quite possible to recover the data on the RAID set.
The first thing to be aware of is that the first disk to fail will
@ -1022,7 +1053,7 @@ If any IO was performed between the time the first component is considered
and when the second component is considered
.Sq failed ,
then the first component to fail will
.Ar not
.Em not
contain correct data, and should be ignored.
When the second component is marked as failed, however, the RAID device will
(currently) panic the system.
@ -1030,10 +1061,10 @@ At this point the data on the RAID set
(not including the first failed component) is still self consistent,
and will be in no worse state of repair than had the power gone out in
the middle of a write to a filesystem on a non-RAID device.
The problem, however, is that the component labels may now have 3
different 'modification counters' (one value on the first component
that failed, one value on the second component that failed, and a
third value on the remaining components).
The problem, however, is that the component labels may now have 3 different
.Sq modification counters
(one value on the first component that failed, one value on the second
component that failed, and a third value on the remaining components).
In such a situation, the RAID set will not autoconfigure,
and can only be forcibly re-configured
with the
@ -1043,11 +1074,13 @@ To recover the RAID set, one must first remedy whatever physical
problem caused the multiple-component failure.
After that is done, the RAID set can be restored by forcibly
configuring the raid set
.Ar without
.Em without
the component that failed first.
For example, if /dev/sd1e and
/dev/sd2e fail (in that order) in a RAID set of the following
configuration:
For example, if
.Pa /dev/sd1e
and
.Pa /dev/sd2e
fail (in that order) in a RAID set of the following configuration:
.Bd -literal -offset indent
START array
1 4 0
@ -1086,7 +1119,9 @@ START queue
fifo 100
.Ed
.Pp
(where /dev/sd6e has no physical device) can be used with
(where
.Pa /dev/sd6e
has no physical device) can be used with
.Bd -literal -offset indent
raidctl -C recover_raid0.conf raid0
.Ed
@ -1101,13 +1136,12 @@ will be required in order to synchronize the component labels.
At this point the filesystems on the RAID set can then be checked and
corrected.
To complete the re-construction of the RAID set,
/dev/sd1e is simply hot-added back into the array, and reconstructed
.Pa /dev/sd1e
is simply hot-added back into the array, and reconstructed
as described earlier.
.Ss RAID on RAID
RAID sets can be layered to create more complex and much larger RAID
sets.
A RAID 0 set, for example, could be constructed from four RAID
5 sets.
RAID sets can be layered to create more complex and much larger RAID sets.
A RAID 0 set, for example, could be constructed from four RAID 5 sets.
The following configuration file shows such a setup:
.Bd -literal -offset indent
START array
@ -1158,8 +1192,7 @@ have become scrambled.
.Pp
Having a system's root file system
.Pq Pa /
on a RAID set is also allowed,
with the
on a RAID set is also allowed, with the
.Sq a
partition of such a RAID set being used for
.Pa / .
@ -1180,9 +1213,8 @@ file system is recognized by the bootblocks, and will properly load the
kernel directly from a RAID 1 component.
For other architectures, or to support the root file system
on other RAID sets, some other mechanism must be used to get a kernel booting.
For example, a small
partition containing only the secondary boot-blocks and an alternate
kernel (or two) could be used.
For example, a small partition containing only the secondary boot-blocks
and an alternate kernel (or two) could be used.
Once a kernel is booting however, and an auto-configuring RAID set is
found that is eligible to be root, then that RAID set will be
auto-configured and used as the root device.
@ -1231,20 +1263,19 @@ as the kernel.
For example, obtaining the kernel from wd0a, and using
sd0e and sd1e for raid0, and the root file system, is fine.
It
.Ar is
.Em is
critical, however, that there be multiple kernels available, in the
event of media failure.
.Pp
Multi-layered RAID devices (such as a RAID 0 set made
up of RAID 1 sets) are
.Ar not
.Em not
supported as root devices or auto-configurable devices at this point.
(Multi-layered RAID devices
.Ar are
.Em are
supported in general, however, as mentioned earlier.)
Note that in
order to enable component auto-detection and auto-configuration of
RAID devices, the line:
Note that in order to enable component auto-detection and
auto-configuration of RAID devices, the line:
.Bd -literal -offset indent
options RAID_AUTOCONFIG
.Ed
@ -1272,7 +1303,7 @@ trial-and-error to get those values most appropriate for a given system.
A whole range of factors come into play, including:
.Bl -enum
.It
Types of components (e.g. SCSI vs. IDE) and their bandwidth
Types of components (e.g., SCSI vs. IDE) and their bandwidth
.It
Types of controller cards and their bandwidth
.It
@ -1359,8 +1390,8 @@ Use
.Xr disklabel 8
to create the components (of type RAID).
.It
Construct a RAID configuration file: e.g.
.Sq raid0.conf
Construct a RAID configuration file: e.g.,
.Pa raid0.conf
.It
Configure the RAID set with:
.Bd -literal -offset indent
@ -1416,8 +1447,12 @@ raidctl -c raid0.conf raid0
.Ed
.Pp
To re-configure the RAID set the next time it is needed, or put
raid0.conf into /etc where it will automatically be started by
the /etc/rc scripts.
.Pa raid0.conf
into
.Pa /etc
where it will automatically be started by the
.Pa /etc/rc.d
scripts.
.El
.Sh SEE ALSO
.Xr ccd 4 ,
@ -1474,16 +1509,16 @@ However the loss of two components of a RAID 4 or 5 system,
or the loss of a single component of a RAID 0 system will
result in the entire file system being lost.
RAID is
.Ar NOT
.Em NOT
a substitute for good backup practices.
.Pp
Recomputation of parity
.Ar MUST
.Em MUST
be performed whenever there is a chance that it may have been compromised.
This includes after system crashes, or before a RAID
device has been used for the first time.
Failure to keep parity correct will be catastrophic should a
component ever fail -- it is better to use RAID 0 and get the
component ever fail \(em it is better to use RAID 0 and get the
additional space and speed, than it is to use parity, but
not keep the parity correct.
At least with RAID 0 there is no perception of increased data security.