haiku/build/scripts/build_haiku_image

365 lines
8.5 KiB
Plaintext
Raw Normal View History

#!/bin/sh
set -o errexit
# The first argument is the shell script that initializes the variables:
# sourceDir
# outputDir
# tmpDir
# addBuildCompatibilityLibDir
# systemPackages - lists of the hpkg packages copied/updated
# repositories - all repository files
Switch build system from optional package to repositories * Build libsolv and the dependency solver part of the package kit for the build platform. * Add build tool get_package_dependencies. Given a list of package files and a list of repository files it determines the additional packages that need to be retrieved from the repositories and prints their URLs. * Add rules to work with external repositories in the build system (build/jam/RepositoryRules): - PackageRepository declares an external repository with all its packages. The URL of the repository file isn't specified. It is computed from a given base URL and the SHA256 hash of the list of package files. - GeneratedRepositoryPackageList generates a file containing the file names of all packages in a repository. - IsPackageAvailable returns whether a package is available in any repository. - PackageURL returns the URL for a package. * Declare the HaikuPorts repository for x86_gcc2 (build/jam/repositories/HaikuPorts/x86_gcc2). * Add rule AddHaikuImagePackages to add a package to the image and rule IsHaikuImagePackageAdded to determine whether a package has been added. * OptionalPackages: Remove all entries that just downloaded and installed an external package. AddHaikuImagePackages can be used instead and is used in the remaining entries. Also move the remaining optional package dependency declarations from OptionalPackageDependencies here. * ExtractBuildFeatureArchives: Instead of the URL parameter a package name must be specified now. This allows to simplify BuildFeatures significantly, since there's no dealing with URLs anymore. "if" out the entries that aren't supported yet. * build_haiku_image: For the packages installed in system and common resolve their dependencies and download and install them as well.
2013-07-05 12:51:42 +04:00
# downloadDir
# The following are only for image types:
# installDir
# isImage
# imagePath
# imageSize
# imageLabel
Switch build system from optional package to repositories * Build libsolv and the dependency solver part of the package kit for the build platform. * Add build tool get_package_dependencies. Given a list of package files and a list of repository files it determines the additional packages that need to be retrieved from the repositories and prints their URLs. * Add rules to work with external repositories in the build system (build/jam/RepositoryRules): - PackageRepository declares an external repository with all its packages. The URL of the repository file isn't specified. It is computed from a given base URL and the SHA256 hash of the list of package files. - GeneratedRepositoryPackageList generates a file containing the file names of all packages in a repository. - IsPackageAvailable returns whether a package is available in any repository. - PackageURL returns the URL for a package. * Declare the HaikuPorts repository for x86_gcc2 (build/jam/repositories/HaikuPorts/x86_gcc2). * Add rule AddHaikuImagePackages to add a package to the image and rule IsHaikuImagePackageAdded to determine whether a package has been added. * OptionalPackages: Remove all entries that just downloaded and installed an external package. AddHaikuImagePackages can be used instead and is used in the remaining entries. Also move the remaining optional package dependency declarations from OptionalPackageDependencies here. * ExtractBuildFeatureArchives: Instead of the URL parameter a package name must be specified now. This allows to simplify BuildFeatures significantly, since there's no dealing with URLs anymore. "if" out the entries that aren't supported yet. * build_haiku_image: For the packages installed in system and common resolve their dependencies and download and install them as well.
2013-07-05 12:51:42 +04:00
# resolvePackageDependencies
# updateAllPackages
# updateOnly
# dontClearImage
# isVMwareImage
#
# addattr
# copyattr
Switch build system from optional package to repositories * Build libsolv and the dependency solver part of the package kit for the build platform. * Add build tool get_package_dependencies. Given a list of package files and a list of repository files it determines the additional packages that need to be retrieved from the repositories and prints their URLs. * Add rules to work with external repositories in the build system (build/jam/RepositoryRules): - PackageRepository declares an external repository with all its packages. The URL of the repository file isn't specified. It is computed from a given base URL and the SHA256 hash of the list of package files. - GeneratedRepositoryPackageList generates a file containing the file names of all packages in a repository. - IsPackageAvailable returns whether a package is available in any repository. - PackageURL returns the URL for a package. * Declare the HaikuPorts repository for x86_gcc2 (build/jam/repositories/HaikuPorts/x86_gcc2). * Add rule AddHaikuImagePackages to add a package to the image and rule IsHaikuImagePackageAdded to determine whether a package has been added. * OptionalPackages: Remove all entries that just downloaded and installed an external package. AddHaikuImagePackages can be used instead and is used in the remaining entries. Also move the remaining optional package dependency declarations from OptionalPackageDependencies here. * ExtractBuildFeatureArchives: Instead of the URL parameter a package name must be specified now. This allows to simplify BuildFeatures significantly, since there's no dealing with URLs anymore. "if" out the entries that aren't supported yet. * build_haiku_image: For the packages installed in system and common resolve their dependencies and download and install them as well.
2013-07-05 12:51:42 +04:00
# getPackageDependencies
# package
# rc
# rmAttrs
# unzip
# The following are only for image types:
# bfsShell
# fsShellCommand
# makebootable
# resattr
# vmdkimage
# The following is only for cd types:
# generate_attribute_stores
# isCD
#
if [ $# -gt 0 ]; then
. $1
shift
fi
if [ ! $isCD ]; then
# If the haiku image path is a symlink resolve it now (makebootable needs the
# path of the actual device path under Linux).
normalizedImagePath=''
if readlink -f "$imagePath" > /dev/null 2>&1 ; then
normalizedImagePath=$(readlink -f "$imagePath")
elif realpath "$imagePath" > /dev/null 2>&1 ; then
normalizedImagePath=$(realpath "$imagePath")
elif greadlink -f "$imagePath" > /dev/null 2>&1 ; then
normalizedImagePath=$(greadlink -f "$imagePath")
fi
if [ -n "$normalizedImagePath" ]; then
imagePath="$normalizedImagePath"
fi
fi
# this adds the build library dir to LD_LIBRARY_PATH
eval "$addBuildCompatibilityLibDir"
# map the shell commands
if [ $isCD ]; then
outputDir=$tmpDir/cdsource
sPrefix=
tPrefix="$outputDir/"
cd=cd
scd=:
cp="$copyattr -d"
copyAttrs="$copyattr"
ln=ln
mkdir=mkdir
rm=rm
elif [ $isImage ]; then
# If FIFOs are used for the communication with the FS shell, prepare them.
if $fsShellCommand --uses-fifos; then
fifoBasePath=/tmp/build_haiku_image-$$-fifo
toFSShellFifo=${fifoBasePath}-to-shell
fromFSShellFifo=${fifoBasePath}-from-shell
rm -f $toFSShellFifo $fromFSShellFifo
mkfifo $toFSShellFifo $fromFSShellFifo
# Open the FIFOs such that they are ready for the fsShellCommand. This
# also makes sure that they remain open until this script exits. When we
# exit while the FS shell is still running and waiting for commands,
# closing of our file descriptors will break the FIFOs and the FS shell
# will exit, too.
# Note: A bit of trickery is needed since opening one end blocks until
# someone opens the other end.
sleep 3<$fromFSShellFifo 1 &
exec 6>$fromFSShellFifo 3<$fromFSShellFifo
sleep 5<$toFSShellFifo 1 &
exec 4>$toFSShellFifo 5<$toFSShellFifo
# Remove the FIFO files again -- we have the open FDs, so they can
# still be used and this makes sure they won't hang around any further.
rm -f $toFSShellFifo $fromFSShellFifo
# Remap the fsShellCommand and bfsShell such that they don't inherit the
# wrong FDs. For both fsShellCommand and bfsShell FD 3 is the input from
# the respectively other program, FD 4 is the output to it.
actualFSShellCommand="$fsShellCommand"
actualBFSShell="$bfsShell"
fsShellCommandWrapper()
{
$actualFSShellCommand 5>&- 6>&- "$@"
}
bfsShellWrapper()
{
$actualBFSShell 3>&5 4<&6 "$@"
}
fsShellCommand=fsShellCommandWrapper
bfsShell=bfsShellWrapper
fi
# set up the other commands
sPrefix=:
tPrefix=/myfs/
cd="$fsShellCommand cd"
scd="$fsShellCommand cd"
cp="$fsShellCommand cp -f"
copyAttrs="$fsShellCommand cp -a"
ln="$fsShellCommand ln"
mkdir="$fsShellCommand mkdir"
rm="$fsShellCommand rm"
mkindex="$fsShellCommand mkindex"
else
sPrefix=
# TODO: This should come from the environment.
tPrefix="$installDir/"
cd=cd
scd=:
cp="$copyattr -d"
copyAttrs="$copyattr"
ln=ln
mkdir=mkdir
rm=rm
mkindex=mkindex
fi
is_in_list()
{
local element
for element in $2; do
if [ "$1" = "$element" ]; then
return 0
fi
done
return 1
}
extractFile()
{
# extractFile <archive> <directory> <extractedSubDir>
archiveFile=$1
targetExtractedDir=$2
extractedSubDir=$3
extractDir=$tmpDir/extract
$rmAttrs -rf "$extractDir"
mkdir -p "$extractDir"
case "$archiveFile" in
*.zip)
echo "Extracting $archiveFile ..."
$unzip -q -d "$extractDir" "$archiveFile"
;;
*.tgz|*.tar.gz)
echo "Extracting $archiveFile ..."
tar -C "$extractDir" -xf "$archiveFile"
;;
*.hpkg)
echo "Extracting $archiveFile ..."
if [ -n "$extractedSubDir" ]; then
$package extract -C "$extractDir" "$archiveFile" \
"$extractedSubDir"
else
$package extract -C "$extractDir" "$archiveFile"
fi
;;
*)
echo "Unhandled archive extension in build_haiku_image" \
"extractFile()"
exit 1
;;
esac
if [ -f $extractDir/.OptionalPackageDescription ]; then
cat $extractDir/.OptionalPackageDescription >> $copyrightsFile
echo >> $copyrightsFile
rm $extractDir/.OptionalPackageDescription
fi
$cp -r "${sPrefix}$extractDir/$extractedSubDir/." \
"${tPrefix}$targetExtractedDir"
$rmAttrs -rf "$extractDir"
}
Switch build system from optional package to repositories * Build libsolv and the dependency solver part of the package kit for the build platform. * Add build tool get_package_dependencies. Given a list of package files and a list of repository files it determines the additional packages that need to be retrieved from the repositories and prints their URLs. * Add rules to work with external repositories in the build system (build/jam/RepositoryRules): - PackageRepository declares an external repository with all its packages. The URL of the repository file isn't specified. It is computed from a given base URL and the SHA256 hash of the list of package files. - GeneratedRepositoryPackageList generates a file containing the file names of all packages in a repository. - IsPackageAvailable returns whether a package is available in any repository. - PackageURL returns the URL for a package. * Declare the HaikuPorts repository for x86_gcc2 (build/jam/repositories/HaikuPorts/x86_gcc2). * Add rule AddHaikuImagePackages to add a package to the image and rule IsHaikuImagePackageAdded to determine whether a package has been added. * OptionalPackages: Remove all entries that just downloaded and installed an external package. AddHaikuImagePackages can be used instead and is used in the remaining entries. Also move the remaining optional package dependency declarations from OptionalPackageDependencies here. * ExtractBuildFeatureArchives: Instead of the URL parameter a package name must be specified now. This allows to simplify BuildFeatures significantly, since there's no dealing with URLs anymore. "if" out the entries that aren't supported yet. * build_haiku_image: For the packages installed in system and common resolve their dependencies and download and install them as well.
2013-07-05 12:51:42 +04:00
downloadFile()
{
url=$1
path=$2
if [ ! -f "$path" ]; then
wget -O "$path" "$url"
fi
}
packageFileName()
{
$package info -f "%fileName%" "$1"
}
mkdir -p $tmpDir
copyrightsFile=$tmpDir/copyrights
$rmAttrs -f $copyrightsFile
if [ $isCD ]; then
# setup output dir
$rmAttrs -rf "$outputDir"
mkdir -p "$outputDir"
fi
# create the image and mount it
if [ $isImage ]; then
echo
imageOffsetFlags=
if [ $isVMwareImage ]; then
imageOffsetFlags="--start-offset 65536"
fi
if [ ! $updateOnly ]; then
echo "Creating image ..."
imageFlags="-i${imageSize}M"
if [ ! "$dontClearImage" ]; then
imageFlags="$imageFlags -c"
fi
if [ $isVMwareImage ]; then
$vmdkimage -h 64k $imageFlags "$imagePath"
else
$createImage $imageFlags "$imagePath"
fi
$bfsShell --initialize $imageOffsetFlags "$imagePath" \
"$imageLabel" "block_size 2048"
$makebootable $imageOffsetFlags "$imagePath"
fi
$bfsShell -n $imageOffsetFlags "$imagePath" > /dev/null &
sleep 1
# Close FDs 5 and 6. Those represent the pipe ends that are used by the
# FS shell. Closing them in the shell process makes sure an unexpected death
# of the FS shell causes writing to/reading from the other ends to fail
# immediately.
exec 5>&- 6>&-
# bail out, if mounting fails
$cd .
fi
# Clean out the old packages directory, if updating all packages.
if [ -n "$updateAllPackages" ]; then
echo "Removing old packages ..."
$rm -rf "${tPrefix}system/packages"
$mkdir -p "${tPrefix}system/packages"
fi
echo "Populating image ..."
while [ $# -gt 0 ]; do
. $1
shift
done
Switch build system from optional package to repositories * Build libsolv and the dependency solver part of the package kit for the build platform. * Add build tool get_package_dependencies. Given a list of package files and a list of repository files it determines the additional packages that need to be retrieved from the repositories and prints their URLs. * Add rules to work with external repositories in the build system (build/jam/RepositoryRules): - PackageRepository declares an external repository with all its packages. The URL of the repository file isn't specified. It is computed from a given base URL and the SHA256 hash of the list of package files. - GeneratedRepositoryPackageList generates a file containing the file names of all packages in a repository. - IsPackageAvailable returns whether a package is available in any repository. - PackageURL returns the URL for a package. * Declare the HaikuPorts repository for x86_gcc2 (build/jam/repositories/HaikuPorts/x86_gcc2). * Add rule AddHaikuImagePackages to add a package to the image and rule IsHaikuImagePackageAdded to determine whether a package has been added. * OptionalPackages: Remove all entries that just downloaded and installed an external package. AddHaikuImagePackages can be used instead and is used in the remaining entries. Also move the remaining optional package dependency declarations from OptionalPackageDependencies here. * ExtractBuildFeatureArchives: Instead of the URL parameter a package name must be specified now. This allows to simplify BuildFeatures significantly, since there's no dealing with URLs anymore. "if" out the entries that aren't supported yet. * build_haiku_image: For the packages installed in system and common resolve their dependencies and download and install them as well.
2013-07-05 12:51:42 +04:00
# resolve package dependencies
if [ -n "$resolvePackageDependencies" ]; then
echo "Resolving package dependencies ..."
packageUrls=`$getPackageDependencies $repositories -- $systemPackages`
Switch build system from optional package to repositories * Build libsolv and the dependency solver part of the package kit for the build platform. * Add build tool get_package_dependencies. Given a list of package files and a list of repository files it determines the additional packages that need to be retrieved from the repositories and prints their URLs. * Add rules to work with external repositories in the build system (build/jam/RepositoryRules): - PackageRepository declares an external repository with all its packages. The URL of the repository file isn't specified. It is computed from a given base URL and the SHA256 hash of the list of package files. - GeneratedRepositoryPackageList generates a file containing the file names of all packages in a repository. - IsPackageAvailable returns whether a package is available in any repository. - PackageURL returns the URL for a package. * Declare the HaikuPorts repository for x86_gcc2 (build/jam/repositories/HaikuPorts/x86_gcc2). * Add rule AddHaikuImagePackages to add a package to the image and rule IsHaikuImagePackageAdded to determine whether a package has been added. * OptionalPackages: Remove all entries that just downloaded and installed an external package. AddHaikuImagePackages can be used instead and is used in the remaining entries. Also move the remaining optional package dependency declarations from OptionalPackageDependencies here. * ExtractBuildFeatureArchives: Instead of the URL parameter a package name must be specified now. This allows to simplify BuildFeatures significantly, since there's no dealing with URLs anymore. "if" out the entries that aren't supported yet. * build_haiku_image: For the packages installed in system and common resolve their dependencies and download and install them as well.
2013-07-05 12:51:42 +04:00
for packageUrl in $packageUrls; do
packageFileName=`basename $packageUrl`
packageFilePath="$downloadDir/$packageFileName"
downloadFile $packageUrl "$packageFilePath"
$cp "${sPrefix}$packageFilePath" "${tPrefix}system/packages"
systemPackages="$systemPackages $packageFilePath"
done
fi
# install default settings for packages
for packageFile in $systemPackages; do
if $package list -p $packageFile | egrep '^settings/' > /dev/null; then
extractFile $packageFile system/settings settings
fi
done
# add the concatenated copyrights as an attribute to AboutSystem
# TODO: That might not be necessary, when all third-party software everything
# is packaged. Though we might not package everything.
# if [ ! $updateOnly ]; then
# if [ -f $copyrightsFile ]; then
# copyrightAttrs=$tmpDir/copyrightAttrs
# $rmAttrs -f $copyrightAttrs
# touch $copyrightAttrs
# $addattr -f $copyrightsFile COPYRIGHTS $copyrightAttrs
# $copyAttrs ${sPrefix}$copyrightAttrs ${tPrefix}system/apps/AboutSystem
# fi
# fi
if [ $isCD ]; then
# generate the attribute stores
echo "Generating attribute stores ..."
$generate_attribute_stores "$tPrefix"
echo "Copying boot image ..."
$cp "$cdBootFloppy" "$outputDir"
if [ $(which mkisofs) ]; then
# build the iso image using mkisofs
echo "Building CD image (mkisofs)..."
mkisofs -uid 0 -gid 0 -b `basename $cdBootFloppy` -R -V "$cdLabel" -o "$cdImagePath" "$tPrefix"
elif [ $(which genisoimage) ]; then
# build the iso image using genisoimage
echo "Building CD image (genisoimage)..."
echo "WARNING: genisoimage fallback has known problems with long filenames!"
echo " Please install mkisofs before making production releases!"
genisoimage -r -iso-level 4 -b `basename $cdBootFloppy` -V "$cdLabel" -o "$cdImagePath" "$tPrefix"
else
echo "you need mkisofs (preferred) or genisoimage to create a CD image."
exit 1
fi
# cleanup output dir
$rmAttrs -rf "$outputDir"
fi
# unmount
if [ $isImage ]; then
echo "Unmounting ..."
$fsShellCommand sync
$fsShellCommand quit
fi