qemu/scripts/oss-fuzz/build.sh

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

118 lines
3.9 KiB
Bash
Raw Normal View History

#!/bin/bash -e
#
# OSS-Fuzz build script. See:
# https://google.github.io/oss-fuzz/getting-started/new-project-guide/#buildsh
#
# The file is consumed by:
# https://github.com/google/oss-fuzz/blob/master/projects/qemu/Dockerfiles
#
# This code is licensed under the GPL version 2 or later. See
# the COPYING file in the top-level directory.
#
# build project
# e.g.
# ./autogen.sh
# ./configure
# make -j$(nproc) all
# build fuzzers
# e.g.
# $CXX $CXXFLAGS -std=c++11 -Iinclude \
# /path/to/name_of_fuzzer.cc -o $OUT/name_of_fuzzer \
# -fsanitize=fuzzer /path/to/library.a
fatal () {
echo "Error : ${*}, exiting."
exit 1
}
OSS_FUZZ_BUILD_DIR="./build-oss-fuzz/"
# There seems to be a bug in clang-11 (used for builds on oss-fuzz) :
# accel/tcg/cputlb.o: In function `load_memop':
# accel/tcg/cputlb.c:1505: undefined reference to `qemu_build_not_reached'
#
# When building with optimization, the compiler is expected to prove that the
# statement cannot be reached, and remove it. For some reason clang-11 doesn't
# remove it, resulting in an unresolved reference to qemu_build_not_reached
# Undefine the __OPTIMIZE__ macro which compiler.h relies on to choose whether
# to " #define qemu_build_not_reached() g_assert_not_reached() "
EXTRA_CFLAGS="$CFLAGS -U __OPTIMIZE__"
if ! { [ -e "./COPYING" ] &&
[ -e "./MAINTAINERS" ] &&
[ -e "./Makefile" ] &&
[ -d "./docs" ] &&
[ -e "./VERSION" ] &&
[ -d "./linux-user" ] &&
[ -d "./system" ];} ; then
fatal "Please run the script from the top of the QEMU tree"
fi
mkdir -p $OSS_FUZZ_BUILD_DIR || fatal "mkdir $OSS_FUZZ_BUILD_DIR failed"
cd $OSS_FUZZ_BUILD_DIR || fatal "cd $OSS_FUZZ_BUILD_DIR failed"
if [ -z ${OUT+x} ]; then
DEST_DIR=$(realpath "./DEST_DIR")
else
DEST_DIR=$OUT
fi
mkdir -p "$DEST_DIR/lib/" # Copy the shared libraries here
# Build once to get the list of dynamic lib paths, and copy them over
../configure --disable-werror --cc="$CC" --cxx="$CXX" --enable-fuzzing \
--prefix="/opt/qemu-oss-fuzz" \
--extra-cflags="$EXTRA_CFLAGS" --target-list="i386-softmmu"
if ! make "-j$(nproc)" qemu-fuzz-i386; then
fatal "Build failed. Please specify a compiler with fuzzing support"\
"using the \$CC and \$CXX environment variables"\
"\nFor example: CC=clang CXX=clang++ $0"
fi
if [ "$GITLAB_CI" != "true" ]; then
for i in $(ldd ./qemu-fuzz-i386 | cut -f3 -d' '); do
cp "$i" "$DEST_DIR/lib/"
done
rm qemu-fuzz-i386
# Build a second time to build the final binary with correct rpath
../configure --disable-werror --cc="$CC" --cxx="$CXX" --enable-fuzzing \
--prefix="/opt/qemu-oss-fuzz" \
--extra-cflags="$EXTRA_CFLAGS" --extra-ldflags="-Wl,-rpath,\$ORIGIN/lib" \
--target-list="i386-softmmu"
make "-j$(nproc)" qemu-fuzz-i386 V=1
fi
# Place data files in the preinstall tree
make install DESTDIR=$DEST_DIR/qemu-bundle
rm -rf $DEST_DIR/qemu-bundle/opt/qemu-oss-fuzz/bin
rm -rf $DEST_DIR/qemu-bundle/opt/qemu-oss-fuzz/libexec
targets=$(./qemu-fuzz-i386 | grep generic-fuzz | awk '$1 ~ /\*/ {print $2}')
scripts/oss-fuzz: give all fuzzers -target names We switched to hardlinks in a942f64cc4 ("scripts/oss-fuzz: use hardlinks instead of copying") The motivation was to conserve space (50 fuzzers built with ASAN, can weigh close to 9 GB). Unfortunately, OSS-Fuzz (partially) treated the underlying copy of the fuzzer as a standalone fuzzer. To attempt to fix, we tried: f8b8f37463 ("scripts/oss-fuzz: rename bin/qemu-fuzz-i386") This was also not a complete fix, because though OSS-Fuzz ignores the renamed fuzzer, the underlying ClusterFuzz, doesn't: https://storage.googleapis.com/clusterfuzz-builds/qemu/targets.list.address https://oss-fuzz-build-logs.storage.googleapis.com/log-9bfb55f9-1c20-4aa6-a49c-ede12864eeb2.txt (clusterfuzz still lists qemu-fuzz-i386.base as a fuzzer) This change keeps the hard-links, but makes them all point to a file with a qemu-fuzz-i386-target-.. name. If we have targets, A, B, C, the result will be: qemu-fuzz-i386-target-A (base file) qemu-fuzz-i386-target-B -> qemu-fuzz-i386-target-A qemu-fuzz-i386-target-C -> qemu-fuzz-i386-target-A The result should be that every file that looks like a fuzzer to OSS-Fuzz/ClusterFuzz, can run as a fuzzer (we don't have a separate base copy). Unfortunately, there is not simple way to test this locally. In the future, it might be worth it to link the majority of QEMU in as a shared-object (see https://github.com/google/oss-fuzz/issues/4575 ) Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Message-Id: <20201108171136.160607-1-alxndr@bu.edu> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-08 20:11:36 +03:00
base_copy="$DEST_DIR/qemu-fuzz-i386-target-$(echo "$targets" | head -n 1)"
cp "./qemu-fuzz-i386" "$base_copy"
# Run the fuzzer with no arguments, to print the help-string and get the list
# of available fuzz-targets. Copy over the qemu-fuzz-i386, naming it according
# to each available fuzz target (See 05509c8e6d fuzz: select fuzz target using
# executable name)
scripts/oss-fuzz: give all fuzzers -target names We switched to hardlinks in a942f64cc4 ("scripts/oss-fuzz: use hardlinks instead of copying") The motivation was to conserve space (50 fuzzers built with ASAN, can weigh close to 9 GB). Unfortunately, OSS-Fuzz (partially) treated the underlying copy of the fuzzer as a standalone fuzzer. To attempt to fix, we tried: f8b8f37463 ("scripts/oss-fuzz: rename bin/qemu-fuzz-i386") This was also not a complete fix, because though OSS-Fuzz ignores the renamed fuzzer, the underlying ClusterFuzz, doesn't: https://storage.googleapis.com/clusterfuzz-builds/qemu/targets.list.address https://oss-fuzz-build-logs.storage.googleapis.com/log-9bfb55f9-1c20-4aa6-a49c-ede12864eeb2.txt (clusterfuzz still lists qemu-fuzz-i386.base as a fuzzer) This change keeps the hard-links, but makes them all point to a file with a qemu-fuzz-i386-target-.. name. If we have targets, A, B, C, the result will be: qemu-fuzz-i386-target-A (base file) qemu-fuzz-i386-target-B -> qemu-fuzz-i386-target-A qemu-fuzz-i386-target-C -> qemu-fuzz-i386-target-A The result should be that every file that looks like a fuzzer to OSS-Fuzz/ClusterFuzz, can run as a fuzzer (we don't have a separate base copy). Unfortunately, there is not simple way to test this locally. In the future, it might be worth it to link the majority of QEMU in as a shared-object (see https://github.com/google/oss-fuzz/issues/4575 ) Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Message-Id: <20201108171136.160607-1-alxndr@bu.edu> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-08 20:11:36 +03:00
for target in $(echo "$targets" | tail -n +2);
do
# Ignore the generic-fuzz target, as it requires some environment variables
# to be configured. We have some generic-fuzz-{pc-q35, floppy, ...} targets
# that are thin wrappers around this target that set the required
# environment variables according to predefined configs.
if [[ $target == "generic-fuzz-"* ]]; then
scripts/oss-fuzz: give all fuzzers -target names We switched to hardlinks in a942f64cc4 ("scripts/oss-fuzz: use hardlinks instead of copying") The motivation was to conserve space (50 fuzzers built with ASAN, can weigh close to 9 GB). Unfortunately, OSS-Fuzz (partially) treated the underlying copy of the fuzzer as a standalone fuzzer. To attempt to fix, we tried: f8b8f37463 ("scripts/oss-fuzz: rename bin/qemu-fuzz-i386") This was also not a complete fix, because though OSS-Fuzz ignores the renamed fuzzer, the underlying ClusterFuzz, doesn't: https://storage.googleapis.com/clusterfuzz-builds/qemu/targets.list.address https://oss-fuzz-build-logs.storage.googleapis.com/log-9bfb55f9-1c20-4aa6-a49c-ede12864eeb2.txt (clusterfuzz still lists qemu-fuzz-i386.base as a fuzzer) This change keeps the hard-links, but makes them all point to a file with a qemu-fuzz-i386-target-.. name. If we have targets, A, B, C, the result will be: qemu-fuzz-i386-target-A (base file) qemu-fuzz-i386-target-B -> qemu-fuzz-i386-target-A qemu-fuzz-i386-target-C -> qemu-fuzz-i386-target-A The result should be that every file that looks like a fuzzer to OSS-Fuzz/ClusterFuzz, can run as a fuzzer (we don't have a separate base copy). Unfortunately, there is not simple way to test this locally. In the future, it might be worth it to link the majority of QEMU in as a shared-object (see https://github.com/google/oss-fuzz/issues/4575 ) Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Message-Id: <20201108171136.160607-1-alxndr@bu.edu> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-08 20:11:36 +03:00
ln $base_copy \
"$DEST_DIR/qemu-fuzz-i386-target-$target"
fi
done
echo "Done. The fuzzers are located in $DEST_DIR"
exit 0