Commit Graph

4038 Commits

Author SHA1 Message Date
Martijn van Beurden 383b77fa08 Let fuzzer_encoder_v2 use tmp files to improve coverage 2022-05-19 20:22:09 +02:00
Rosen Penev 2b4ba6013e
tests: Run through shellcheck (#134)
Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-05-18 21:27:19 +02:00
Martijn van Beurden f180900b95 Fix memory leak
When resetting the decoder, the MD5 buffer wasn't freed but the pointer
set to zero anyway.
2022-05-18 17:12:28 +02:00
Martijn van Beurden 9e8d7389dc Fix initialization of decoder fuzzer
The decoder fuzzer initialized before setting up. This is now swapped
without upsetting the way the datasource is queried
2022-05-18 17:12:28 +02:00
Martijn van Beurden e5e99dcdc2
Simplify and merge win_utf8 files and remove from DLL interface
libFLAC DLLs were exposing windows_unicode_filename.h functions
because flac and metaflac needed to set flac_internal_set_utf8_
filenames. Files windows_unicode_filename.{c/h} and
win_utf8_io.[c/h] are merged, and all non-utf8 parts are removed.

With this commit, the libFLAC DLL interface is the same as the
libFLAC interface of shared libraries on other platforms
2022-05-17 22:05:45 +02:00
Martijn van Beurden 3d40818489
Revert part of commit 5df56db
Commit 5df56db introduced four completely rewritten functions with
intrinsics, but it turns out two of them have integers that can
overflow. Because those two functions were barely faster than
what they replaced, fixing these overflows will probably make the
functions slower than what they replaced, so this is reverted.

Credit: Oss-Fuzz
Issue: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47416
2022-05-16 19:41:59 +02:00
Martijn van Beurden 61649d09aa
Add checks for decoding from and encoding to UTF-8 filenames 2022-05-12 22:21:39 +02:00
Martijn van Beurden 1bec35e337 Add residual limit checking
This patch adds a (fast) check in the bitreader to detect overflow
of the residual
2022-05-11 10:34:49 +02:00
Martijn van Beurden becba351f0 Make it possible for fuzzer to disable instruction set optimizations
This should greatly increase code coverage of the encoder fuzzer
2022-05-11 10:34:49 +02:00
Martijn van Beurden 87a5f3e952 Add abort on encoding error that should not happen
This commit lets the following errors trigger an abort:
- FLAC__STREAM_ENCODER_OGG_ERROR
- FLAC__STREAM_ENCODER_VERIFY_DECODER_ERROR
- FLAC__STREAM_ENCODER_VERIFY_MISMATCH_IN_AUDIO_DATA
- FLAC__STREAM_ENCODER_IO_ERROR
- FLAC__STREAM_ENCODER_FRAMING_ERROR
- FLAC__STREAM_ENCODER_MEMORY_ALLOCATION_ERROR

FLAC__STREAM_ENCODER_CLIENT_ERROR does not trigger an abort because
it is used to signal invalid input. FLAC__STREAM_ENCODER_UNINITIALIZED
does not trigger an abort because init fails when invalid parameters
are given
2022-05-11 10:34:49 +02:00
Martijn van Beurden 6122337f73 Check whether verify decoder is still valid
Errors returned by the verifing decoder were ignored, while an error
often invalidates the assumptions the encoder made about the state
of the verifying decoder. If an error was ignored, it could be the
decoder outputs an (invalid) frame with a different number of channels
than the encoder expects. This could cause heap overflows. This
commit adds checks for this occuring.
2022-05-11 10:34:49 +02:00
Martijn van Beurden dd15db8f93 Fix overflows in fixed_compute_best_predictor
Credit: Oss-Fuzz
Issue: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47187
2022-05-09 22:08:39 +02:00
Martijn van Beurden 8b148b257e Rename FLAC__get_decoder_client_data to ...
... FLAC__stream_decoder_get_client_data. Also the function
definition was moved to its 'siblings'. This way, the function
fits better in the API with respect to naming.
2022-05-09 17:29:46 +02:00
Martijn van Beurden b433bbc5cf
Rewrite of fixed_compute_best_predictor functions
The code of fixed_compute_best_predictor was presumable once optimized,
but it appears a much more readable form is now equally fast or even faster,
see https://github.com/xiph/flac/pull/337 for details.
2022-05-08 20:14:36 +02:00
Martijn van Beurden b64ea1171b Fix fuzzer_encoder_v2 timeouts
This fixes the 'false positive' oss-fuzz bug filed under
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47264

The fuzzer was timing out because of an enormous input (932kB)
combined with very slow settings. The commit disables certain
slow settings on inputs larger than 2^18 byte.
2022-05-07 21:58:40 +02:00
Martijn van Beurden b6e2f4f014 Fix distcheck 2022-05-06 12:28:17 +02:00
Martijn van Beurden cda987f7c5 Lengthen test_compression.sh material for benchmarking on CI
This is necessary to compare CI WITH_ASM=1 and WITH_ASM=0 to be able
to check whether this switch does anything
2022-05-03 11:10:15 +02:00
Martijn van Beurden e3a9212bfc [CMake] Fix ARM64 intrinsics activation
In 95e2c52 the autotools build worked fine, but CMake missed a few
defines. Activition of the lpc_compute_residual_from_qlp_coefficients
depended on encoder->private_->cpuinfo.use_asm for no reason, so
this dependency is removed
2022-05-03 11:10:15 +02:00
Martijn van Beurden d9554fb7c9 Add --limit-min-bitrate to flac and add test for it 2022-05-02 07:59:01 +02:00
Martijn van Beurden b5f4a1535c Add API functions to limit minimum bitrate
Quite a lot of decoders have trouble streaming or seeking in a
file with frames that only consist of constant subframes because
of the large difference between the largest and smallest subframe.

To remedy this, this commit makes it possible to disable the use
of constant subframes for the last subframe in case all others are
constant with a new API function. This means the minimum bitrate
for a FLAC file encoded with this function used is raised to
1bit/sample (i.e. 48kbit/s for 48kHz material). This commit also
adds tests to the test suite
2022-05-02 07:59:01 +02:00
Martijn van Beurden 3fc5ba4637 Replace seek error with handling when hitting EOF while reading frame 2022-05-01 17:35:36 +02:00
Martijn van Beurden ee9c07aa22 Fix memory leak in stream_decoder.c
Memory leaked in case FLAC__stream_encoder_set_metadata was used
but subsequent init failed
2022-04-30 21:20:50 +02:00
Martijn van Beurden b52734242e Add fuzzer_encoder_v2
Add additional fuzzer for the encoder, aiming to reach more coverage
than the current encoder fuzzer. This is done by incorporating
metadata handling and fixing a problem with different settings in
the current fuzzer
2022-04-30 21:20:50 +02:00
Martijn van Beurden e0a874e84b
Silence a bunch of MSVC warnings
See https://github.com/xiph/flac/issues/313
2022-04-30 11:55:17 +02:00
Fabrice Fontaine 6cd28e855c Fix uclibc build
Commit 4fbb6d4f2e reverted commit
44036c9a9b and so broke again the build on
uclibc

So put back the sys/auxv.h check as well as a getauxval check in a way
that doesn't break iOS

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
2022-04-30 09:46:47 +02:00
Martijn van Beurden 63ac1c37be
Silence oss-fuzz integer overflow warnings in audio data path
Because fuzzing feeds bogus predictors and residual samples to the
decoder, having overflows in certain functions is unavoidable. Also,
because the calculated values are audio path only, there is little
potential for security problems

Should 'fix' the following reports
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44824
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46964
2022-04-30 09:25:16 +02:00
Martijn van Beurden ef4ad99231 Add ARM64 NEON intrinsics lpc_compute_autocorrelation routines 2022-04-29 21:44:29 +02:00
Rosen Penev bfe5ff9455 change if checks to ifdef
More similar to the rest of the code.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-04-29 19:17:26 +02:00
RonenGvili 95e2c52980
Adding ARM64 support and optimized Neon implementation (#270)
Add NEON intrinsics routines for lpc_compute_residual_from_qlp_coefficients
and lpc_compute_residual_from_qlp_coefficients_wide
2022-04-29 14:46:07 +02:00
Martijn van Beurden 5df56dbcf5
Speed up intrinsic routines for fixed_compute_best_predictor
New intrinsic routines are added to compute the best fixed
predictor. According to my testing, the new routine is about
3 times as fast as the existing routine. This means an
overall speedup for preset -2 of about 30% and for preset
-0 of about 15% for CDDA
2022-04-28 21:19:10 +02:00
Martijn van Beurden 0caf685360
[CMake] Enable building ogg together with flac (#325)
This reverts the part of commit 2013738 that searched for a local
ogg build
2022-04-27 19:53:13 +02:00
Martijn van Beurden 1793632ee6
Rework error handling (#283)
This commit reworks the code decoding a frame, to add silence when
frames are missing and output silence when something other than the
frame header seems corrupted. Tests are added to the test suite for
this functionality. Also, decoded values are checked to be within bps
2022-04-27 12:16:15 +02:00
Martijn van Beurden 7e785eb9a8 Fix seeking bug
Commit 159cd6c introduced a bug that only triggered upon seeking
from the start of a headerless FLAC file to the first frame (so
really not a seek at all). Furthermore that commit did nothing
else in any other circumstance. This commit fixes that, by both
fixing the problem and the behaviour the commit mentioned earlier
meant to introduce.

Co-authored-by: Robert Kausch <robert.kausch@freac.org>
2022-04-27 10:29:34 +02:00
Martijn van Beurden 9c495ae3ac Enable flac to skip extra STREAMINFO and check number of samples
This commit fixed problems with two files mentioned here:
https://hydrogenaud.io/index.php?topic=121478.msg1004022#msg1004022

The first file was truncated and had no MD5. flac didn't warn
about the number of samples not being correct. The second file
had corrupted metadata in which flac detected two STREAMINFO
blocks, triggering quite odd behaviour.

This commit adds checks for the number of samples and skips
a STREAMINFO block if it has already received one
2022-04-27 07:26:04 +02:00
Martijn van Beurden d4131c0aff Remove 'news' from html doc 2022-04-26 22:09:05 +02:00
Martijn van Beurden a21bdc4a9e Documentation fixes: preset description and subset blocksizes
As described in github issue #236, the description of compression
presets in flac.sgml wasn't updated. Also, the description of which
blocksizes are considered subset has been lagging behind in the
manpages as well as the build-in help of FLAC for years, as this
was changed back in 2007. This commit fixes those issues
2022-04-26 22:09:05 +02:00
Martijn van Beurden f59cb6f7b4 Update links to Xiph git in documentation 2022-04-26 22:09:05 +02:00
Martijn van Beurden f6b78e69e0 [CMake] Honor WITH_AVX
Fixes https://github.com/xiph/flac/issues/286
2022-04-26 20:52:20 +02:00
Martijn van Beurden c157be0f77 Add a license to fuzzing code
The flac fuzzers were taken from
https://github.com/guidovranken/flac-fuzzers and
https://github.com/guidovranken/fuzzing-headers

I couldn't find a mention of a license anywhere, so I contacted
the author: https://github.com/guidovranken/flac-fuzzers/issues/2

Following this, a licence statement is added to all files taken
from these repositories
2022-04-25 20:28:17 +02:00
Martijn van Beurden 8c46644a8d [CMake] Add check for PPC64, POWER8/9 and vec_doubleh
Add checks for POWER8/9 intrinsics similar to those in autotools
to cmake
2022-04-25 20:13:36 +02:00
Martijn van Beurden d15bcf80a0 Add check for certain POWER8/9 intrinsic in altivec.h to configure 2022-04-25 20:13:36 +02:00
Martijn van Beurden a4156cd78b Modify Travis to only use partner queues
Currently github actions only runs CI on x86. Travis has 'partner
queues' which do not use credits. By changing .travis.yml to only
use these, forks can use Travis to test on arm64 and ppc64le. I
intend to use this to run travis every now and then on my own
clone of xiph/flac
2022-04-25 20:13:36 +02:00
Martijn van Beurden f304a4cf86 Remove partition order limit on last block
The partition order was forced to 0 on the last block of a stream.
This limitation is not necessary, as further down in the code more
generic limits are already in place. This improved compression on
the last block, so especially very short files benefit
2022-04-24 22:27:02 +02:00
Cameron Cawley 32d51aade6 Only define _XOPEN_SOURCE and __EXTENSIONS__ when necessary 2022-04-22 17:19:49 +02:00
Cameron Cawley ce18b591a0 Always use utimbuf when building for Windows 2022-04-22 17:19:49 +02:00
Martijn van Beurden 11b72938c1 Fix format_mistake printf
If on encoding a different format header is detected than is guessed
from file extension, format_mistake in invoked. On merging wave64
support (commit d7f5344), updating the printable names in
format_mistake was missed
2022-04-22 16:59:04 +02:00
Martijn van Beurden b288acb109 No not stop decoding FLAC file
As discussed in https://github.com/xiph/flac/issues/243, this is
probably the root cause of CVE-2021-0561
2022-04-22 15:30:40 +02:00
pkubaj 36b5b63b36 Fix hwcaps type for FreeBSD/powerpc*
According to elf_aux_info() manpage, AT_HWCAP2 is of u_long type.
Feature detection works anyway in this case, but u_long is more correct.
While here, also remove FreeBSD 11 support, it's EOL.
2022-04-22 15:27:54 +02:00
NotTsunami b171e767e2 Add missing langinfo check for CMake 2022-04-21 19:35:58 +02:00
Martijn van Beurden f579b163fe
Add git commit tag, hash and date to vendor string when available
In some circles (Hydrogenaud.io for example) people share binaries
compiled from git, i.e. inbetween official releases. Files created
with these binaries cannot be discerned from others. To improve
troubleshooting, compiles from a git repository are marked with
the commit hash and date when configured with autotools or CMake
2022-04-20 21:47:07 +02:00