Commit Graph

4025 Commits

Author SHA1 Message Date
Martijn van Beurden
b433bbc5cf
Rewrite of fixed_compute_best_predictor functions
The code of fixed_compute_best_predictor was presumable once optimized,
but it appears a much more readable form is now equally fast or even faster,
see https://github.com/xiph/flac/pull/337 for details.
2022-05-08 20:14:36 +02:00
Martijn van Beurden
b64ea1171b Fix fuzzer_encoder_v2 timeouts
This fixes the 'false positive' oss-fuzz bug filed under
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47264

The fuzzer was timing out because of an enormous input (932kB)
combined with very slow settings. The commit disables certain
slow settings on inputs larger than 2^18 byte.
2022-05-07 21:58:40 +02:00
Martijn van Beurden
b6e2f4f014 Fix distcheck 2022-05-06 12:28:17 +02:00
Martijn van Beurden
cda987f7c5 Lengthen test_compression.sh material for benchmarking on CI
This is necessary to compare CI WITH_ASM=1 and WITH_ASM=0 to be able
to check whether this switch does anything
2022-05-03 11:10:15 +02:00
Martijn van Beurden
e3a9212bfc [CMake] Fix ARM64 intrinsics activation
In 95e2c52 the autotools build worked fine, but CMake missed a few
defines. Activition of the lpc_compute_residual_from_qlp_coefficients
depended on encoder->private_->cpuinfo.use_asm for no reason, so
this dependency is removed
2022-05-03 11:10:15 +02:00
Martijn van Beurden
d9554fb7c9 Add --limit-min-bitrate to flac and add test for it 2022-05-02 07:59:01 +02:00
Martijn van Beurden
b5f4a1535c Add API functions to limit minimum bitrate
Quite a lot of decoders have trouble streaming or seeking in a
file with frames that only consist of constant subframes because
of the large difference between the largest and smallest subframe.

To remedy this, this commit makes it possible to disable the use
of constant subframes for the last subframe in case all others are
constant with a new API function. This means the minimum bitrate
for a FLAC file encoded with this function used is raised to
1bit/sample (i.e. 48kbit/s for 48kHz material). This commit also
adds tests to the test suite
2022-05-02 07:59:01 +02:00
Martijn van Beurden
3fc5ba4637 Replace seek error with handling when hitting EOF while reading frame 2022-05-01 17:35:36 +02:00
Martijn van Beurden
ee9c07aa22 Fix memory leak in stream_decoder.c
Memory leaked in case FLAC__stream_encoder_set_metadata was used
but subsequent init failed
2022-04-30 21:20:50 +02:00
Martijn van Beurden
b52734242e Add fuzzer_encoder_v2
Add additional fuzzer for the encoder, aiming to reach more coverage
than the current encoder fuzzer. This is done by incorporating
metadata handling and fixing a problem with different settings in
the current fuzzer
2022-04-30 21:20:50 +02:00
Martijn van Beurden
e0a874e84b
Silence a bunch of MSVC warnings
See https://github.com/xiph/flac/issues/313
2022-04-30 11:55:17 +02:00
Fabrice Fontaine
6cd28e855c Fix uclibc build
Commit 4fbb6d4f2e reverted commit
44036c9a9b and so broke again the build on
uclibc

So put back the sys/auxv.h check as well as a getauxval check in a way
that doesn't break iOS

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
2022-04-30 09:46:47 +02:00
Martijn van Beurden
63ac1c37be
Silence oss-fuzz integer overflow warnings in audio data path
Because fuzzing feeds bogus predictors and residual samples to the
decoder, having overflows in certain functions is unavoidable. Also,
because the calculated values are audio path only, there is little
potential for security problems

Should 'fix' the following reports
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44824
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46964
2022-04-30 09:25:16 +02:00
Martijn van Beurden
ef4ad99231 Add ARM64 NEON intrinsics lpc_compute_autocorrelation routines 2022-04-29 21:44:29 +02:00
Rosen Penev
bfe5ff9455 change if checks to ifdef
More similar to the rest of the code.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-04-29 19:17:26 +02:00
RonenGvili
95e2c52980
Adding ARM64 support and optimized Neon implementation (#270)
Add NEON intrinsics routines for lpc_compute_residual_from_qlp_coefficients
and lpc_compute_residual_from_qlp_coefficients_wide
2022-04-29 14:46:07 +02:00
Martijn van Beurden
5df56dbcf5
Speed up intrinsic routines for fixed_compute_best_predictor
New intrinsic routines are added to compute the best fixed
predictor. According to my testing, the new routine is about
3 times as fast as the existing routine. This means an
overall speedup for preset -2 of about 30% and for preset
-0 of about 15% for CDDA
2022-04-28 21:19:10 +02:00
Martijn van Beurden
0caf685360
[CMake] Enable building ogg together with flac (#325)
This reverts the part of commit 2013738 that searched for a local
ogg build
2022-04-27 19:53:13 +02:00
Martijn van Beurden
1793632ee6
Rework error handling (#283)
This commit reworks the code decoding a frame, to add silence when
frames are missing and output silence when something other than the
frame header seems corrupted. Tests are added to the test suite for
this functionality. Also, decoded values are checked to be within bps
2022-04-27 12:16:15 +02:00
Martijn van Beurden
7e785eb9a8 Fix seeking bug
Commit 159cd6c introduced a bug that only triggered upon seeking
from the start of a headerless FLAC file to the first frame (so
really not a seek at all). Furthermore that commit did nothing
else in any other circumstance. This commit fixes that, by both
fixing the problem and the behaviour the commit mentioned earlier
meant to introduce.

Co-authored-by: Robert Kausch <robert.kausch@freac.org>
2022-04-27 10:29:34 +02:00
Martijn van Beurden
9c495ae3ac Enable flac to skip extra STREAMINFO and check number of samples
This commit fixed problems with two files mentioned here:
https://hydrogenaud.io/index.php?topic=121478.msg1004022#msg1004022

The first file was truncated and had no MD5. flac didn't warn
about the number of samples not being correct. The second file
had corrupted metadata in which flac detected two STREAMINFO
blocks, triggering quite odd behaviour.

This commit adds checks for the number of samples and skips
a STREAMINFO block if it has already received one
2022-04-27 07:26:04 +02:00
Martijn van Beurden
d4131c0aff Remove 'news' from html doc 2022-04-26 22:09:05 +02:00
Martijn van Beurden
a21bdc4a9e Documentation fixes: preset description and subset blocksizes
As described in github issue #236, the description of compression
presets in flac.sgml wasn't updated. Also, the description of which
blocksizes are considered subset has been lagging behind in the
manpages as well as the build-in help of FLAC for years, as this
was changed back in 2007. This commit fixes those issues
2022-04-26 22:09:05 +02:00
Martijn van Beurden
f59cb6f7b4 Update links to Xiph git in documentation 2022-04-26 22:09:05 +02:00
Martijn van Beurden
f6b78e69e0 [CMake] Honor WITH_AVX
Fixes https://github.com/xiph/flac/issues/286
2022-04-26 20:52:20 +02:00
Martijn van Beurden
c157be0f77 Add a license to fuzzing code
The flac fuzzers were taken from
https://github.com/guidovranken/flac-fuzzers and
https://github.com/guidovranken/fuzzing-headers

I couldn't find a mention of a license anywhere, so I contacted
the author: https://github.com/guidovranken/flac-fuzzers/issues/2

Following this, a licence statement is added to all files taken
from these repositories
2022-04-25 20:28:17 +02:00
Martijn van Beurden
8c46644a8d [CMake] Add check for PPC64, POWER8/9 and vec_doubleh
Add checks for POWER8/9 intrinsics similar to those in autotools
to cmake
2022-04-25 20:13:36 +02:00
Martijn van Beurden
d15bcf80a0 Add check for certain POWER8/9 intrinsic in altivec.h to configure 2022-04-25 20:13:36 +02:00
Martijn van Beurden
a4156cd78b Modify Travis to only use partner queues
Currently github actions only runs CI on x86. Travis has 'partner
queues' which do not use credits. By changing .travis.yml to only
use these, forks can use Travis to test on arm64 and ppc64le. I
intend to use this to run travis every now and then on my own
clone of xiph/flac
2022-04-25 20:13:36 +02:00
Martijn van Beurden
f304a4cf86 Remove partition order limit on last block
The partition order was forced to 0 on the last block of a stream.
This limitation is not necessary, as further down in the code more
generic limits are already in place. This improved compression on
the last block, so especially very short files benefit
2022-04-24 22:27:02 +02:00
Cameron Cawley
32d51aade6 Only define _XOPEN_SOURCE and __EXTENSIONS__ when necessary 2022-04-22 17:19:49 +02:00
Cameron Cawley
ce18b591a0 Always use utimbuf when building for Windows 2022-04-22 17:19:49 +02:00
Martijn van Beurden
11b72938c1 Fix format_mistake printf
If on encoding a different format header is detected than is guessed
from file extension, format_mistake in invoked. On merging wave64
support (commit d7f5344), updating the printable names in
format_mistake was missed
2022-04-22 16:59:04 +02:00
Martijn van Beurden
b288acb109 No not stop decoding FLAC file
As discussed in https://github.com/xiph/flac/issues/243, this is
probably the root cause of CVE-2021-0561
2022-04-22 15:30:40 +02:00
pkubaj
36b5b63b36 Fix hwcaps type for FreeBSD/powerpc*
According to elf_aux_info() manpage, AT_HWCAP2 is of u_long type.
Feature detection works anyway in this case, but u_long is more correct.
While here, also remove FreeBSD 11 support, it's EOL.
2022-04-22 15:27:54 +02:00
NotTsunami
b171e767e2 Add missing langinfo check for CMake 2022-04-21 19:35:58 +02:00
Martijn van Beurden
f579b163fe
Add git commit tag, hash and date to vendor string when available
In some circles (Hydrogenaud.io for example) people share binaries
compiled from git, i.e. inbetween official releases. Files created
with these binaries cannot be discerned from others. To improve
troubleshooting, compiles from a git repository are marked with
the commit hash and date when configured with autotools or CMake
2022-04-20 21:47:07 +02:00
Martijn van Beurden
1ca7b38f3c Add FLAC__STREAM_DECODER_ERROR_STATUS_BAD_METADATA 2022-04-20 12:06:42 +02:00
Martijn van Beurden
0077d3b697 Add checks in parsing metadata blocks
In parsing certain types of metadata blocks, it was not checked
whether certain length fields were at all possible. A small file
of 64 byte was enough to let the decoder allocate 4GiB of memory
by simply stating a field was 4GiB in length, while metadata
blocks in FLAC cannot be larger than 16MiB.

This commit adds a mechanism to the bitreader to set a length
limit upon reading. This limit is checked at each read, at each
memory allocation and after reading the block. In case anything
is deemed wrong, parsing metadata is aborted, as 'resyncing' is
only possible on audio frames, not on metadata blocks.

Credit: Oss-Fuzz
Issue: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=37865
2022-04-20 12:06:42 +02:00
Martijn van Beurden
0f98b940f8 Update fuzzers
This changes the names of the two fuzzers so they align with
current oss-fuzz usage (to provide continuity in test data and
corpora) and adds a dictionary to the decoder fuzzer. It also
removes some unnecessary configuration, as it clashed with oss-fuzz
defaults
2022-04-20 11:10:08 +02:00
Martijn van Beurden
b48ed95fcb Deduplicate VSX and SSE2 autocorelation calculation code 2022-04-19 20:38:37 +02:00
Martijn van Beurden
ee18d1b892 Switch to autoc[] as double
This changes the calculation of autocorrelation from single precision
float to double precision. With this, compression improves on average
by about 0.5% with compression level 4, with some tracks (espcially
material with high samplerates) seeing improvements of 10% or more.

This commit removes SSE intrinsics and MMX assembler routins, adds
SSE2 and changes VSX autoc[] routines and changes the regular C
code.
2022-04-19 20:38:37 +02:00
Martijn van Beurden
b19ed90a5b
Remove api doc note untrue since commit c9c0d13
This has been untrue for almost 20 years now
2022-04-19 19:05:24 +02:00
Martijn van Beurden
7385dac383
Fix overflow of streaminfo total samples (github issue 237)
When writing a FLAC file with over 2^36 samples, the total samples
entry in streaminfo will overflow and wrap around. Setting it to 0
(which means unknown number of samples) makes sure the decoder
doesn't rely on this number

Co-authored-by: Ralph Giles <giles@thaumas.net>
2022-04-19 17:45:22 +02:00
NotTsunami
7406eabe17 Remove unused macros
Cleanup of ecede0bc72
2022-04-19 10:57:46 +02:00
Till Schäfer
ff818486dc Avoid creating a new variable in md5 gcc workaround
https://sourceforge.net/p/flac/bugs/478/
2022-04-19 10:55:23 +02:00
Martijn van Beurden
4b7e79f974 Fix stack protector detection for C++ 2022-04-19 10:55:23 +02:00
Martijn van Beurden
cde4b8c783 Add more overflow checks for residual_bits calculation
commit 585e3b added an overflow check in the residual bits
calculation, but actually hitting this causes problems further
down the line. This fixes the problem described in
https://hydrogenaud.io/index.php?topic=121349.msg1001227;topicseen#new

Also, UINT_MAX is changed to UINT32_MAX as uint32_t is used, not unsigned
2022-04-17 21:22:02 +02:00
Martijn van Beurden
21d0640ea9 Limit maximum size of bitwriter buffer
In the past, various bugs have triggered libFLAC to write enormous
coded residuals. While these bugs have been fixed, this check is
added to limit the impact of possible remaining or future bugs

When a bitwriter is grown past the maximum sane size of a frame
(8 channels of 65535 samples with 24 + 4 bits per sample) this
indicates that such a bug was triggered. Instead of a possible
crash or creating unreadable files, bitwriter_grow_ fails as if
it could not allocate more memory.
2022-04-17 19:17:02 +02:00
Martijn van Beurden
2e50ea341d Remove special treatment of partition_order == 0
in set_partitioned_rice, there was a special treatment in case
partition_order == 0. In this case, a rice parameter was passed
as an argument. However, in case of an LPC block, this rice
parameter was calculated from the LPC error, which is based on the
samples after windowing. Quite often, this resulted in a subframe
with two partition with the same rice parameter. Also, in case of
partial_tukey and punchout_tukey windows, the suggested rice
parameter could be quite sub-optimal.

This patch removes the special treatment of partition_order == 0
and the passing of a rice_parameter as an argument
2022-04-17 19:17:02 +02:00