Improve the list of Travis CI jobs by using the build matrix feature to
test all combinations of compilers and architectures on the latest
version of Ubuntu, and by adding more jobs for older versions of Ubuntu.
Don't try to detect and use different compilers, since it's better to
specify this via the environment (e.g. via the Travis CI build matrix).
While doing this, also deduplicate the logic for testing with valgrind
and UBSAN, improve the log messages, and add a test with -O3.
Now that run_tests.sh has been cleaned up to remove (or move) test
groups that weren't very useful, remove the concept of test groups and
just run all the tests.
The reason that run_tests.sh supported running checksum_benchmarks.sh is
that as a side effect, checksum_benchmarks.sh runs the 'test_checksums'
program with all combinations of CPU features.
However, commit ec60cb48d11c ("tools/run_tests.sh: test different
combinations of CPU features") made run_tests.sh handle this elsewhere.
So having run_tests.sh run checksum_benchmarks.sh is no longer useful.
Keep checksum_benchmarks.sh around for manual benchmark runs, however.
android_tests is only useful for local testing, and it wasn't being run
in Travis CI. Move it into a separate script to avoid complicating
run_tests.sh.
This was only useful for me to do local testing, I don't have the needed
MIPS router available anymore, and its main purpose was test a big
endian system but that's now covered by testing s390x with Travis CI.
This script only worked for me to do local testing and wasn't otherwise
used. In particular, the Windows build tests in Travis CI don't use
this script, nor does the make-windows-releases script use it.
Compression is based on heuristics, so we can't guarantee in every
circumstance that the compression ratio will improve as the compression
level increases. The tests need to be 100% reliable though, so drop
this part of the test.
Move the user-specified CFLAGS to the end of the CFLAGS definition, so
that warnings the Makefile enables can be disabled using -Wno-$foo.
This is useful when old compilers give false positive warnings.
This is needed to avoid the following error when using
-fsanitize=undefined with gcc:
lib/x86/adler32_impl.h:214:2: runtime error: signed integer overflow:
1951294680 + 1956941400 cannot be represented in type 'int'
Note that this isn't seen when using -fsanitize=undefined with clang.
Old compilers don't have unsigned vector types, so work around that.
Add a CRC32 implementation that uses the ARM CRC32 instructions.
This is simpler and faster than the PMULL implementation. On AWS
Graviton2, the performance improvement is about 70%. On Hikey960, the
performance improvement is about 30% for the Cortex-A53 cores or about
5% for the Cortex-A73 cores.
Based on work by Greg V <greg@unrelenting.technology>
(https://github.com/ebiggers/libdeflate/pull/45)
and Andrew Steinborn <git@steinborn.me>
(https://github.com/ebiggers/libdeflate/pull/76).
If support for CRC32 instructions is detected, set
ARM_CPU_FEATURE_CRC32. Also define
COMPILER_SUPPORTS_CRC32_TARGET_INTRINSICS when appropriate, and update
run_tests.sh to toggle the crc32 feature for testing.
android_build.sh no longer works with recent NDKs, and it has a lot of
logic to use old NDKs directly that wasn't really necessary because it
could have just required standalone toolchains instead.
Recent NDKs (r19 and later) come with standalone toolchains by default.
Also, they now only include clang, not gcc.
Modify the script to just support these recent NDKs. Also, default to
arm64 and add support for enabling CRC instructions.
Some users may require a valid DEFLATE, zlib, or gzip stream but know
ahead of time that particular inputs are not compressible. zlib
supports "level 0" for this use case. Support this in libdeflate too.
Resolves https://github.com/ebiggers/libdeflate/issues/86
To test the different CRC-32 and Adler-32 implementations, use
LIBDEFLATE_DISABLE_CPU_FEATURES instead of running some hack-ish
'sed' commands to edit the source code.
