53 Commits

Author SHA1 Message Date
Eric Biggers
333eff73b2 tools/run_tests.sh: run all test programs in exec_tests.sh 2018-12-28 10:25:44 -06:00
Eric Biggers
73017f08e5 lib/x86/adler32: add an AVX-512BW optimized Adler32 implementation 2018-12-24 17:36:07 -06:00
Eric Biggers
57cab078f1 lib: optimize decompressing repeated static Huffman blocks
Improve libdeflate's worst-case performance decompressing malicious
DEFLATE streams by about 14x, bringing it within a factor of about 2x of
zlib, by skipping rebuilding the decode tables for the static Huffman
codes when they're already loaded into the decompressor.

This improves performance decompressing a stream of all empty static
Huffman blocks from about 0.36 MB/s to 175 MB/s, or the original
reproducer given on the Github issue from about 3.3 MB/s to 219 MB/s.
A regression test is added for these cases as well as the empty dynamic
Huffman blocks case to verify worst-case performance comparable to zlib.

Resolves https://github.com/ebiggers/libdeflate/issues/33
2018-12-23 12:03:00 -06:00
Eric Biggers
7fad94b8c9 Include import library in Windows binary releases
Previously:
	- libdeflate.dll: the dynamic library
	- libdeflate.lib: the static library

Now:
	- libdeflate.dll: the dynamic library
	- libdeflate.lib: the import library
	- libdeflatestatic.lib: the static library
2018-12-06 20:13:18 -08:00
Eric Biggers
e9d1014161 tools/checksum_benchmarks.sh: fix detecting/disabling NEON on AArch64 2018-03-03 13:04:13 -08:00
Eric Biggers
fc2ea22b44 lib/arm: add ARM PMULL implementation of CRC-32
Add an ARM PMULL implementation of CRC-32.  This is based on a patch by
Jun He <jun.he@linaro.org> as well as the x86 PCLMUL implementation.
2018-02-18 23:03:26 -08:00
Eric Biggers
0c62e25464 tools/run_tests.sh: detect gcc without multilib support 2018-02-18 23:03:26 -08:00
Eric Biggers
5f3afad793 tools/run_tests.sh: run checksum benchmarks 2018-02-18 23:03:26 -08:00
Eric Biggers
2f4315c21c tools/run_tests.sh: more Android tests 2018-02-18 23:03:26 -08:00
Eric Biggers
794a40401d tools/android_build.sh: move -pie to LDFLAGS 2018-02-18 23:03:26 -08:00
Eric Biggers
4282583b9b tools/android_build.sh: support crypto extensions 2018-02-18 23:03:26 -08:00
Eric Biggers
e7aa4666e0 tools/checksum_benchmarks.sh: various improvements
Make it compatible with the new code organization, make it run the
test_checksums program for each implementation, and run each
implementation in both 64-bit and 32-bit modes.
2018-02-18 23:03:26 -08:00
Eric Biggers
5a9d25a892 Support multi-member gzip files 2017-11-20 00:35:24 -08:00
Eric Biggers
65a119ddfd run_tests.sh: test for same output on big endian CPU 2017-05-29 18:25:07 -07:00
Eric Biggers
8067f44e8c run_tests.sh: portability improvements
Support running the run_tests.sh script on more types of systems.
2017-05-29 17:44:50 -07:00
Eric Biggers
27c13370cb run_tests.sh: run tests on arm64 2017-03-19 12:29:49 -07:00
Eric Biggers
f2f0df7274 deflate_compress: fix corruption with long literal run
When the block splitting algorithm was implemented, it became possible
for the compressor to use longer blocks, up to ~300KB.  Unfortunately it
was overlooked that this can allow literal runs > 65535 bytes, while in
one place the length of a literal run was still being stored in a u16.
To overflow the litrunlen and hit the bug the data would have had to
have far fewer matches than random data, which is possible but very
unusual.  Fix the bug by reserving more space to hold the litrunlen, and
add a test for it.
2017-01-14 20:51:03 -08:00
Eric Biggers
28cc14994b run_tests.sh: look for other clang versions 2016-11-04 21:27:50 -07:00
Eric Biggers
62cc3d71b4 Add PCLMUL/AVX-optimized CRC32 2016-10-27 20:33:35 -07:00
Eric Biggers
0776bc94d7 Add script to test gcc profile-guided optimizations 2016-10-23 21:59:57 -07:00
Eric Biggers
99d6e72b89 Add gzip program tests 2016-10-23 14:13:36 -07:00
Eric Biggers
7662390f55 run_tests.sh: fix Android device detection 2016-10-16 12:14:21 -07:00
Eric Biggers
14007c05fa run_tests.sh: improve reporting of Android test failures 2016-10-16 12:14:16 -07:00
Eric Biggers
0cd875d811 run_tests.sh: improve/fix test group handling 2016-10-16 12:13:02 -07:00
Eric Biggers
6879294990 Add script to run Adler-32 and CRC-32 benchmarks 2016-10-15 21:57:00 -07:00
Eric Biggers
c37a251655 New test program: checksum 2016-10-15 20:38:34 -07:00
Eric Biggers
1190be5a68 run_tests.sh: include test programs in static analysis 2016-10-15 20:38:29 -07:00
Eric Biggers
146d91c03f Test script improvements 2016-10-15 18:57:36 -07:00
Eric Biggers
0c043dd602 Add PCLMUL-accelerated CRC-32 2016-10-15 11:01:18 -07:00
Eric Biggers
7d3e2a997c Update licensing status
I've decided to simplify and standardize the licensing status for the
library by using the MIT license instead of CC0 (a.k.a. "public
domain").  This eliminates the somewhat controversial 4(a) clause in
CC0, and, for this and other reasons, should (somewhat ironically) make
it easier for some people to use and contribute to the project.

Note: copyright will apply to new changes and to new versions of the
work as a whole.  Of course, versions previously released as public
domain remain public domain where legally recognized.
2016-09-09 19:34:58 -07:00
Eric Biggers
8081ce7853 Improved Android test script 2016-09-05 00:02:18 -07:00
Eric Biggers
c2c8005591 Add tools/arm64_test.sh 2016-09-03 16:02:13 -07:00
Eric Biggers
683832e824 tools/arm_test.sh: push testdata only if needed 2016-09-03 16:02:12 -07:00
Eric Biggers
3992efa340 Update API to avoid naming collisions 2016-08-27 23:11:32 -07:00
Eric Biggers
6ac4a94203 tools/make-windows-release: use -Werror 2016-05-28 16:13:30 -05:00
Eric Biggers
f2c3a5b4e9 Various reorganization and cleanups
* Bring in common headers and program code from xpack project
* Move program code to programs/
* Move library code to lib/
* GNU89 and MSVC2010 compatibility
* Other changes
2016-05-21 15:38:15 -05:00
Eric Biggers
6acb3054f8 Add produce_gzip_benchmark_table.sh 2016-05-21 10:34:02 -05:00
Eric Biggers
fb9016cd86 benchmark: fix display of fractional part of percentage 2016-05-09 22:31:21 -05:00
Eric Biggers
5cf3ab19ee gzip: fix some error messages 2016-04-17 15:33:40 -05:00
Eric Biggers
05079b24af Remove subdirectory from Windows releases 2016-04-11 23:24:29 -05:00
Eric Biggers
f73db15203 Add tools/make-windows-releases 2016-04-11 23:19:06 -05:00
Eric Biggers
aa38ead6c4 Use standard CC0 public domain dedication 2016-04-11 20:09:11 -05:00
Eric Biggers
d467d99740 ARM and MIPS test script tweaks 2016-01-30 21:54:06 -06:00
Eric Biggers
3aecf2e057 Add afl-fuzz files 2016-01-30 20:42:39 -06:00
Eric Biggers
95fa3faba8 Update years 2016-01-30 17:47:55 -06:00
Eric Biggers
fb1de37c0e Support for Microsoft C / Visual Studio 2016-01-30 17:27:02 -06:00
Eric Biggers
596bd3c661 Add gzip program and improve MinGW builds 2016-01-24 16:09:32 -06:00
Eric Biggers
ee1535ecc1 Make decompress API functions return a result code rather than a 'bool' 2016-01-23 11:38:56 -06:00
Eric Biggers
7be1effacc Make decompression routines optionally provide the actual uncompressd size 2016-01-23 01:04:28 -06:00
Eric Biggers
de7c13c659 Add MIPS test script 2015-12-05 00:33:59 -06:00