The '-t' option of GNU gzip allows checking whether a gzip file is valid
without writing the data anywhere. It's relatively straightforward to
support in libdeflate-gzip too, so add support for it.
Resolves https://github.com/ebiggers/libdeflate/issues/125
[EB - updated commit message]
Include sys/types.h to avoid the following build failure on uclibc:
In file included from programs/gzip.c:28:0:
programs/prog_util.h:159:1: error: unknown type name ‘ssize_t’
ssize_t xread(struct file_stream *strm, void *buf, size_t count);
^
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Avoid confusion with the GNU extension 'program_invocation_name', which
is described by 'man 3 program_invocation_name'. The GNU version isn't
supposed to be exposed without defining _GNU_SOURCE, which we don't in
any of the relevant files, but it's best to avoid any confusion.
Some users may require a valid DEFLATE, zlib, or gzip stream but know
ahead of time that particular inputs are not compressible. zlib
supports "level 0" for this use case. Support this in libdeflate too.
Resolves https://github.com/ebiggers/libdeflate/issues/86
stdint.h is better compatible with freestanding support as it can be
compiled even if target platform lacks I/O capabilities.
[EB - adjusted the include locations, and avoided breaking the build
for old MSVC versions.]
Allow building libdeflate without linking to any libc functions by using
'make FREESTANDING=1'. When using such a library build, the user will
need to call libdeflate_set_memory_allocator() before anything else,
since malloc() and free() will be unavailable.
[Folded in fix from Ingvar Stepanyan to use -nostdlib, and made
freestanding_tests() check that no libs are linked to.]
Update https://github.com/ebiggers/libdeflate/issues/62
Adds programs/test_zlib.c which currently does the following:
1. Create some dummy data and compress it
2. Try to decompress with libdeflate_zlib_decompress
3. Try to decompress with libdeflate_zlib_decompress, with unnecessary
trailing bytes after the compressed data
4. Try to decompress with libdeflate_zlib_decompress_ex, with
unnecessary trailing bytes after the compressed data
In each step, we check that we get back the original data.
Not all programming languages support the cdecl calling convention.
stdcall is what the Win32 API uses, and it seems to be the better choice
for maximum compatibility with other programming languages.
So, switch from cdecl to stdcall.
Resolves https://github.com/ebiggers/libdeflate/issues/58
Apparently, mmap'ing /dev/zero fails on macOS. So contrary to the
intent, it's not actually portable. Switch to MAP_ANONYMOUS instead,
with a fallback to MAP_ANON if it's not defined.
libdeflate's 'gunzip' program fails if the uncompressed file size is a
nonzero multiple of 4 GiB because then the ISIZE field in the gzip file
is 0, so gunzip allocates no space for the decompressed data. Then when
libdeflate returns LIBDEFLATE_INSUFFICIENT_SPACE, gunzip doubles the
buffer size which stays 0, causing it to report "file corrupt or too
large to be processed by this program".
While neither libdeflate nor its 'gunzip' program is designed for single
gzip streams this large anyway, this particular bug can easily be fixed
by always allocating at least one byte for the decompressed data.
Move program utility functions that are used only by "test programs"
(i.e. not by gzip/gunzip) from prog_util.{c,h} into test_util.{c,h}.
This reduces the code that is compiled for the default build target,
which excludes the test programs.
Another build_decode_table() optimization: rather than filling all the
entries for each codeword using strided stores, just fill one initially
and fill the rest by memcpy()s as the table is incrementally expanded.
Also make some other cleanups and small optimizations.
Improve libdeflate's worst-case performance decompressing malicious
DEFLATE streams by about 14x, bringing it within a factor of about 2x of
zlib, by skipping rebuilding the decode tables for the static Huffman
codes when they're already loaded into the decompressor.
This improves performance decompressing a stream of all empty static
Huffman blocks from about 0.36 MB/s to 175 MB/s, or the original
reproducer given on the Github issue from about 3.3 MB/s to 219 MB/s.
A regression test is added for these cases as well as the empty dynamic
Huffman blocks case to verify worst-case performance comparable to zlib.
Resolves https://github.com/ebiggers/libdeflate/issues/33
