27 Commits

Author SHA1 Message Date
Eric Biggers
166084acaa lib/deflate_compress: select min_size_to_compress based on level
The cutoff for outputting uncompressed data is currently < 16 bytes for
all compression levels.  That isn't ideal, since the higher the
compression level, the more we should bother with very small inputs; and
the lower the compression level, the less we should bother.

Use a formula that produces the following cutoffs:

        Level  Cutoff
        -----  ------
        0      56
        1      52
        2      48
        3      44
        4      40
        5      36
        6      32
        7      28
        8      24
        9      20
        10     16
        11     12
        12     8

Update https://github.com/ebiggers/libdeflate/issues/67
2020-10-18 18:37:51 -07:00
Eric Biggers
4c92394eaa Support level 0, "no compression"
Some users may require a valid DEFLATE, zlib, or gzip stream but know
ahead of time that particular inputs are not compressible.  zlib
supports "level 0" for this use case.  Support this in libdeflate too.

Resolves https://github.com/ebiggers/libdeflate/issues/86
2020-10-10 22:31:15 -07:00
Eric Biggers
27d5a74f03 lib: add freestanding support
Allow building libdeflate without linking to any libc functions by using
'make FREESTANDING=1'.  When using such a library build, the user will
need to call libdeflate_set_memory_allocator() before anything else,
since malloc() and free() will be unavailable.

[Folded in fix from Ingvar Stepanyan to use -nostdlib, and made
 freestanding_tests() check that no libs are linked to.]

Update https://github.com/ebiggers/libdeflate/issues/62
2020-04-17 22:32:49 -07:00
Eric Biggers
944500af9f lib: wrap the memory allocation functions
In preparation for adding custom memory allocator support, don't call
the standard memory allocation functions directly but rather wrap them
with libdeflate_malloc() and libdeflate_free().
2020-04-17 21:28:49 -07:00
Eric Biggers
66bd59c4be lib: rename the aligned allocation functions
In preparation for adding libdeflate_malloc() and libdeflate_free(),
rename the aligned allocation functions to match.
2020-04-17 21:28:49 -07:00
Eric Biggers
d1b6a825ab lib: merge aligned_malloc.h into lib_common.h
It's simpler to declare the library utility functions in lib_common.h
rather than use a separate header.
2020-04-17 21:28:49 -07:00
Eric Biggers
3acda56db0 Declare __stdcall correctly for MSVC
Unfortunately, MSVC only accepts __stdcall after the return type, while
gcc only accepts __attribute__((visibility("default"))) before the
return type.  So we need a macro in each location.

Also, MSVC doesn't define __i386__; that's gcc specific.  So instead use
'_WIN32 && !_WIN64' to detect 32-bit Windows.
2019-12-28 13:20:50 -06:00
Eric Biggers
2a2e24dc8b lib: fix some typos in comments 2019-08-24 17:38:50 -07:00
Eric Biggers
5038748d61 lib/deflate_compress: fix return value for output >= 4 GiB
The API returns the compressed size as a size_t, so
deflate_flush_output() needs to return size_t as well, not u32.
Otherwise sizes >= 4 GiB are truncated.

This bug has been there since the beginning.

(This only matters if you compress a buffer that large in a single go,
which obviously is not a good idea, but no matter -- it's a bug.)

Resolves https://github.com/ebiggers/libdeflate/issues/44
2019-05-21 21:13:59 -07:00
Eric Biggers
aadf6d8198 deflate_compress: produce same results on all CPUs 2017-05-29 17:44:50 -07:00
Eric Biggers
f2f0df7274 deflate_compress: fix corruption with long literal run
When the block splitting algorithm was implemented, it became possible
for the compressor to use longer blocks, up to ~300KB.  Unfortunately it
was overlooked that this can allow literal runs > 65535 bytes, while in
one place the length of a literal run was still being stored in a u16.
To overflow the litrunlen and hit the bug the data would have had to
have far fewer matches than random data, which is possible but very
unusual.  Fix the bug by reserving more space to hold the litrunlen, and
add a test for it.
2017-01-14 20:51:03 -08:00
Eric Biggers
0205f005bc deflate_compress: don't pass NULL to memcpy on empty input 2016-10-23 13:54:52 -07:00
Eric Biggers
7d3e2a997c Update licensing status
I've decided to simplify and standardize the licensing status for the
library by using the MIT license instead of CC0 (a.k.a. "public
domain").  This eliminates the somewhat controversial 4(a) clause in
CC0, and, for this and other reasons, should (somewhat ironically) make
it easier for some people to use and contribute to the project.

Note: copyright will apply to new changes and to new versions of the
work as a whole.  Of course, versions previously released as public
domain remain public domain where legally recognized.
2016-09-09 19:34:58 -07:00
Eric Biggers
e9a956efca Avoid "exporting" symbols from static library
It was reported that API symbols were being "exported" from the static
library built with MSVC, causing them to remain exported after being
linked into another program.  It turns out this was actually a problem
outside of MSVC as well.  The solution is to always build the static and
shared libraries from different object files, where the API symbols are
exported from the shared library object files but not from the static
library object files.

Reported-by: Joergen Ibsen <ji@ibse.dk>
2016-09-04 01:18:26 -07:00
Eric Biggers
c4f51b1311 Rename heapsort() to heap_sort() to avoid naming collision 2016-08-29 00:09:39 -07:00
Eric Biggers
18b149f52e Drop 'unsigned' from compression_level argument 2016-08-27 23:15:47 -07:00
Eric Biggers
3992efa340 Update API to avoid naming collisions 2016-08-27 23:11:32 -07:00
Eric Biggers
f649a4b8db Compressor updates 2016-06-11 22:41:45 -05:00
Eric Biggers
6bcfb00c54 C99 compatibility 2016-05-26 00:14:29 -05:00
Eric Biggers
992eb78578 Move block split statistics into compressor struct 2016-05-23 20:59:32 -05:00
Eric Biggers
453a628996 Fix compiling without near-optimal parsing support 2016-05-23 20:50:02 -05:00
Eric Biggers
7458bea118 Rename optimum -> optimum_nodes 2016-05-23 20:35:38 -05:00
Eric Biggers
4d81654402 Use tighter bound on max number of nodes 2016-05-23 20:35:38 -05:00
Eric Biggers
7bfba00a19 Fix comment 2016-05-23 19:08:20 -05:00
Eric Biggers
4f7d134b39 Fix check for match cache overflow 2016-05-23 08:28:02 -05:00
Eric Biggers
7e19d17e49 Fix deflate_compress_bound() to account for extra word at end 2016-05-21 17:35:35 -05:00
Eric Biggers
f2c3a5b4e9 Various reorganization and cleanups
* Bring in common headers and program code from xpack project
* Move program code to programs/
* Move library code to lib/
* GNU89 and MSVC2010 compatibility
* Other changes
2016-05-21 15:38:15 -05:00