Add another performance example to README

This commit is contained in:
Marcus Holland-Moritz 2023-07-12 12:25:52 +02:00
parent 1b77fb77b3
commit 1b21be8804

View File

@ -53,6 +53,45 @@ than SquashFS compression**, it's **6 times faster to build the file
system**, it's typically faster to access files on DwarFS and it uses system**, it's typically faster to access files on DwarFS and it uses
less CPU resources. less CPU resources.
To give you an idea of what DwarFS is capable of, here's a quick comparison
of DwarFS and SquashFS on a set of video files with a total size of 39 GiB.
The twist is that each unique video file has two sibling files with a
different set of audio streams (I didn't make this up, this is
[an actual use case](https://github.com/mhx/dwarfs/discussions/63)). So
there's redundancy in both the video and audio data, but as the streams
are interleaved and identical blocks are typically very far apart, it's
quite challenging to make use of that redundancy for compression. SquashFS
essentially fails to compress the source data at all, whereas DwarFS is
able to reduce the size by almost a factor of 3, which is close to the
theoretical maximum:
```
$ du -hs dwarfs-video-test
39G dwarfs-video-test
$ ls -lh dwarfs-video-test.*fs
-rw-r--r-- 1 mhx users 14G Jul 2 13:01 dwarfs-video-test.dwarfs
-rw-r--r-- 1 mhx users 39G Jul 12 09:41 dwarfs-video-test.squashfs
```
While this is already impressive, it gets even better. When mounting
the SquashFS image and performing a random-read throughput test using
[fio](https://github.com/axboe/fio/)-3.34, both `squashfuse` and
`squashfuse_ll` top out at around 230 MiB/s:
```
$ fio --readonly --rw=randread --name=randread --bs=64k --direct=1 \
--opendir=mnt --numjobs=4 --ioengine=libaio --iodepth=32 \
--group_reporting --runtime=60 --time_based
[...]
READ: bw=230MiB/s (241MB/s), 230MiB/s-230MiB/s (241MB/s-241MB/s), io=13.5GiB (14.5GB), run=60004-60004msec
```
DwarFS, however, manages to sustain **random read rates of 20 GiB/s**:
```
READ: bw=20.2GiB/s (21.7GB/s), 20.2GiB/s-20.2GiB/s (21.7GB/s-21.7GB/s), io=1212GiB (1301GB), run=60001-60001msec
```
Distinct features of DwarFS are: Distinct features of DwarFS are:
- Clustering of files by similarity using a similarity hash function. - Clustering of files by similarity using a similarity hash function.