mirror of
https://github.com/mhx/dwarfs.git
synced 2025-08-03 17:56:12 -04:00
Add another performance example to README
This commit is contained in:
parent
1b77fb77b3
commit
1b21be8804
39
README.md
39
README.md
@ -53,6 +53,45 @@ than SquashFS compression**, it's **6 times faster to build the file
|
||||
system**, it's typically faster to access files on DwarFS and it uses
|
||||
less CPU resources.
|
||||
|
||||
To give you an idea of what DwarFS is capable of, here's a quick comparison
|
||||
of DwarFS and SquashFS on a set of video files with a total size of 39 GiB.
|
||||
The twist is that each unique video file has two sibling files with a
|
||||
different set of audio streams (I didn't make this up, this is
|
||||
[an actual use case](https://github.com/mhx/dwarfs/discussions/63)). So
|
||||
there's redundancy in both the video and audio data, but as the streams
|
||||
are interleaved and identical blocks are typically very far apart, it's
|
||||
quite challenging to make use of that redundancy for compression. SquashFS
|
||||
essentially fails to compress the source data at all, whereas DwarFS is
|
||||
able to reduce the size by almost a factor of 3, which is close to the
|
||||
theoretical maximum:
|
||||
|
||||
```
|
||||
$ du -hs dwarfs-video-test
|
||||
39G dwarfs-video-test
|
||||
$ ls -lh dwarfs-video-test.*fs
|
||||
-rw-r--r-- 1 mhx users 14G Jul 2 13:01 dwarfs-video-test.dwarfs
|
||||
-rw-r--r-- 1 mhx users 39G Jul 12 09:41 dwarfs-video-test.squashfs
|
||||
```
|
||||
|
||||
While this is already impressive, it gets even better. When mounting
|
||||
the SquashFS image and performing a random-read throughput test using
|
||||
[fio](https://github.com/axboe/fio/)-3.34, both `squashfuse` and
|
||||
`squashfuse_ll` top out at around 230 MiB/s:
|
||||
|
||||
```
|
||||
$ fio --readonly --rw=randread --name=randread --bs=64k --direct=1 \
|
||||
--opendir=mnt --numjobs=4 --ioengine=libaio --iodepth=32 \
|
||||
--group_reporting --runtime=60 --time_based
|
||||
[...]
|
||||
READ: bw=230MiB/s (241MB/s), 230MiB/s-230MiB/s (241MB/s-241MB/s), io=13.5GiB (14.5GB), run=60004-60004msec
|
||||
```
|
||||
|
||||
DwarFS, however, manages to sustain **random read rates of 20 GiB/s**:
|
||||
|
||||
```
|
||||
READ: bw=20.2GiB/s (21.7GB/s), 20.2GiB/s-20.2GiB/s (21.7GB/s-21.7GB/s), io=1212GiB (1301GB), run=60001-60001msec
|
||||
```
|
||||
|
||||
Distinct features of DwarFS are:
|
||||
|
||||
- Clustering of files by similarity using a similarity hash function.
|
||||
|
Loading…
x
Reference in New Issue
Block a user