forked from Minki/linux
5c1aab1dd5
Add zstd compression and decompression support to BtrFS. zstd at its
fastest level compresses almost as well as zlib, while offering much
faster compression and decompression, approaching lzo speeds.
I benchmarked btrfs with zstd compression against no compression, lzo
compression, and zlib compression. I benchmarked two scenarios. Copying
a set of files to btrfs, and then reading the files. Copying a tarball
to btrfs, extracting it to btrfs, and then reading the extracted files.
After every operation, I call `sync` and include the sync time.
Between every pair of operations I unmount and remount the filesystem
to avoid caching. The benchmark files can be found in the upstream
zstd source repository under
`contrib/linux-kernel/{btrfs-benchmark.sh,btrfs-extract-benchmark.sh}`
[1] [2].
I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
16 GB of RAM, and a SSD.
The first compression benchmark is copying 10 copies of the unzipped
Silesia corpus [3] into a BtrFS filesystem mounted with
`-o compress-force=Method`. The decompression benchmark times how long
it takes to `tar` all 10 copies into `/dev/null`. The compression ratio is
measured by comparing the output of `df` and `du`. See the benchmark file
[1] for details. I benchmarked multiple zstd compression levels, although
the patch uses zstd level 1.
| Method | Ratio | Compression MB/s | Decompression speed |
|---------|-------|------------------|---------------------|
| None | 0.99 | 504 | 686 |
| lzo | 1.66 | 398 | 442 |
| zlib | 2.58 | 65 | 241 |
| zstd 1 | 2.57 | 260 | 383 |
| zstd 3 | 2.71 | 174 | 408 |
| zstd 6 | 2.87 | 70 | 398 |
| zstd 9 | 2.92 | 43 | 406 |
| zstd 12 | 2.93 | 21 | 408 |
| zstd 15 | 3.01 | 11 | 354 |
The next benchmark first copies `linux-4.11.6.tar` [4] to btrfs. Then it
measures the compression ratio, extracts the tar, and deletes the tar.
Then it measures the compression ratio again, and `tar`s the extracted
files into `/dev/null`. See the benchmark file [2] for details.
| Method | Tar Ratio | Extract Ratio | Copy (s) | Extract (s)| Read (s) |
|--------|-----------|---------------|----------|------------|----------|
| None | 0.97 | 0.78 | 0.981 | 5.501 | 8.807 |
| lzo | 2.06 | 1.38 | 1.631 | 8.458 | 8.585 |
| zlib | 3.40 | 1.86 | 7.750 | 21.544 | 11.744 |
| zstd 1 | 3.57 | 1.85 | 2.579 | 11.479 | 9.389 |
[1] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/btrfs-benchmark.sh
[2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/btrfs-extract-benchmark.sh
[3] http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia
[4] https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.11.6.tar.xz
zstd source repository: https://github.com/facebook/zstd
Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>
94 lines
3.0 KiB
Plaintext
94 lines
3.0 KiB
Plaintext
config BTRFS_FS
|
|
tristate "Btrfs filesystem support"
|
|
select CRYPTO
|
|
select CRYPTO_CRC32C
|
|
select ZLIB_INFLATE
|
|
select ZLIB_DEFLATE
|
|
select LZO_COMPRESS
|
|
select LZO_DECOMPRESS
|
|
select ZSTD_COMPRESS
|
|
select ZSTD_DECOMPRESS
|
|
select RAID6_PQ
|
|
select XOR_BLOCKS
|
|
select SRCU
|
|
|
|
help
|
|
Btrfs is a general purpose copy-on-write filesystem with extents,
|
|
writable snapshotting, support for multiple devices and many more
|
|
features focused on fault tolerance, repair and easy administration.
|
|
|
|
The filesystem disk format is no longer unstable, and it's not
|
|
expected to change unless there are strong reasons to do so. If there
|
|
is a format change, file systems with a unchanged format will
|
|
continue to be mountable and usable by newer kernels.
|
|
|
|
For more information, please see the web pages at
|
|
http://btrfs.wiki.kernel.org.
|
|
|
|
To compile this file system support as a module, choose M here. The
|
|
module will be called btrfs.
|
|
|
|
If unsure, say N.
|
|
|
|
config BTRFS_FS_POSIX_ACL
|
|
bool "Btrfs POSIX Access Control Lists"
|
|
depends on BTRFS_FS
|
|
select FS_POSIX_ACL
|
|
help
|
|
POSIX Access Control Lists (ACLs) support permissions for users and
|
|
groups beyond the owner/group/world scheme.
|
|
|
|
To learn more about Access Control Lists, visit the POSIX ACLs for
|
|
Linux website <http://acl.bestbits.at/>.
|
|
|
|
If you don't know what Access Control Lists are, say N
|
|
|
|
config BTRFS_FS_CHECK_INTEGRITY
|
|
bool "Btrfs with integrity check tool compiled in (DANGEROUS)"
|
|
depends on BTRFS_FS
|
|
help
|
|
Adds code that examines all block write requests (including
|
|
writes of the super block). The goal is to verify that the
|
|
state of the filesystem on disk is always consistent, i.e.,
|
|
after a power-loss or kernel panic event the filesystem is
|
|
in a consistent state.
|
|
|
|
If the integrity check tool is included and activated in
|
|
the mount options, plenty of kernel memory is used, and
|
|
plenty of additional CPU cycles are spent. Enabling this
|
|
functionality is not intended for normal use.
|
|
|
|
In most cases, unless you are a btrfs developer who needs
|
|
to verify the integrity of (super)-block write requests
|
|
during the run of a regression test, say N
|
|
|
|
config BTRFS_FS_RUN_SANITY_TESTS
|
|
bool "Btrfs will run sanity tests upon loading"
|
|
depends on BTRFS_FS
|
|
help
|
|
This will run some basic sanity tests on the free space cache
|
|
code to make sure it is acting as it should. These are mostly
|
|
regression tests and are only really interesting to btrfs
|
|
developers.
|
|
|
|
If unsure, say N.
|
|
|
|
config BTRFS_DEBUG
|
|
bool "Btrfs debugging support"
|
|
depends on BTRFS_FS
|
|
help
|
|
Enable run-time debugging support for the btrfs filesystem. This may
|
|
enable additional and expensive checks with negative impact on
|
|
performance, or export extra information via sysfs.
|
|
|
|
If unsure, say N.
|
|
|
|
config BTRFS_ASSERT
|
|
bool "Btrfs assert support"
|
|
depends on BTRFS_FS
|
|
help
|
|
Enable run-time assertion checking. This will result in panics if
|
|
any of the assertions trip. This is meant for btrfs developers only.
|
|
|
|
If unsure, say N.
|