diff --git a/doc/book/src/reference_manual/configuration.md b/doc/book/src/reference_manual/configuration.md index 4e292b95..9f88b232 100644 --- a/doc/book/src/reference_manual/configuration.md +++ b/doc/book/src/reference_manual/configuration.md @@ -102,12 +102,24 @@ and changing it everywhere at the time), but is not officially supported. ### `compression_level` -Zstd compression level to use for storing blocks. Can be a positive or negative integer, or -"none" to disable compression. Default value is 1, which consume low amount of cpu and should -work faster than line speed in most situations, while saving some space and intra-cluster +Zstd compression level to use for storing blocks. + +Values between `1` (faster compression) and `19` (smaller file) are standard compression +levels for zstd. From `20` to `22`, compression levels are referred as "ultra" and must be +used with extra care as it will use lot of memory. A value of `0` will let zstd choose a +default value (currently `3`). Finally, zstd has also compression designed to be faster +than default compression levels, they range from `-1` (smaller file) to `-99` (faster +compression). + +If you do not specify a `compression_level` entry, garage will set it to `1` for you. With +this parameters, zstd consumes low amount of cpu and should work faster than line speed in +most situations, while saving some space and intra-cluster bandwidth. -Higher values allow for more compression, at the cost of more cpu usage. Compression is done -synchronously, setting a value too high will add latency to write queries. + +If you want to totally deactivate zstd in garage, you can pass the special value `'none'`. No +zstd related code will be called, your chunks will be stored on disk without any processing. + +Compression is done synchronously, setting a value too high will add latency to write queries. This value can be different between nodes, compression is done by the node which receive the API call. diff --git a/script/test-smoke.sh b/script/test-smoke.sh index cfface65..1f900ece 100755 --- a/script/test-smoke.sh +++ b/script/test-smoke.sh @@ -30,6 +30,7 @@ dd if=/dev/urandom of=/tmp/garage.1.rnd bs=1k count=2 # No multipart, inline sto dd if=/dev/urandom of=/tmp/garage.2.rnd bs=1M count=5 # No multipart but file will be chunked dd if=/dev/urandom of=/tmp/garage.3.rnd bs=1M count=10 # by default, AWS starts using multipart at 8MB +# data of lower entropy, to test compression dd if=/dev/urandom bs=1k count=2 | base64 -w0 > /tmp/garage.1.b64 dd if=/dev/urandom bs=1M count=5 | base64 -w0 > /tmp/garage.2.b64 dd if=/dev/urandom bs=1M count=10 | base64 -w0 > /tmp/garage.3.b64