Key + Value Bloat

[victorstewart]

i'm trying to decide whether it's worth shaving a few dozen bytes off some of my values...

but i assume the minimum size of a value on disk is equal to either the physical or logical block size (as compacting would probably serve little practical value)? or maybe 4096 so that it's not device dependent?

and are the key and value put on the same block when possible or?

[smoyergh]

Values are packed tightly together and are not tied to the device block size, so with enough value data in the system you will realize the savings. I also recommend specifying value compression for your KVS.

Keys and values are stored separately, except in the case of tiny values (<= 8B) which are stored with their corresponding keys.

[victorstewart]

so leaving latency aside since its about 2 levels of magnitude below the operation times (~10us), using lz4 level 1 to write then read a 1MiB value with one NVMe drive writing at 2,480MiB/s and reading at 3,052MiB/s ...

~3077us compression time then ~210us transfer time to do compressed writes at ~304 MiB/s

~171us transfer time then ~424us decompression time to do compressed reads at ~1681 MiB/s

so write throughput drops by ~88% and read throughput drops by ~45%

obviously this gets much worse with striping.

are these back of the napkin costs of compression in the right ballpark?

Edit: forget to account for new compressed size, fixed that.

lz4 
level 1 
ratio 1.92
compression ~325 MiB/s
decompression ~1.2 Gib/s

https://quixdb.github.io/squash-benchmark/

[smoyergh]

lz4 compression/decompression throughput is a function of the size and nature of the data, core speed, and memory speed. The lz4 site (https://github.com/lz4/lz4) gives examples of 780 MB/s compression and 4970 MB/s decompression. YMMV.

One thing to keep in mind is these lz4 throughputs are per core, so if you have a multi-threaded app concurrently writing (or reading) KV pairs then you will get higher effective app throughput.

Another thing to keep in mind is the benefit you may get from the reduced physical size of the data. E.g., you can write 500KB of compressed data at one-half the throughput of 1MB of uncompressed data and achieve the same effective throughput.

In the end you really need to do some testing to determine if compression is a net win for your particular application from a performance perspective.