btrfs compression

btrfs has built-in compression. (support for both lzo and zlib) This allows you to automatically compress data for the entire filesystem, subvolumes or even down to individual files. A mount option is all that’s needed to specify which compression type to use: compress=lzo,zlib. Not only will this use less diskspace, but will also improve IO at the small cost of CPU time and some memory.

To try this out, I took a simple setup where I used a linux kernel tree as a testcase. Here are the results:

mount the volume on /mnt and untar a linux-2.6.32.tar.bz2 tree onto /mnt

# btrfs filesystem df /mnt
Data, RAID1: total=1.00GB, used=369.81MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=57.75MB
Metadata: total=8.00MB, used=0.00

Now, lets do this again,

# mount -o compress=lzo /dev/sdf /mnt

# btrfs filesystem df /mnt
Data, RAID1: total=1.00GB, used=188.95MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=41.37MB
Metadata: total=8.00MB, used=0.00

So, 188.95M instead of 369.81M - pretty darned cool... go lzo. but.. what about zlib!

# mount -o compress=zlib /dev/sdf /mnt

# btrfs file df /mnt
Data, RAID1: total=1.00GB, used=128.75MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=35.45MB
Metadata: total=8.00MB, used=0.00

Down to 128M! Of course from a filesystem point of view you don’t have to do anything, just pass a mount option or set the attribute. As I mentioned earlier, btrfs lets you do this at a file level, a directory level (and inherit down for that directory) or all the way to the top level.

More cool bits : The compression uses kernel threads and will make use of as many threads as there are cpus. So compression gets loadbalanced/spread out across all threads in a server even if it’s a single big file, we will split up the big files into 128kb chunks and compress in parallel.
Another cool bit : if you have an existing uncompressed filesystem, and want to compress it, or even just compress a file on it, you can do that with btrfs filesystem defragment. The defragment command has an option -c that lets you specify zlib or lzo.