kanzi

Kanzi is a modern, modular, expendable and efficient lossless data compressor implemented in Go.

  • modern: state-of-the-art algorithms are implemented and multi-core CPUs can take advantage of the built-in multi-tasking.
  • modular: entropy codec and a combination of transforms can be provided at runtime to best match the kind of data to compress.
  • expandable: clean design with heavy use of interfaces as contracts makes integrating and expanding the code easy. No dependencies.
  • efficient: the code is optimized for efficiency (trade-off between compression ratio and speed).

Kanzi supports a wide range of compression ratios and can compress many files more than most common compressors (at the cost of decompression speed). It is not compatible with standard compression formats.

For more details, check https://github.com/flanglet/kanzi-go/wiki.

Credits

Matt Mahoney, Yann Collet, Jan Ondrus, Yuta Mori, Ilya Muravyov, Neal Burns, Fabian Giesen, Jarek Duda, Ilya Grebnov

Disclaimer

Use at your own risk. Always keep a backup of your files.

Build Status Go Report Card Total alerts Documentation

Silesia corpus benchmark

i7-7700K @4.20GHz, 32GB RAM, Ubuntu 20.04

go1.16rc1

Kanzi version 1.9 Go implementation. Block size is 100 MB.

Compressor Encoding (sec) Decoding (sec) Size
Original 211,938,580
Zstd 1.4.8 -2 –long=30 1.2 0.3 68,761,465
Zstd 1.4.8 -2 -T6 –long=30 0.7 0.3 68,761,465
Kanzi -l 1 2.8 1.3 68,471,355
Kanzi -l 1 -j 6 0.9 0.4 68,471,355
Pigz 1.6 -6 -p6 1.4 1.4 68,237,849
Gzip 1.6 -6 6.1 1.1 68,227,965
Brotli 1.0.9 -2 –large_window=30 1.5 0.8 68,033,377
Pigz 1.6 -9 -p6 3.0 1.6 67,656,836
Gzip 1.6 -9 14.0 1.0 67,631,990
Kanzi -l 2 3.6 1.3 64,522,501
Kanzi -l 2 -j 6 1.3 0.4 64,522,501
Brotli 1.0.9 -4 –large_window=30 4.1 0.7 64,267,169
Zstd 1.4.8 -9 –long=30 5.3 0.3 59,937,600
Zstd 1.4.8 -9 -T6 –long=30 2.8 0.3 59,937,600
Kanzi -l 3 4.8 2.3 59,647,212
Kanzi -l 3 -j 6 1.7 0.8 59,647,212
Zstd 1.4.8 -13 –long=30 16.0 0.3 58,065,257
Zstd 1.4.8 -13 -T6 –long=30 9.2 0.3 58,065,257
Orz 1.5.0 7.7 2.0 57,564,831
Brotli 1.0.9 -9 –large_window=30 36.7 0.7 56,232,817
Lzma 5.2.2 -3 24.1 2.6 55,743,540
Kanzi -l 4 10.6 6.9 54,996,858
Kanzi -l 4 -j 6 3.8 2.3 54,996,858
Bzip2 1.0.6 -9 14.9 5.2 54,506,769
Zstd 1.4.8 -19 –long=30 59.9 0.3 53,039,786
Zstd 1.4.8 -19 -T6 –long=30 59.7 0.4 53,039,786
Kanzi -l 5 12.4 6.5 51,745,795
Kanzi -l 5 -j 6 4.2 2.1 51,745,795
Brotli 1.0.9 –large_window=30 356.2 0.9 49,383,136
Lzma 5.2.2 -9 65.6 2.5 48,780,457
Kanzi -l 6 15.6 10.8 48,067,846
Kanzi -l 6 -j 6 5.3 3.7 48,067,846
BCM 1.6.0 -7 18.0 22.1 46,506,716
Kanzi -l 7 22.2 17.3 46,446,991
Kanzi -l 7 -j 6 8.0 6.2 46,446,991
Tangelo 2.4 83.2 85.9 44,862,127
zpaq v7.14 m4 t1 107.3 112.2 42,628,166
zpaq v7.14 m4 t12 108.1 111.5 42,628,166
Kanzi -l 8 63.4 64.6 41,830,871
Kanzi -l 8 -j 6 22.5 21.8 41,830,871
Tangelo 2.0 302.0 310.9 41,267,068
Kanzi -l 9 84.8 86.5 40,369,883
Kanzi -l 9 -j 6 33.8 33.5 40,369,883
zpaq v7.14 m5 t1 343.1 352.0 39,112,924
zpaq v7.14 m5 t12 344.3 350.4 39,112,924

enwik8

i7-7700K @4.20GHz, 32GB RAM, Ubuntu 20.04

go1.16rc1

Kanzi version 1.9 Go implementation. Block size is 100 MB. 1 thread

Compressor Encoding (sec) Decoding (sec) Size
Original 100,000,000
Kanzi -l 1 1.49 0.75 32,650,127
Kanzi -l 2 2.03 0.74 31,018,886
Kanzi -l 3 2.41 1.19 27,328,809
Kanzi -l 4 5.10 3.40 25,670,935
Kanzi -l 5 5.02 2.60 22,484,700
Kanzi -l 6 7.15 4.45 21,232,218
Kanzi -l 7 10.84 7.97 20,935,522
Kanzi -l 8 23.86 23.90 19,671,830
Kanzi -l 9 31.84 32.55 19,097,962

Build

There are no dependencies, making the project easy to build.

Option 1: go get

cd $GOPATH

go get github.com/flanglet/kanzi-go

cd src/github.com/flanglet/kanzi-go/app

go build Kanzi.go BlockCompressor.go BlockDecompressor.go InfoPrinter.go

Option 2: git clone

cd $GOPATH/src

mkdir github.com; cd github.com

mkdir flanglet; cd flanglet

git clone https://github.com/flanglet/kanzi-go.git

cd kanzi-go/app

go build Kanzi.go BlockCompressor.go BlockDecompressor.go InfoPrinter.go

GitHub

https://github.com/flanglet/kanzi-go