[OpenWrt-Devel] [PATCH 2/2] build: compress kernel debuginfo using zstd

[Matthias Schiffer]

On 5/16/20 3:38 AM, Paul Spooren wrote:
> Could we use that for the SDK/ImageBuilder as well?
> 
> Best,
> Paul

Sure, have some benchmark results with the current ath79 snapshot IB:

* openwrt-imagebuilder-ath79-generic.Linux-x86_64.tar: 515M
* xz -7e (current implementation): 96M, ~100s
* zstd -3 (default setting): 248M, ~1s
* zstd -19: 112M, ~73s
* zstd --ultra -20: 93M, ~97s
* zstd --ultra -22: 78M, ~136s

So it seems at --ultra -20 we have a sweet spot where we surpass xz -7e in
both compression ratio and speed. --ultra just unlocks the highest
compression levels >19, at the cost of addional memory use for both
compression and decompression.

All of the above measurements were made without multithreading. The
multithreaded mode of xz (as used in OpenWrt master) makes compression
worse, while zstd produces an idential file with and without multithreading:

* xz -7e -T32: 113M, ~17s
* zstd -19 -T32: 112M: ~12s
* zstd --ultra -20 -T32: 93M, ~30s

IMO the fact that xz compresses differently depending on the number of
threads used is also a big problem, as it makes it harder to make builds
fully reproducible.

If we switch to zstd for SDK/IB, I would go for level -19: it's
significantly faster than --ultra -20, especially in multithreaded mode,
and the size difference seems acceptable to me (in fact, our current
snapshot builds use xz in multithreaded mode, so the current IB tar.xz on
the download server is 113M, not 96M.

Matthias

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.infradead.org/pipermail/openwrt-devel/attachments/20200516/4bc42cc9/attachment.sig>
-------------- next part --------------
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel