[OpenWrt-Devel] [PATCH 2/2] build: compress kernel debuginfo using zstd

[Paul Spooren]

Could we use that for the SDK/ImageBuilder as well?

Best,
Paul

-- 

May 13, 2020 9:20:39 AM Matthias Schiffer 
<mschiffer at universe-factory.net>:
> zstd with its default settings (compression level -3) compresses better
> than bzip2 -9 (which is the default setting), and is an order of 
magnitude
> faster.
>
> I made the following measurements for the most common compression tools
> (all standard Debian Buster versions, default flags unless noted
> otherwise), using the debug information of a large x86-64 kernel with
> ALL_KMODS:
>
> * kernel-debug.tar: 376M
> * kernel-debug.tar.gz: 101M, compressed in ~12s
> * kernel-debug.tar.bz2: 91M, compressed in ~15s
> * kernel-debug.tar.xz: 57M, compressed in ~101s
> * kernel-debug.tar.zst: 86M, compressed in ~1s
>
> With zstd, there is still some room for improvement by increasing the
> compression, but the slight increase in compression ratio
> (22.83% -> 19.46%) does not justify the significant increase in
> compression time (about 5 times on my machine) in my opinion.
>
> Note that multithreaded compression (-T argument) does not affect
> reproducibility with zstd.
>
> Signed-off-by: Matthias Schiffer <mschiffer at universe-factory.net>
> ---
> include/kernel-build.mk | 2 +-
> tools/Makefile          | 1 +
> 2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/include/kernel-build.mk b/include/kernel-build.mk
> index c371e78ab9b0..32c91a5b8359 100644
> --- a/include/kernel-build.mk
> +++ b/include/kernel-build.mk
> @@ -70,7 +70,7 @@ ifdef CONFIG_COLLECT_KERNEL_DEBUG
> $(FIND) $(KERNEL_BUILD_DIR)/debug -type f | $(XARGS) 
$(KERNEL_CROSS)strip --only-keep-debug
> $(TAR) c -C $(KERNEL_BUILD_DIR) debug \
> $(if $(SOURCE_DATE_EPOCH),--mtime="@$(SOURCE_DATE_EPOCH)") \
> -   | bzip2 -c -9 > $(BIN_DIR)/kernel-debug.tar.bz2
> +   | zstd -T0 -f -o $(BIN_DIR)/kernel-debug.tar.zst
> endef
> endif
>
> diff --git a/tools/Makefile b/tools/Makefile
> index 02556d4b83f3..169f36c5bb69 100644
> --- a/tools/Makefile
> +++ b/tools/Makefile
> @@ -37,6 +37,7 @@ tools-y += lzma squashfskit4 zip
> tools-$(BUILD_B43_TOOLS) += b43-tools
> tools-$(BUILD_ISL) += isl
> tools-$(CONFIG_USE_SPARSE) += sparse
> +tools-$(CONFIG_COLLECT_KERNEL_DEBUG) += zstd
> tools-$(CONFIG_TARGET_apm821xx)$(CONFIG_TARGET_gemini) += genext2fs
> tools-$(CONFIG_TARGET_tegra) += cbootimage cbootimage-configs
>
> --
> 2.26.2
>
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel at lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
>

_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel