[OpenWrt-Devel] [RFC PATCH] build: re-enable parallelism for mksquashfs

[Daniel Santos]

Ahh, thank you both for the clarification.

On 11/08/2018 01:35 AM, Felix Fietkau wrote:
> On 2018-11-08 02:20, Daniel Santos wrote:
>> On 11/07/2018 01:52 PM, Felix Fietkau wrote:
>>> On 2018-11-05 00:19, Daniel Santos wrote:
>>>> This was disabled by commit dcd0e4a6727611f03eb3d3a75f073235f5f1229c due
>>>> to a threading bug back in 2009.  The specifics of the bug are not given
>>>> in the commit message and squashfs-tools has had several updates to it's
>>>> parallelism since this time.  There are currently no open issues related
>>>> to parallelism in their issue tracker:
>>>> https://github.com/plougher/squashfs-tools/issues
>>>>
>>>> It now "works for me" with 16 threads, and while this is a terrible test
>>>> for a race condition I still propose we remove this work-around
>>>> unless and until we have specific knowledge of a current bug.
>>>>
>>>> Signed-off-by: Daniel Santos <daniel.santos at pobox.com>
>>> Are the images still reproducible after that change?
>>> If I remember correctly, threading would break that.
>> Hello.  I'm not sure what you mean by the images being reproducible. 
>> I've been building with TARGET_ROOTFS_SQUASHFS=y and
>> TARGET_ROOTFS_SQUASHFS=256 for a few weeks now without any problem.  I'm
>> presuming they've fixed it, but I didn't see specific mention of this
>> bug in their github issue tracker.  They may have discussed it elsewhere
>> prior to porting to github.
> Reproducible as in https://reproducible-builds.org/
> You're supposed to be able to generate the binary-identical rootfs image
> on different machines, as long as you're using the same version and the
> same configuration.
> >From my understanding, using multiple threads for the squashfs build can
> make the order in which data appears in the filesystem image
> non-deterministic.
>
> - Felix

I see.  I have definitely not tested that.  I would assume they could
vary from one build to another based upon my understanding of how
multi-threaded compression is done in general.

How about a patch that allows multi-threaded compression via a .config
option?

Thanks,
Daniel



_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel