perl/host/compile: fails on macOS 11 Big Sur, workaround

Tue Nov 24 14:11:51 EST 2020

> On Nov 23, 2020, at 2:52 PM, Rosen Penev <rosenp at gmail.com> wrote:
> 
> On Mon, Nov 23, 2020 at 10:39 AM Georgi Valkov <gvalkov at abv.bg> wrote:
>> 
>> Dear Rosen,
>> 
>> Thanks to your help I discovered the feeds/packages repository, however I was not able to resolve the race condition during build and discover why Errno.pm is not built when multiple workers are used e.g.: make package/feeds/packages/perl/host/{clean,compile} -j 12
>> 
>> The good news is that if we move to perl 5.30.0 or newer, the race condition is resolved.
> That's very good.
> 
>> We also need to add a few flags to files/base.config:
>> d_attribute_always_inline=‘define'
>> d_setlocale_accepts_any_locale_name=‘undef'
>> d_sockaddr_storage=‘undef’
>> d_towlower=‘undef'
>> d_towupper=‘undef'
>> d_wcrtomb=‘undef'
>> i_wctype=‘define’
>> 
>> Then we can compile perl/host 5.30.0 up to 5.32.0. However building perl for the target fails the install phase:
>> 
>> error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: input file: /Volumes/test/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0/ipkg-install/usr/bin/perl5.30.0 is not a Mach-O file
>> Cannot update /Volumes/test/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0/ipkg-install/usr/bin/perl5.30.0 dependency paths
>> make[3]: *** [makefile:445: install-all] Error 1
>> make[3]: Leaving directory '/Volumes/test/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0'
>> make[2]: *** [Makefile:152: /Volumes/test/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0/.built] Error 2
>> 
>> Another symptom I see, is that with 5.30.0 build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/perl/perl-5.30.0/ipkg-install/usr/bin contains only perl5.30.0, while with the original 5.28.1 it contains many executables.
>> 
>> Can you please update perl to 5.30.0 or 5.32.0? Just make sure it compiles correctly on Linux or whatever environment you are using, and I will test it on macOS 11. I seem to be missing something, and it might take me a very long time to learn my way through, because I lack experience.
> I don't maintain perl. That's https://github.com/pprindeville . CC'ed

If you have a patch to update to 5.30, I’ll look at it.

-Philip

>> 
>> 
>> Thank you again for any help!
>> Georgi Valkov
>> 
>> 
>>> On 2020-11-18, at 10:32 AM, Georgi Valkov <gvalkov at abv.bg> wrote:
>>> 
>>> 
>>> 
>>>> On 2020-11-18, at 4:24 AM, Rosen Penev <rosenp at gmail.com> wrote:
>>>> 
>>>> On Tue, Nov 17, 2020 at 3:16 PM Georgi Valkov <gvalkov at abv.bg> wrote:
>>>>> 
>>>>> Dear OpenWRT team,
>>>>> 
>>>>> I just updated my laptop to macOS 11 Big Sur. Presently only the perl hostpkg package fails during the build. Once I get past perl, everything else compiles correctly. Below is a patch, workaround, and details from my research. My target is Marvell EBU Armada, Linksys WRT3200ACM.
>>>>> 
>>>>> Before applying the patch, make package/feeds/packages/perl/host/compile V=sc fails with the following error:
>>>>> Which of these apply, if any? [darwin]
>>>>> 
>>>>> *** Unexpected product version 11.0.
>>>>> ***
>>>>> *** Try running sw_vers and see what its ProductVersion says.
>>>>> 
>>>>> After the patch, make package/feeds/packages/perl/host/compile succeeds, however when multiple workers are used: make package/feeds/packages/perl/host/compile -j 12 V=sc, the build fails:
>>>>> Writing Makefile for CPAN
>>>>> make[4]: Entering directory '/Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/Archive-Tar'
>>>>> make[4]: Entering directory '/Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/CPAN'
>>>>> Generating a Unix-style Makefile
>>>>> Writing Makefile for Errno
>>>>> make[4]: Entering directory '/Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/ext/Errno'
>>>>> "../../miniperl" "-I../../lib" Errno_pm.PL Errno.pm
>>>>> Can't locate Errno.pm in @INC (you may need to install the Errno module) (@INC contains: /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/AutoLoader/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/Carp/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/PathTools /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/PathTools/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/ExtUtils-Install/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/ExtUtils-MakeMaker/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/ExtUtils-Manifest/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/File-Path/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/ext/re /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/Term-ReadLine/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/Exporter/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/ext/File-Find/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/Text-Tabs/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/constant/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/cpan/version/lib /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/lib .) at /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/dist/PathTools/Cwd.pm line 444.
>>>>> Unsuccessful Makefile.PL(cpan/ExtUtils-MakeMaker): code=512 at make_ext.pl line 518.
>>>>> make[3]: *** [GNUmakefile:599: cpan/ExtUtils-MakeMaker/pm_to_blib] Error 2
>>>>> 
>>>>> I have also seen errors where the build is trying to use /miniperl instead of ./miniperl, as well as /build_dir/hostpkg/perl/perl-5.28.1/perl instead of /Volumes/test/openwrt/build_dir/hostpkg/perl/perl-5.28.1/perl. The will occur if we try to build perl without host/clean and clean. Seems like a race condition due to dependency misconfiguration.
>>>>> 
>>>>> Here is the list of all commands in order to compile OpenWRT starting from a clean build root:
>>>>> make -j 12
>>>>> make package/feeds/packages/perl/host/clean
>>>>> make package/feeds/packages/perl/clean
>>>>> make package/feeds/packages/perl/compile
>>>>> make -j 12
>>>>> 
>>>>> The first make fails, then we perform host-clean and clean for perl, next compile will also do host-compile - we have to use a single worker here. Finally we can complete the rest of the build process. I should note that Languages, Perl, perl is not selected in menuconfig. I can understand if the hostpkg/perl is required during the build process, however perl/host{clean,compile} is not enough, I also need to perform perl/{clean,compile}.
>>>>> 
>>>>> Here is the quick patch I made in order to verify if there are any other broken packages: no. Only perl fails to compile. I am also not sure how to handle patches and pull-requests for feeds, so I thought I should rather share my research here and ask for help.
>>>>> 
>>>>> feeds/packages/lang/perl/patches/302-macos_11_support.patch
>>>>> Index: perl-5.28.1/hints/darwin.sh
>>>>> ===================================================================
>>>>> --- perl-5.28.1.orig/hints/darwin.sh
>>>>> +++ perl-5.28.1/hints/darwin.sh
>>>>> @@ -327,7 +327,7 @@ EOM
>>>>>   # "ProductVersion:    10.11"     "10.11"
>>>>>       prodvers=`sw_vers|awk '/^ProductVersion:/{print $2}'|awk -F. '{print $1"."$2}'`
>>>>>   case "$prodvers" in
>>>>> -    10.*)
>>>>> +    10.*|11.*)
>>>>>     add_macosx_version_min ccflags $prodvers
>>>>>     add_macosx_version_min ldflags $prodvers
>>>>>     ;;
>>>>> 
>>>> This patch is for https://github.com/openwrt/packages
>>>> 
>>>> Please submit a PR there.
>>>> 
>>>> Speaking of perl, it could probably use an update.
>>> 
>>> I would only make a pull-request when I’m certain I have corrected all issues and tested thoroughly.
>>> Indeed the perl package must be updated. I’m not sure if I am allowed to create patches for /feeds, because
>>> /feeds and /package/feeds are in .gitignore, so I shouldn't force changes inside these directories.
>>> At least I’m not aware of a proper way to do so.
>>> 
>>>>> 
>>>>> Thank you for any help!
>>>>> Georgi Valkov
>>>>> 
>>>>> https://bugs.openwrt.org/index.php?do=details&task_id=3458
>>>>> 