[OpenWrt-Devel] [PATCH] build: refactor JSON info files

[Paul Spooren]

Hi,

>> This PR refactors the JSON creation to store individual files in
>> $(KDIR)/tmp and create an single overview file called `profiles.json` in
>> the target dir.
>>
>> As before, this creation is enabled by default only for the BUILDBOT.
>>
>> To archive the previous behaviour the option JSON_INDIVIDUAL_JSON_INFO
>> can be set.
> Why do we need to preserve that previous behaviour?

Two reasons:

* We have to create those files anyway before a merge because image 
creation happens in parallel, so no single overview can be added to. If 
this is a wrong assumption please step in
* For ImageBuilders it is convenient to have a per profile file. It does 
not add much complexity as the file is simply copied.

I'd be in favor of keeping it if you wouldn't mind. It would be disabled 
for buildbots and users per default.

> FYI those individual files are broken[1] on some buildhosts (8C/16T, E5-2650) anyway:
>
>   Traceback (most recent call last):
>    File "/builds/ynezz/openwrt/scripts/json_add_image_info.py", line 49, in <module>
>      device_info = json.load(json_file)
>    File "/usr/lib/python3.5/json/__init__.py", line 268, in load
>      parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
>    File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
>      return _default_decoder.decode(s)
>    File "/usr/lib/python3.5/json/decoder.py", line 342, in decode
>      raise JSONDecodeError("Extra data", s, end)
>   json.decoder.JSONDecodeError: Extra data: line 35 column 2 (char 823)
>
> Would be nice to fix that and making this errors fatal so it's more visible and
> can be fixed sooner then later.
I'm looking into this issue but haven't figured it out yet. Either the 
JSON generation or file writing is broken. The odd thing about it is 
that it also happens if a file is opened only once (as in only a single 
image is created), so it is unlikely an issue with parallel writing into 
a file... If anyone has ideas, please share.
>> +jsonoverviewimageinfo: FORCE
> nitpickbutyoucanusedashorundescoreifyouwantforsuchmultiwordtargetsormakethetargetnameshorter :)
No this is a value comment and no nit pick, I just assumed wrongly that 
dashes/underscores should be avoided as they're not used in the Makefile 
so far.
> 1. https://ynezz.gitlab.io/-/openwrt/-/jobs/447337426/artifacts/logs/target/linux/install.txt

Best,
Paul


_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel