[OpenWrt-Devel] OpenWRT: how to include byte-compile .pyc files into sysupgrade image?

[Alexandru Ardelean]

Hey Xuebing,

Apologies for the late response.
Short version: please checkout this PR
https://github.com/openwrt/packages/pull/4130
I am hoping you can backport these changes to OpenWrt CC 15.05

I've managed to package Python & Python3 with .pyc bytecodes.
And it seems to work fine. I tested with mpc85xx & x86_64, but please
come back with a reply if it does not work for you and I can
investigate.
Maybe it's different for your architecture.

That PR may be a bit more than you need, since it splits Python
packages into Python Source packages and Python packages [ which now
ship with .pyc files ].
Funny story: I was approached about another user about this, but he
was happy with just using  `python -m compileall`

I tested both Python & Python3 versions.
All looked good from what I could tell.

Also, please notice anything in that PR that you could use to save
some space on your flash device.
I noticed that packages were shipping with test files ; I removed them.

Hope this helps.

Thanks
Alex

On Sat, Feb 18, 2017 at 1:26 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
> Hi Alex,
>
> Thank you so much for your help.
>
> xuebing wang
>
>
>
> On 2017年02月17日 23:37, Alexandru Ardelean wrote:
>>
>> @Xuebing: so, there doesn't seem to be a very easy way to do this at build
>> time.
>> I mean, at first, it can seem easy, because all that is needed, is to
>> add script that compiles here:
>>
>> https://github.com/openwrt/packages/blob/for-15.05/lang/python/files/python-package.mk#L94
>>
>> That script needs to recursively compile all .py files and
>> [optionally]  remove them.
>> So, feel free to try/hack something for your case.
>>
>> Maybe a shell script that uses this:
>> http://effbot.org/zone/python-compile.htm
>>
>>> compile.py
>>
>> import os
>> import sys
>> import py_compile
>> for file in sys.argv:
>>      py_compile.compile(file)
>>      os.remove(file)
>>
>> find . -name "\.py" | xargs compile.py
>>
>>
>> For the general case [in trunk], I think I have to start considering
>> splitting the Python source files and Python byte-compiled files into
>> 2 package types.
>> There was a older discussion about this ; the idea is to package the
>> byte coded files, and offer the sources and python source packages.
>> Example: Have `python` package and `python-sources`  package.
>> `python` == byte-compiled .pyc files, `python-sources` .py files
>>
>> Will try to make an effort in the next couple of days to implement the
>> general solution for this.
>> But, it's Friday now here, and I usually don't do much in the weekend.
>> So, hopefully next week I could allocate some time for this.
>>
>> Thanks
>> Alex
>>
>> On Fri, Feb 17, 2017 at 9:42 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
>>>
>>> Hi Alex,
>>>
>>> Thanks a lot for your help.  I am using Python 2.7.12, branch
>>> chaos_calmer
>>> from (git://github.com/openwrt/openwrt.git), feeds branch is for-15.05.
>>>
>>> Also, I am using Atheros platform based on MIPS architecture.
>>>
>>> Thanks again.
>>>
>>> xuebing wang
>>>
>>>
>>> On 2017年02月17日 15:31, Alexandru Ardelean wrote:
>>>>
>>>> Oh.
>>>>
>>>> This is an old topic.
>>>> Let me think about it, for a bit.
>>>>
>>>> Quick q: which Python version are you using and from which repo/branch ?
>>>> I usually test against trunk.
>>>>
>>>> Thanks
>>>> Alex
>>>>
>>>> On Fri, Feb 17, 2017 at 8:40 AM, Xuebing Wang <xbing6 at gmail.com> wrote:
>>>>>
>>>>> Hi Community and Alexandru:
>>>>>
>>>>> I am using OpenWrt chaos_calmer for an Atheros based platform, we have
>>>>> a
>>>>> complicated Python application which takes time to run.
>>>>> - On the target board, I can "python -m compileall ." for Python
>>>>> standard
>>>>> libraries in directories "/usr/lib/python2.7", and boot time can
>>>>> improve
>>>>> significantly.
>>>>>
>>>>> Would you please advise how can I change in files
>>>>> feeds/packages/lang/python
>>>>> to byte-compile Python libraries and include .pyc files in the
>>>>> sysupgrade
>>>>> image?
>>>>>
>>>>> I am using below packages:
>>>>> CONFIG_PACKAGE_python-base=y
>>>>> CONFIG_PACKAGE_python-light=y
>>>>> CONFIG_PACKAGE_python-codecs=y
>>>>> CONFIG_PACKAGE_python-openssl=y
>>>>> CONFIG_PACKAGE_python-sqlite3=y
>>>>> CONFIG_PACKAGE_python-logging=y
>>>>>
>>>>> Thanks.
>>>>> xuebing wang
>>>>>
>
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel