[OpenWrt-Devel] couple questions about downloading for a new build

[Yousong Zhou]

On 11 September 2014 21:12, Robert P. J. Day <rpjday at crashcourse.ca> wrote:
> On Thu, 11 Sep 2014, Yousong Zhou wrote:
>
>> On 11 September 2014 20:30, Robert P. J. Day <rpjday at crashcourse.ca> wrote:
>> >
>> >   first, i created a local mirror with all the tarballs i
>> > downloaded during an earlier build, and configured a new build to
>> > use that, and i noticed that the download operation *copied* over
>> > the tarballs into the dl/ directory. is there any reason that
>> > symlinks aren't used, rather than doing actual copies? wouldn't
>> > symlinks work just as well?
>>
>> I take mirrors as places where the build system will try to fetch
>> those tarballs, into CONFIG_DOWNLOAD_FOLDER which defaults to
>> $(TOPDIR)/dl/ which in my local tree is a symbolic link to the
>> actually directory holding those tarballs.  It works quite well.
>
>   i realize that's *one* way to do it, and it's how i did it
> originally, but i chose instead to set up a single local mirror
> directory that each build can "fetch" from into their personal dl/
> directory, whereupon i noticed that fetching from that local mirror
> really is doing a full copy, whereas it *seems* that just doing a
> symlink would work just as well and not use up all that disk space.
>
>   i'm just wondering why symlinks aren't used.

Well, the current design decision works for me and probably for many
others.  If it is not good enough and causing some inconvenience, then
patches are expected to enhance it.

>
>> >   also, i wondered if building the host tools would take advantage
>> > of already-installed host tools that match the exact version that
>> > openwrt needs. in my new build, the version of "patch" required is
>> > ostensibly 2.7.1, which is precisely the version already installed
>> > on my host, but it appears the patch-2.7.1.tar.xz tarball was
>> > still copied (from the local mirror, admittedly) into the dl/
>> > directory.
>>
>> IMHO, for a build system complex like OpenWrt, it's better to be
>> self-contained and do things in a more predictable/controlled way.
>
>   i agree, it was just the way i was reading the comment below that i
> think confused me.
>
>> >   i haven't followed the Makefile structures far enough yet to see
>> > what should happen, but a comment in tools/Makefile reads:
>> >
>> > # in case there is no patch tool on the host we need to make patch
>> > # tool a dependency for tools which have patches directory
>> >
>> > which *seems* to suggest that the build will look for patch already
>> > installed on the host and take advantage of it if it's there.
>>
>> No, it tries to handle the case that some host tools needs to be
>> patched (having patches/ directory) before getting compiled in which
>> case those tools/XXX/compile needs to have a dependence on
>> tools/patch/install.
>
>   ok, i think i see what that comment means ... if any tool needs
> patching, then patch is registered as a necessary host tool and it
> *will* be downloaded and built, even if it already exists on the host,
> correct?
>
>   however, assuming i understand that, there are now two new things
> that puzzle me. first, i selected CONFIG_CCACHE and, as i read it,
> that would normally mean i would need the ccache utility to be
> downloaded and built as a host tool (even if it's already installed,
> yes?). but it's not. even though:
>
> CONFIG_CCACHE=y
>
> i see no ccache tarball in my dl/ direectory. ccache is, in fact,
> installed on my build host, but i thought we just agreed that that
> shouldn't make a difference. so what would force ccache to be
> downloaded and built?
>

Hmm, i haven't tried ccache myself, at least not in the way i am fully
aware of.  But as said above, I tend to think that the current shape
of OpenWrt is formed gradually by needs, of users or developers.  If
the then implementation or design decision does not work, patches are
applied to solve it, in trunk or in developers' local tree.

>   finally, mtd-utils -- the recipe specifies both a version number and
> git info in the Makefile:
>
> PKG_NAME:=mtd-utils
> PKG_VERSION:=1.4.5
>
> PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
> PKG_SOURCE_URL:=git://git.infradead.org/mtd-utils.git
> PKG_SOURCE_PROTO:=git
> PKG_SOURCE_VERSION:=5319b84974fcb71504aed2d1b8285e9c0a4a4bb8
> PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
> PKG_CAT:=zcat
>
>   now, the first time i downloaded all the tools, git cloned the
> mtd-utils repository, and created the tarball mtd-utils-1.4.5.tar.gz.
> however, even though i added that to my local mirror directory, the
> download uses a "git clone" every time. should it not look in my local
> mirror, and notice that the tarball is there? or does that not apply
> to source that is cloned from a git repo?

Maybe it's just that the current code does not bother :)  as there are
(easy) ways to work around this.

The current build system only checks availabilities of tarballs in
CONFIG_DOWNLOAD_FOLDER.  Myabe mirrors are supposed to be remote.
Anyway, you can patch this in your local tree or try to convince
developers to accept the change into OpenWrt trunk.


               yousong
_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel