Revisiting ABI_VERSION handling policy

[Jo-Philipp Wich]

Hi,

> The goal of reducing unnecessary build churn makes perfect sense to me,
> but I think we need to reconsider the trade-off we're making here, and
> the failure modes of each option.

the motivation wasn't about build-churn at all but to ensure that library
packages can be cleanly upgraded and that different versions of libraries can
coexist on the same system without trashing userspace mid-upgrade. The other
major consideration is hard package version dependencies to ensure that
library users actually depend on the binary library ABI version they've been
linked against at build time.

> I believe that having to manually consider whether the ABI changed when
> updating a package comes with a nasty foot-gun that can deliver very
> subtle and hard to debug failure cases.

Upgrading a very central library package like wolfSSL which is used by uclient
(and thus opkg), wpad (wireless connectivity) and uhttpd through ustream
(webinterface access) is a sensible operation. In my view, sanely maintaining
the ABI_VERSION is simply part of the work when dealing with such a package.

Throwing all that over board just because you've been annoyed by a bug that
cost you a few hours is not the right way forward. By that metric, we
should've dropped most of buildroot years ago already.

> To fix this situation, I would like to switch wolfssl and maybe a few
> other packages prone to ABI breakage back to version based ABI_VERSION.
> At least wolfssl also needs a short suffix generated from config vars.

It makes little sense to peg the ABI_VERSION to the package version, at least
without revising the packaging at the same time. The ABI version should only
change if the shipped shared objects become incompatible (or more
specifically, if the SONAME changes).

By tying the ABI version to the package version without also ensuring that the
shipped .so files are differently named, you'll also introduce file level
package conflicts. A sanely managed ABI_VERSION should ensure that it is
possible for libfoo1 (ABI_VERSION 1) and libfoo2 (ABI_VERSION 2) to coexist on
the system which is a precondition for being able to cleanly opkg-upgrading
library packages without trashing the currently running userspace.

Especially for libraries with frequent security updates like wolfSSL, I'd
rather retain this ability.

> If we want to reduce build churn, we should find a way to automate
> setting the ABI version, since clearly our package maintainers + review
> process cannot (and in my opinion also should not) be trusted with this
> extra responsibility.

It should be possible to use readelf/objdump to examine the exported symbols
of a given shared object and see if either the size of a symbol changed or if
previously present ones were removed - addition should be fine.

Maybe this is something that can be added to the FIXUP=1 stuff.

> Maybe we can replace it with a hash of all installed header files. This
> might be a better way to resolve cases like this wolfssl instance, but
> we likely will still be rebuilding dependent packages on most updates.

A hash of installed header files makes little sense imho as it does not really
tell which symbols are exported by a shared object in the end. To me it seems
you conflate two different problems here:

a) maintaining ABI versions
b) ABI depending on compile time options

Unless I missed something, it appears your original issue was due to b) and it
can be solved by appending a hash of CONFIG_* symbol values to the explicitly
set ABI version.



Regards,
Jo

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openwrt.org/pipermail/openwrt-devel/attachments/20210209/c95422f1/attachment.sig>