[OpenWrt-Devel] blob vs. blobmsg - simplifying & cleaning libubox API

[Rafał Miłecki]

* Introduction *

OpenWrt heavily uses blob-s and blobmsg-s implemented in libubox. They
are used in ubus and when parsing JSON-s.

Short summary for two formats by Jo:

[14:51] <jow> I think a blobmsg is a structure nested into a blob
[14:52] <jow> a blob is  id_len (8 bits type (aka ID), 24 bits length) + data payload
[14:53] <jow> a blobmsg follows the id_len and consists of a name_len (16bit) and a variable length name string
[14:53] <jow> any payload data then follows the variable length name (32bit aligned)
[14:54] <jow> I think the extra set of blobmsg_*() apis is needed to handle the blob attributes while somehow transparently dealing with the name TLV squeezed in front
[14:55] <jow> due to that, there's different notions of "payload"
[14:55] <jow> a blob payload is simply the data after the id_len member (pointing to the start of the blobmsg hdr)
[14:56] <jow> a blobmsg payload is the actual payload after the id_len member, the namelen member and the variable length name


* Problem *

Some/many developers are confused regarding what do they deal with (blob
vs. blobmsg). There is single struct blob_attr that gets passed around
but it isn't clear if it contains blob or blobmsg.

Depending on blob vs. blobmsg type a correct set of API functions should
be used. E.g.
1. blob_len() vs. blobmsg_data_len()
2. blob_for_each_attr() vs. blobmsg_for_each_attr()

There are many cases where blobmsg_data() is used for blob format. It
works thanks to some extra check in the blobmsg_data().

Naming is confusing too:
1. "blobmsg" could be "namedblob" or "blob_named" or "blob_with_name"
2. blobmsg_parse() requires passing *blob* data not blobmsg data


* Cleaning API usage *

One idea for cleaning up blob(msg) dependant code was to make it always
call right functions (blob_* for blob and blobmsg_* for blobmsg). I
started looking at this but it's not that obvious.

I realized that ubus method handlers always receive struct blob_attr
that is *blob*. Some projects call blobmsg_data() and blobmsg_len() on
it that isn't 100% clean solution. Of course having blobmsg_parse() name
doesn't help.

I planned to replace all such blobmsg_data() calls with blob_data() but
then I realized that *nested* tables will actually need blobmsg_data().
Parsing fails if blobmsg data gets passed.


* Looking for ideas *

I'm looking for ideas how to simplify and clean blob(msg) API. The only
idea I have is merging blob & blogmsg APIs so all functions get aware of
both types. I suggested that but Felix pointed out it's not a good idea:

[17:12] <nbd> not sure if that makes sense, they're used differently
[17:12] <nbd> blob is indexed by id
[17:12] <nbd> in blobmsg, the id specifies the type and fields are indexed by name
(...)
[17:39] <nbd> well, blobmsg is a layer around blob
[17:39] <nbd> so i don't think the blob api should deal with blobmsg specifics directly

Any suggestions?

_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel