Persistent HTTP(S) connections in opkg

[Hauke Mehrtens]

On 12/31/20 8:40 PM, Hauke Mehrtens wrote:
> On 12/30/20 7:14 PM, Baptiste Jonglez wrote:
>> Following the discussion in 
>> http://lists.openwrt.org/pipermail/openwrt-devel/2020-November/032297.html 
>>
>>
>> There are basically two options to implement persistent connections in 
>> opkg:
>>
>> 1) keep calling "wget", but pass the list of all packages to download at
>>     once.  On the host, wget already implements persistent connections.
>>     For devices, we would need to implement persistent connections in
>>     uclient-fetch (which is what "wget" actually points to).
>>
>> 2) switch to using a HTTP library (libuclient or libcurl), so that we can
>>     keep some TCP/TLS/HTTP context between downloads.
>>
>> The first solution has a major drawback on devices: all packages would
>> need to be downloaded to /tmp, which will consume memory.  Currently, 
>> opkg
>> processes packages individually, so only one package at a time is stored
>> in /tmp.
>>
>> The second solution adds a new library dependency, and we need to make
>> sure that it works both on the host and on targets.  Currently, we don't
>> make libuclient available to the host build system.  We would need to
>> build it for the host and link opkg statically against it (like it's done
>> for libubox).
>>
>> Overall, I think the second solution makes more sense and is easier to 
>> integrate.
>> I would go with libuclient because we already have it available on
>> devices.
>>
>> Any thoughts?
>>
>> Thanks,
>> Baptiste
> 
> Hi,
> 
> I looked into performance problems of LuCI when using https some time ago.
> 
> The slow part was the handshake, the normal stream cipher is relatively 
> fast, even very slow devices should be able to do multiple MB/s.
> 
> On the server side the ECC handshake was much faster, I think I measured 
> values like 1 second (RSA) vs. 0.3 seconds (ECC) for the handshake on a 
> Lantiq MIPS 24Kec CPU with mbedtls, the RSA handshake was much faster 
> (0.5 seconds) with openssl.
> 
> We should activate support for ECC certificates on 
> https://downloads.openwrt.org, I think it is possible to use both RSA 
> and ECC on the server and then decide based on what the client supports 
> and wants. In OpenWrt we could use then ECC to authenticate the server. 
> The crypto parts should already be there as we need ECDH for SAE in 
> hostapd.
> 
> We could use TLS Session Resumption, the SSL libraries should support 
> it, this way we can easily reuse the same session for the next download. 
> Browsers do this to only do one SSL handshake and then have multiple TCP 
> connections to the server to download the material in parallel.
> 
> Hauke

The changes I did are here:
https://git.openwrt.org/f2c8f6dc3249b506b915741d12905402dfffe162
https://git.openwrt.org/e8a1469

These are only for mbedtls and not for openssl/wolfssl it could be that 
this causes some problems. I would also assume that openssl is much 
faster than wolfssl, so we do not see these problems so much with openssl.

Hauke

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openwrt.org/pipermail/openwrt-devel/attachments/20201231/a0809363/attachment.sig>