[OpenWrt-Devel] [RFC] [PATCH v4] lantiq: IRQ balancing, ethernet driver, wave300

Mon Mar 25 19:24:36 EDT 2019

Hi Petr

On 3/14/19 6:46 AM, Petr Cvek wrote:
> Hello again,
> 
> I've managed to enhance few drivers for lantiq platform. They are still
> in ugly commented form (ethernet part especially). But I need some hints
> before the final version. The patches are based on a kernel 4.14.99.
> Copy them into target/linux/lantiq/patches-4.14 (cleaned from any of my
> previous patch).

Thanks for working on this.

> The eth+irq speedup is up to 360/260 Mbps (the vanilla was 170/80 on my
> setup). The iperf3 benchmark (2 passes for both vanilla and changed
> versions) altogether with script are in the attachment.
> 
> 1) IRQ with SMP and balancing support:
> 
> 	0901-add-icu-smp-support.patch
> 	0902-enable-external-irqs-for-second-vpe.patch
> 	0903-add-icu1-node-for-smp.patch
> 
> As requested I've changed the patch heavily. The original locking from
> k3b source code (probably from UGW) didn't work and in heavy load the
> system could have froze (smp affinity change during irq handling). This
> version has this fixed by using generic raw spinlocks with irq.
> 
> The SMP IRQ now works in a way that before every irq_enable (serves as
> unmask too) the VPE will be switched. This can be limited by writing
> into /proc/irq/X/smp_affinity (it can be possibly balanced from
> userspace too).
> 
> I've rewritten the device tree reg fields so there are only 2 arrays
> now. One per an icu controller. The original one per module was
> redundant as the ranges were continuous. The modules of a single ICU are
> now explicitly computed in a macro:
> 
> 	ltq_w32((x), ltq_icu_membase[vpe] + m*0x28 + (y))
> 	ltq_r32(ltq_icu_membase[vpe] + m*0x28 + (x))
> 
> before there was a pointer for every 0x28 block (there shouldn't be
> speed downgrade, only a multiplication and an addition for every
> register access).
> 
> Also I've simplified register names from LTQ_ICU_IM0_ISR to LTQ_ICU_ISR
> as "IM0" (module) was confusing (the real module number 0-4 was a part
> of the macro).
> 
> The code is written in a way it should work fine on a uniprocessor
> configuration (as the for_each_present_cpu etc macros will cycle on a
> single VPE on uniprocessor). I didn't test the no CONFIG_SMP yet, but I
> did check it with "nosmp" kernel parameter. It works.
> 
> Anyway please test if you have the board where the second VPE is used
> for FXS.
> 
> The new device tree structure is now incompatible with an old version of
> the driver (and old device tree with the new driver too). It seems icu
> driver is used in Danube, AR9, AmazonSE and Falcon chipset too. I don't
> know the hardware for these boards so before a final patch I would like
> to know if they have a second ICU too (at 0x80300 offset).

Normally the device tree should stay stable, but I already though about
the same change and I am not aware that any device ships a U-Boot with
an embedded device tree, so this should be fine.

The Amazon and Amazon SE only have one ICU block because they only have
one CPU with one VPE.
The Danube SoC has two ICU blocks one for each CPU, each CPU only has
one VPE. The CPUs are not cache coherent, SMP is not possible.

Falcon, AR9, VR9, AR10, ARX300, GRX300, GRX330 have two ICU blocks one
for each VPE of the single CPU.
GRX350 uses a MIPS InterAptiv CPU with a MIPS GIC.

> More development could be done with locking probably. As only the
> accesses in a single module (= 1 set of registers) would cause a race
> condition. But as the most contented interrupts are in the same module
> there won't be much speed increase IMO. I can add it if requested (just
> spinlock array and some lookup code).

I do not think that this improves the performance significantly, I
assume that the CPUs only have to wait there in rare conditions anyway.

> 2) Reworked lantiq xrx200 ethernet driver:
> 
> 	0904-backport-vanilla-eth-driver.patch
> 	0905-increase-dma-descriptors.patch
> 	0906-increase-dma-burst-size.patch
> 
> The code is still ugly, but stable now. There is a fragmented skb
> support and napi polling. DMA ring buffer was increased so it handle
> faster speeds and I've fixed some code weirdness. A can split the
> changes in the future into separate patches.

It would be nice if you could also do the same changes to the upstream
driver in mainline Linux kernel and send this for inclusion to mainline
Linux.

> I didn't test the ICU and eth patches separate, but I've tested the
> ethernet driver on a single VPE only (by setting smp affinity and
> nosmp). This version of the ethernet driver was used for root over NFS
> on the debug setup for like two weeks (without problems).
> 
> Tell me if we should pursue the way for the second DMA channel to PPE so
> both VPEs can send frames at the same time.

I think it should be ok to use both DMA channels for the CPU traffic.

> 3) WAVE300
> 
> In the two past weeks I've tried to fix a mash together various versions
> of wave300 wifi driver (there are partial version in GPL sources from
> router vendors). And I've managed to put the driver into "not
> immediately crashing" mode. If you are interested in the development,
> there is a thread in openwrt forum. The source repo here:
> 
> https://repo.or.cz/wave300.git
> https://repo.or.cz/wave300_rflib.git
> 
> (the second one must be copied into the first one)
> 
> The driver will often crash when meeting an unknown packet, request for
> encryption (no encryption support), unusual combination of configuration
> or just by module unloading. The code is _really_ ugly and it will
> server only as hardware specification for better GPL driver development.
> If you want to help or you have some tips you can join the forum (there
> are links for firmwares and intensive research of available source codes
> from vendors).
> 
> Links:
> https://forum.openwrt.org/t/support-for-wave-300-wi-fi-chip/24690/129
> https://forum.openwrt.org/t/how-can-we-make-the-lantiq-xrx200-devices-faster/9724/70
> https://forum.openwrt.org/t/xrx200-irq-balancing-between-vpes/29732/25
> 
> Petr
Hauke

_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel