[OpenWrt-Devel] Squashfs breakage lottery with UBI WAS: [PATCH RFC 2/2] amp821xx: use newly added pad-squashfs for Meraki MR24

[Russell Senior]

trimming a bit for easier reading ...

> [Fixed Mathias Email]

> > > [...]
> > > OpenWrt's mksquashfs for the rootfs (which is usually
> > > squashfs) is instructed to skip the padding via the nopad
> > > option because the rootfs will be padded by functions like
> > > pad-rootfs to the eraseblocksize which is usually much
> > > bigger at somewhere 64KiB.
> > 
> > Note also, e.g. tplink,tl-wdr3600-v1:
> >
> > [    0.428860] m25p80 spi0.0: en25q64 (8192 Kbytes)
> > [    0.433638] 3 fixed-partitions partitions found on MTD device spi0.0
> > [    0.440112] Creating 3 MTD partitions on "spi0.0":
> > [    0.444991] 0x000000000000-0x000000020000 : "u-boot"
> > [    0.450914] 0x000000020000-0x0000007f0000 : "firmware"
> > [    0.459935] 2 tplink-fw partitions found on MTD device firmware
> > [    0.465951] Creating 2 MTD partitions on "firmware":
> > [    0.471047] 0x000000000000-0x0000001b6b1b : "kernel"
> > [    0.476924] 0x0000001b6b1c-0x0000007d0000 : "rootfs"
> > 
> > netgear,wndr3800:
> > 
> > [    0.671227] m25p80 spi0.0: mx25l12805d (16384 Kbytes)
> > [    0.676366] 4 fixed-partitions partitions found on MTD device spi0.0
> > [    0.682724] Creating 4 MTD partitions on "spi0.0":
> > [    0.687508] 0x000000000000-0x000000050000 : "u-boot"
> > [    0.693223] 0x000000050000-0x000000070000 : "u-boot-env"
> > [    0.699163] 0x000000070000-0x000000ff0000 : "firmware"
> > [    0.707493] 2 netgear-fw partitions found on MTD device firmware
> > [    0.713550] Creating 2 MTD partitions on "firmware":
> > [    0.718507] 0x000000000000-0x0000001bd440 : "kernel"
> > [    0.724195] 0x0000001bd440-0x000000f80000 : "rootfs"
> > 
> > and netgear wgt634u:
> > 
> > [    1.245465] 3 bcm47xxpart partitions found on MTD device
> > physmap-flash.0
> > [    1.252454] Creating 3 MTD partitions on "physmap-flash.0":
> > [    1.258364] 0x000000000000-0x0000000a0000 : "boot"
> > [    1.286600] 0x0000000a0000-0x0000007e0000 : "firmware"
> > [    1.298172] 3 trx partitions found on MTD device firmware
> > [    1.303639] Creating 3 MTD partitions on "firmware":
> > [    1.308944] 0x00000000001c-0x000000000948 : "loader"
> > [    1.331486] 0x000000000948-0x000000139800 : "linux"
> > [    1.348577] 0x000000139800-0x000000740000 : "rootfs"
> > 
> > as examples where the rootfs starts at unaligned addresses. If you
> > padded the rootfs individually, the combination of kernel+rootfs would
> > unnecessarily waste space.
> Aren't these all devices with spi-nor? They all just place the kernel +
> rootfs (squashfs) in a "firmware" mtd partitions and the mtdsplit is doing
> its magic. [...]

Yes, agreed. These are examples of why we shouldn't just remove the
-nopad, and actually mostly just my own sanity check that I remembered
this on some devices (other spi-nor devices, such as the buffalo
wzr600dhp seem to align the rootfs). Oh, and (irrelevant detail) the
wgt634u is NOR, but not spi.

> I think this is a bit out of the scope. This patch should not
> interfere with them and the unaligned squashfs rootfs starts is not a
> problem from what I can tell. That said removing the "-nopad" switch or
> adding the padding with "pad-squashfs" you made should make no difference
> in regards to the missing padding between the linux/kernel and rootfs
> since this isn't the problem.

I was under the impression that removing the -nopad switch would pad out
the root.squashfs out to a 4k boundary before concatenation, leading to
a further padding of the concatenation since the padding will (in
general) hang over a 4k boundary.

> > > But this is a problem for squashfs as rootfs in ubi
> > > partitions. Currently no explicit padding is being
> > > set and it currently works for the most time because
> > > ubi volumes are always aligned to LEBs which could
> > > be close enough for 4KiB paddings...
> > > 
> > > Digging down deeper revealed that squashfs excepts blocks
> > 
> > trivial spelling fix, that should be "accepts", I think...
> Not sure if it's trivial or not, but yes the "excepts" is wrong,
> I think it should have been "expects". But... I've still hope
> that "-nopad will be the way" so I'm prepared to just drop this
> patch again :D.
> 
> > > to be up to PAGE_SIZE. This is explained in this bug report
> > > that states that the 4k padding for ARCHs with 64KiB pages
> > > resulted in "attempt access beyond end of device" errors:
> > > <https://sourceforge.net/p/squashfs/mailman/message/28307755/>
> > 
> > AFAICT, the PAGE_SIZE on Meraki MR24 is 4k. In the kernel's
> > include/asm-generic/page.h, we have:
> The APM821xx SoC supports 4KiB, 16KiB and 64KiB page sizes.
> Ie: <https://cateee.net/lkddb/web-lkddb/PPC_64K_PAGES.html>
> OpenWrt currently defaults to 4KiB, but the 16KiB and 64KiB
> do provide a throughput boost and they are easily enabled
> by just editing config-default a bit.

The MR24's NAND is only 32MB (okay, that's not tiny), but I'm all for
optimizing for size.

> > When Jonas and I were discussing this, we noted that sysupgrade changes
> > won't be installed the first time you do this, so a lack of padding to
> > the root.squashfs can still result in boot looping.
> >
> > Since Meraki MR24 (afaict) doesn't use the ubinize-image.sh script, it
> > won't be protected.
> Are you using an alternative flashing approach?
> 
> The wiki: <https://openwrt.org/toh/meraki/mr24> notes that for the initial
> install, the MR24 should boot off an tftp-loaded initramfs and then the 
> user has to use sysupgrade to install the real image.
> Yes, existing initramfs + installs will have to be updated before this will
> work. But then, this bug has sadly been present from the beginning and on
> many other routers as well and nobody fixed it yet. It's definitely a bad
> bug though.

Reguar sysupgrade.bin is a tarball. I normally use sysupgrade. The
initial install from stock meraki firmware is tftp booting an initramfs,
and that shouldn't be a problem. The problem I'm referring to is going
from an existing openwrt install to a new "fixed" one without tftp
booting.  If we are depending on the already-on-device sysupgrade
nand.sh process, it hasn't been fixed yet. Padding the root.squashfs as
delivered in the tarball will avoid the bug no matter the state of
nand.sh.


-- 
Russell Senior, President
russell at personaltelco.net

_______________________________________________
openwrt-devel mailing list
openwrt-devel at lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel