W5500 RX path corrupts a heap neighbour: LoadProhibited in w5500_tsk when an extra internal-DRAM allocation lands next to the emac

[iliabaranov]

Symptom

HIL DUT 01d7f344 (Ethernet primary, W5500 over SPI, IDF 5.5.4): LoadProhibited in w5500_tsk ~150 s after boot, reproducible every boot, on any image where one extra ~4.3 KB internal-DRAM allocation is made at tether start (a FreeRTOS TCB + 4 KB stack). Same code with that allocation moved to PSRAM: no crash in 360 s+. main alone: no crash. Bisected in #135 (three builds); details in that PR body.

Coredump (decoded against the crashing ELF, v1.2-21-gb467f85)

exccause 0x1c LoadProhibited, excvaddr 0x0
#1 uninstall_priv_desc (spi_master.c:1176)  memcpy(orig_rx_buffer, trans_buf->buffer_to_rcv, ...)
#5 w5500_spi_read (value=0x3fcccbbc, len=1494)
#6 w5500_read (emac=0x5d6 ...)                      <- emac pointer garbage
#7 w5500_read_buffer (buffer=0x3fcccbbc, len=1494)  <- emac->rx_buffer is a w5500_tsk STACK address
#8 emac_w5500_receive (mac=0x3fccb388, buf=0x3fcf74d0)
#9 emac_w5500_task

emac->rx_buffer should be the heap_caps_malloc(ETH_MAX_PACKET_SIZE, MALLOC_CAP_DMA) block from emac_w5500_new; here it reads as a stack address, and trans_buf->buffer_to_rcv is NULL. The emac_w5500_t struct is being overwritten by a neighbour in internal DRAM. The FreeRTOS stack canary never fired on any task, so it is not a task-stack overflow; the write comes from whoever owns the block adjacent to emac, and which block that is depends on allocation order (hence "only with one more allocation").

Raw coredump: ops/coredumps/2026-09-17-dut-w5500-rx-v1.2-21-gb467f85.bin (ESP coredump format, 24-byte header). Decode with esp_coredump info_corefile -t raw against the ELF in ops/elf-archive/ (build is not bit-reproducible, so strip the header and load in gdb, or rebuild b467f85).

Not yet known

• Which allocation writes past its end. Candidates by proximity at tether start: esp_netif_new internals, TinyUSB task/queues (tinyusb_driver_install), the W5500 alloc_recv_buf (1522 B per frame, freed per frame — heap churn right at the crash site).
• Whether heap_caps_check_integrity_all() at 1 Hz on a bench build catches the first corrupt block (cheap next step).

Why it matters

#135 sidesteps it (PSRAM stack), but the corruption is main's and will surface again with the next unrelated internal-DRAM allocation. Also motivated #145 (the crash loop never tripped the rollback ladder).

[iliabaranov]

Status 2026-09-21: still open, not yet reproduced with instrumentation.

• Not seen again on the DUT since 2026-09-17 across ~30 h of soak on v1.3-25 → v1.3.1 images (which all carry the extra internal-DRAM allocations that triggered it, e.g. the 64-slot TX ring metadata). Note that v1.3.1 runs the W5500 driver in polling mode (fix(eth): W5500 RX via SDK polling mode; INT-line stall gauge (#128) #162); that changes how w5500_tsk is woken, not its 4 KB stack or the emac allocation next to it, so the hypothesis (driver-task stack overflow into the adjacent heap block; rx_buffer read back as a w5500_tsk stack address) stands untested.
• Reproducer image is built and archived: the crashing commit b467f85 + CONFIG_HEAP_POISONING_COMPREHENSIVE + CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK (ELF 1f321c470). Crash expected within ~150 s of boot; a stack-overflow watchpoint would name the task directly, heap poisoning would name the corrupted block.
• Next step: flash it on the DUT after the v1.3.1 qualification soak, pull /api/coredump, decode against that ELF. If confirmed, the fix is the driver task stack size (rx_task_stack_size) or moving the emac block off the internal-DRAM neighbourhood — a one-line Kconfig/config change either way.

[iliabaranov]

Reproduction attempt 2026-09-21 — did not reproduce; downgrading

Two debug images of the crashing commit b467f85, both with CONFIG_HEAP_POISONING_COMPREHENSIVE + CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK (+ the default stack canary), run on the same DUT (01d7f344, Ethernet primary):

image	change	result
plain b467f85 + poisoning/watchpoint (ELF 1f321c470)	none	13 min, no crash, no reboot; tether never started (no Ethernet blip occurred, so the trigger allocation never happened)
same + tether started at boot on the Ethernet unit (ELF a257c5d8d) — reproduces the #135 bisect condition deterministically (ml_usb_tx_init() → drain task with an internal 4 KB stack next to the W5500 emac)	internal heap_min dropped from ~15 KB to 46 KB used (the extra ~30 KB of tether allocations landed)	13 min + a restart, no crash, no watchpoint, no poisoning fault. rst_hist only my OTAs.

So with the allocation in place and the heap fully poisoned, nothing overran in 26 min, where the original failed every boot at ~150 s. Either the corruption needed the exact allocation layout of that 2026-09-17 build (poisoning itself shifts every block by the canary bytes, which can mask a neighbour overrun), or it needed the runtime path (supervisor-driven bring-up after an Ethernet blip, which no bench control can produce here) rather than a boot-time start.

Side observation: on b467f85 the unit stuck in RECONNECTING (ml_state 5) after the OTA and never re-registered, even after a restart — that is the pre-#156 coordination-recovery weakness, since fixed. Restored to v1.3.1 over the LAN.

Status: not reproducible on demand; 30+ h of v1.3.x soak on this unit (with the 64-slot ring and the same internal-DRAM neighbourhood) show no recurrence. Keeping open at low priority with the coredump and both reproducer ELFs archived (ops/elf-archive/); if it recurs on any unit the coredump will land in /api/coredump and crash_task in /state.json will say w5500_tsk. Next attempt, if ever: a non-poisoned build with a heap-integrity check on a timer, so the layout stays identical to the crashing one.