Skip to content

fix(kernel): make the ADSP remote heap no-map - #915

Open
TheWeaklessOne wants to merge 1 commit into
armada-os:mainfrom
TheWeaklessOne:adsp-remote-heap-no-map
Open

TheWeaklessOne wants to merge 1 commit into
armada-os:mainfrom
TheWeaklessOne:adsp-remote-heap-no-map

Conversation

@TheWeaklessOne

Copy link
Copy Markdown

What's wrong

Patch 0623 adds adsp-rpc-remote-heap to qcs8550.dtsi, sm8550.dtsi and sm8650.dtsi as a reusable
shared-dma-pool. On 7.2.6 that is safe. It stops being safe with the next kernel.

Mainline 9e47ad9c562e ("misc: fastrpc: Allocate entire reserved memory for Audio PD in probe") is in 7.4-rc1 and
was backported to 7.3.4. With it, fastrpc assigns the whole region to the DSP at probe and never allocates it from
the pool. A reusable pool is CMA, so Linux keeps lending those 12 MiB to the page cache and to anonymous memory.
When memory gets low, the kernel writes to a page the DSP owns, and the board takes a synchronous external abort
or resets with nothing logged.

Zaviya's 7.4 kernel has the same node and hit exactly this: titooo7's Odin 2 Pro aborted while copying files
(hashtagbasit/Zaviya#102), and my Odin 2 Portal reset during a page-cache fill. no-map fixed both. Upstream made
the same change for the mainline remote heaps of sm8250, sm8750, kodiak, glymur, talos and agatti; those were
applied to the Qualcomm tree on 8 November and are not in a release yet.

What this changes

In patch 0623, the adsp-rpc-remote-heap node in all three hunks gets no-map instead of reusable, and the
comment above it says so. The paragraph in the patch description that explained the reusable pool now says why it
is no-map. Each changed line replaces one added line, so the hunk headers stay as they were.

How it was tested

The patch, against your tree: v7.2.6 qcs8550.dtsi, sm8550.dtsi and sm8650.dtsi, with the earlier series
patch that touches them (0070) applied first, then 0623 the way build-kernel.sh applies it
(patch -p1 --batch --forward -F0, dry run with --verbose): 5 of 5 hunks succeed, the old and the new 0623
alike, and git apply --check passes. The five later series patches that touch those files still apply on top.
The only difference in the result is the six changed lines. Armada's kernel was not built from this branch.

The change itself, on my AYN Odin 2 Portal (qcs8550, 8 GB), with kernels built from Zaviya's SM8550 recipes:

  • 7.4.0-rc1, where the bug is. The same node as no-map, booted from RAM, then installed. dmesg says
    nomap non-reusable adsp-rpc-remote-heap, CmaTotal is 32768 kB (45056 before), msm/adsp/sensor_pd is up,
    the IMU sends 754 events in 2 s. With vm.min_free_kbytes at its default I read 12 GB of files at 80 MB/s,
    dropped the page cache and read 12 GB again unthrottled, on each boot. The default CMA pool filled from 515 to
    7939 of 8192 pages and CmaFree reached 0, with no reset. Without the change the same fill reset the board
    when that pool was at 6405 pages.
  • 7.2.8, close to your 7.2.6. The property changed in the built DTB, four RAM boots, against two boots of
    the same kernel without it. It behaves the same: the sensors PD comes up at about 5 s, the IMU reports
    (734-768 events in 2 s, 732-742 without the change), the sound card and its sinks are there, a 1 s tone plays
    without an error. One extra warning appears, "reserved memory is beyond device's set DMA address range":
    fastrpc attaches the region before it sets its 32-bit DMA mask, and the region is below 4 GiB.
  • 7.2.8, the heap itself. A small test makes the kernel take the heap the way a static PD would
    (INIT_CREATE_STATIC with a PD name the DSP refuses, so the buffer is allocated, assigned to the DSP, taken
    back and freed). With no-map, 4 MiB and 8 MiB are served from 0xfc000000, twice in a row at the same address,
    so the block is freed in between. A 12 MiB request gets ENOMEM before the DSP is asked: the pool hands out
    power-of-two blocks, 12 MiB rounds up to 16 and the pool has 12. With reusable, 12, 8 and 8 MiB are served.
    No assign errors and no ADSP restart in either; sensors and audio were unchanged afterwards.

What it means on 7.2.6

armada-ssc starts hexagonrpcd with -s (attach), and in the 7.2 driver an attach does not take the heap, so
the sensors do not notice. What does change on 7.2: the 12 MiB are no longer lent to Linux, and a single request
for the whole 12 MiB fails where it used to succeed. If you'd rather not carry that on 7.2, hold this until the
kernel moves to 7.3.4+ or 7.4. It has to be in by then.

To check a build on a device:

dmesg | grep -iE 'reserved mem.*adsp-rpc|adsp-rpc-remote-heap'
grep -E 'CmaTotal|CmaFree' /proc/meminfo

map reusable adsp-rpc-remote-heap means exposed once the kernel is 7.3.4+ or 7.4; nomap non-reusable means
fixed.

Scope: I tested qcs8550 (the Portal). sm8550.dtsi and sm8650.dtsi get the same one-line change, and I have no
board on them.

Related:

Found and measured with help from Claude and Codex (AI assistants). The device and the tests are mine.

0623 adds the ADSP FastRPC remote heap as a reusable shared-dma-pool.
That is safe with the 7.2 driver, which takes a buffer from the pool
before it assigns it to the DSP. From Linux 7.3.4 and 7.4 (mainline
9e47ad9c562e) fastrpc assigns the whole region to the DSP at probe and
never allocates it, while CMA keeps lending the same pages to Linux.
Under memory pressure the kernel then writes to a page the DSP owns:
a synchronous external abort or a reset with nothing logged, as seen on
the Odin 2 Pro and Odin 2 Portal with Zaviya's 7.4 kernel.

Make the node no-map in qcs8550, sm8550 and sm8650 now, so a rebase
onto 7.3.4+ or 7.4 does not bring the resets with it. The hunks keep
their line counts and the patch applies to 7.2.6 as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant