[linux-nvidia-6.18-next]: Backport "mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios"

[jamieNguyenNVIDIA]

Target: linux-nvidia-6.18-next · Topic: jamien/6.18-next/gup-fast-null-mapping · Base: da0bc2776f136 (6.18.40)

Clean cherry-pick from linux-next:

(cherry picked from commit c494788faffe67216c56623d240541fde50139c3 linux-next)

John Hubbard, Acked-by David Hildenbrand, applied by Andrew Morton.
Lore thread.

Same pick already applied to 26.04_linux-nvidia and 26.04_linux-nvidia-bos
(#531, #532). gup_fast_folio_allowed() is byte-identical on all three branches, so
the change and the diff are identical.

Problem

f002882ca369 (in mainline since v6.10, so present on this branch) made
gup_fast_folio_allowed() bail to the slow path for any order-0 folio with a NULL
->mapping when CONFIG_SECRETMEM=y:

	if (!mapping)
		return false;

Pages from alloc_page() + vm_insert_page() legitimately have a NULL mapping, so
every pin_user_pages_fast() over such a range misses the fast path — nvidia-fs
(GPUDirect Storage) allocates its shadow buffers exactly this way
(nvfs-mmap.c:750/761), then pins them with pin_user_pages_fast(..., FOLL_WRITE, ...)
and no FOLL_LONGTERM, which is precisely the case the patch restores.

The NULL check was meant to catch truncated file-backed pages, not secretmem.
Secretmem folios are published via filemap_add_folio(), which always sets
->mapping, so a NULL mapping proves the folio is not secretmem. Returning
!reject_file_backed keeps long-term writable pins on the slow path and restores the
fast path otherwise — exactly the pre-f002882ca369 behaviour.

arch/arm64/configs/full-64k.config sets CONFIG_SECRETMEM=y, so this is live on the
shipping 64K configuration.

Measurement (GH200, 288 cores, 64K pages)

Two kernels built from this branch tip (6.18.40), differing only by this patch,
configured with
scripts/kconfig/merge_config.sh arch/arm64/configs/full-64k.config arch/arm64/configs/nvidia.config.
A module reproduces the nvidia-fs pattern (alloc_page + vm_insert_page, then
pin_user_pages_fast(..., FOLL_WRITE, ...)), with an anonymous-memory control the
patch cannot affect. get_user_pages_fast_only() gives the GUP-fast verdict directly.

Mechanism, at every thread count:

6.18.40-g618b (unpatched):  fast_only = 0 / N      GUP-fast rejects
6.18.40-g618f (patched):    fast_only = N / N      GUP-fast accepts

Median ns/page, 512 pages/thread, 5 reps:

threads	unpatched	patched	speedup	anon control (unp / pat)
1	37	30	1.2x	32 / 30
8	2076	31	67x	34 / 30
32	7082	32	221x	32 / 31
128	19710	35	563x	87 / 34

The anon control held at ~30-34 ns/page on both kernels, so only the affected range
moved. Single-threaded it is a wash; the win appears under the concurrency a real IO
submitter uses. On the equivalent 7.0 pair, perf showed the unpatched slow path is
~90% lock contention (queued_spin_lock_slowpath), which disappears once patched — so
this is contention that scales with thread count, not a fixed per-page cost.

Risk

Low. One line in a static function with three callers, all in GUP-fast. Only
reject_file_backed == false && check_secretmem && mapping == NULL changes behaviour;
long-term writable pins are untouched.

A secretmem folio caught mid-truncate cannot happen here: secretmem_setattr() refuses
to shrink, there is no .fallocate (so no punch-hole), and secretmem_migrate_folio()
returns -EBUSY. Only inode eviction remains, which requires every VMA gone — no VMA,
no PTE for GUP-fast to walk. Raised by David Hildenbrand on v1 and resolved before he
Acked.

Testing

Applies cleanly to the branch tip; gup_fast_folio_allowed() is byte-identical to the
tree the patch was written against. Built for arm64 64K using the in-tree
full-64k.config + nvidia.config fragments, no new warnings. Verified in the binary,
not just the source: unpatched emits mov w0, #0x0 on the NULL-mapping path, patched
emits eor w0, w0, #0x1. Both kernels were built from this branch tip, booted on
GH200, and benchmarked as above.

Bugs

nvbug: 6064778
LP: https://bugs.launchpad.net/ubuntu/+source/linux-nvidia-7.0/+bug/2162917

[nvmochs]

Verified backport is the same as the 7.0 kernels, no issues.

Acked-by: Matthew R. Ochs <mochs@nvidia.com>

[sforshee]

Acked-by: Seth Forshee <sforshee@nvidia.com>

[sforshee]

Applied to linux-nvidia-6.18-next, closing PR.

7f6fd1bec515 mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[nirmoy]

BaseOS Kernel Review

Summary

The GUP-fast change is safe, but its rationale is inaccurate: a truncated secretmem folio can have a NULL mapping. Safety instead depends on truncation unmapping PTEs before folio removal and GUP validating the sampled PTE.

Findings: Critical: 0, High: 0, Medium: 0, Low: 1

Latest watcher review: open review

Generated test plan: open test plan

Kernel deb build: successful (download debs, 4 files)

Head: d9502ecbeaaf

This comment is maintained by nv-pr-bot. It is updated when the GitHub watcher publishes a newer review.