Rebase USB4 GDA DV branch and add smoke probes - #52
Draft
georgewhewell wants to merge 14 commits into
Draft
Conversation
Contributor
Author
|
P2P/GDA smoke update from live Strix testing:
Caveats: this is intentionally gated behind |
Contributor
Author
|
HIP receiver smoke test update from
Conclusion: destination GPU dmabuf import/registration works, but this is not true RX-side GPU DMA yet. The current RDMA WRITE receive path still lands in kernel RX buffers and then calls the CPU-copy path ( |
Introduce the software-RNIC ABI surface that the upcoming GDA (GPU
Direct Async) work will hang off. v1 of this PR series only wires the
QUERY_CAPS method so userspace consumers can detect ABI presence and
size assumptions; CREATE_QUEUE / DESTROY_QUEUE / KICK and the
GPU-produced WQE consumer come in follow-ups, each gated by a bit in
struct usb4_rdma_dv_query_caps_resp::caps.
What lands here:
- userspace/usb4_rdma/usb4_rdma_dv.h: the full v2 ABI header. Declares
all opcodes, the 128-byte producer/consumer split doorbell record,
WQE/CQE structs, the generation protocol, and the queue-memory
coherence requirements. Reviewing this header is the single most
important part of this PR — it pins the contract every subsequent
GDA commit will build against.
- kernel/dv.c: the QUERY_CAPS handler plus the uverbs object/method
declarations and the uapi_definition table. Caps is intentionally 0
for now; later commits OR in USB4_RDMA_DV_CAP_* bits as the
matching kernel paths are wired.
- kernel/ibdev.c: assigns tbv_uapi_defs to ib_device.driver_def under
CONFIG_INFINIBAND_USER_ACCESS so the private uverbs methods are
exposed.
- userspace/bench/tbv_dv_caps_probe.c: standalone QUERY_CAPS probe.
Uses the raw RDMA_VERBS_IOCTL ABI directly rather than rdma-core's
private execute_ioctl() helper, so it can be built outside the
provider tree. Also reports the exact byte layout of the doorbell
record and WQE generation field so the contract is self-documenting
at runtime.
- nix/dv-probes.nix + flake.nix: package the probe as
tbv-dv-caps-probe (also wired into apps/checks/hydraJobs).
- nix/module.nix: extend the kernel source filter to include the DV
ABI header so the out-of-tree module build can pick it up.
Build-time guarantees in kernel/dv.c BUILD_BUG_ON the on-the-wire
struct sizes (WQE/CQE/doorbell lines/record) against the corresponding
USB4_RDMA_DV_*_SIZE constants, so any future field addition without an
ABI version bump fails to compile rather than silently desyncing kernel
and userspace.
Verified:
- nix flake check --no-build passes
- nix build .#thunderbolt-ibverbs --builders '' builds the module
- nix build .#tbv-dv-caps-probe --builders '' builds the probe
- tbv-dv-caps-probe -l prints the doorbell layout matching the header:
producer line (gpu→kernel): sq_tail@0/4, cq_head@4/4, generation@8/4
consumer line (kernel→gpu): sq_head@64/4, cq_tail@68/4, qp_state@72/4, generation@76/4
wqe generation field at offset 44
No DV transport opcodes are wired in this commit; this is the ABI
introduction only. Runtime QUERY_CAPS returns caps=0 to reflect that.
Wire the QP-lifecycle half of the DV ABI: userspace can attach a host-pinned SQ/CQ/doorbell triple to a QP and detach it. No transport opcodes are wired through the DV consumer yet, so caps stays 0; KICK and the actual WQE consumer come in a follow-up commit. Per-QP DV state is held in a new struct tbv_dv_qp_state embedded in tbv_qp. ibdev.c owns the embedding and exposes two thin accessors (tbv_qp_from_ibqp, tbv_qp_dv_state); dv.c works with the substate without needing the full tbv_qp layout. dv.c provides the lifecycle helpers (init, teardown, active-check) and the uverbs handlers. GDA-exclusive QP semantics are enforced at post_send: once a DV queue is active on a QP, standard ibv_post_send returns -EBUSY. post_recv is intentionally unaffected — v1 keeps the receive queue kernel-owned and the peer continues to post recvs through standard verbs while the QP has an active DV queue (matches the design in the ABI header). CREATE_QUEUE: - Validates abi_version, alignment, entry counts, stride against the ABI constants, and overflow-checks the address ranges. - ib_umem_get pins SQ/CQ/doorbell from userspace. - Bumps the QP's DV generation, stores the umem references, and publishes the LIVE state into the doorbell consumer cacheline with packed (index=0, generation) tail words. - Returns qp_num + generation. Rolls back on any later failure by bumping generation again and releasing the umem. - Second CREATE_QUEUE on a still-attached QP is rejected with -EBUSY. DESTROY_QUEUE: - Bumps generation under dv->mutex (so any stale producer write landing on the doorbell after this point is generation-mismatched and detectable). - Releases SQ/CQ umem, keeps doorbell umem long enough to write the QP_DEAD sentinel + bumped generation, then releases that too. - Second DESTROY_QUEUE returns -ENOENT. QP destroy (tbv_destroy_qp) calls tbv_dv_qp_state_teardown early so the umem pins are released even if userspace never called DESTROY_QUEUE explicitly. This is the safety net for crashed clients. The kernel doesn't export a public `ib_umem_copy_to` (only `_from`), so dv.c carries a small `tbv_dv_umem_write` helper that walks the umem's scatterlist and kmap_local_page()s each affected page. Today the only writer is the doorbell consumer-line update (16 bytes); the implementation is general so subsequent commits can reuse it for CQE writes without revisiting the umem API. tbv_dv_caps_probe gains a -q mode that: - Allocates aligned SQ/CQ/doorbell host memory. - Creates a standard verbs PD/CQ/QP. - Issues CREATE_QUEUE, verifies qp_num matches and generation is nonzero. - Reads back the doorbell consumer line and verifies it matches the documented LIVE state (sq_head/cq_tail packed with generation, qp_state=LIVE). - Confirms a second CREATE_QUEUE returns -EBUSY. - DESTROY_QUEUE, verifies the consumer line transitions to DEAD with bumped generation. - Confirms a second DESTROY_QUEUE returns -ENOENT. Verified: - nix flake check --no-build passes - nix build .#thunderbolt-ibverbs --builders '' builds the module - nix build .#tbv-dv-caps-probe --builders '' builds the probe - tbv-dv-caps-probe -h / -l still work (no device required) Runtime probe (-q) requires a usb4_rdma device + matching module loaded; that lands as part of the hardware smoke after this branch hits Hydra.
georgewhewell
force-pushed
the
codex/gda-v2-rebased-port
branch
from
July 3, 2026 11:12
3cd65ab to
164961c
Compare
The device was reporting IB_LINK_LAYER_ETHERNET with RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP, which routed every QP through the RoCE code path. The RoCE path requires get_eth_mac/update_qp_av/set_roce_udp_encap callbacks this driver does not implement, so every ibv_modify_qp(RTR/RTS) returned -ENODATA: no AV resolution could be constructed for an Ethernet device without Ethernet AV resolvers. USB4 native RDMA is IB semantics over a point-to-point link, not RoCE. Switch to IB_LINK_LAYER_INFINIBAND + RDMA_CORE_CAP_IB_MAD. The pre-existing tbv_query_port() already reports lid=0, gid_tbl_len=TBV_IBDEV_GID_TBL_LEN, pkey_tbl_len=1, and the GID table is already IB-mode (fe80 + node-GUID interface_id), so no other port-data changes are required. Validated: ib_write_bw 1309 MiB/s, ib_send_bw 1322 MiB/s, ib_read_bw 1305 MiB/s, ib_write_bw --bidirectional 1925 MiB/s, ib_write_lat 6.15us typical, uc_oneway 10853 Mbit/s with 1000/1000 done, rc_write_verify recv_check=0. 41001 RDMA WRs across both nodes, zero errors on every error counter.
When the kernel RX path lands a WRITE fragment into a dmabuf MR, the
existing bounce-buffer path (kmap_local_page + memcpy in
tbv_umem_copy_to_iova) fails on ZONE_DEVICE pages because the CPU cannot
kmap them. The fragment never lands in GPU memory, and the WC is pushed
with LOC_PROT_ERR.
The fix is a parallel RX path: when a fragment's destination MR is a
dmabuf MR (mr->umem->is_dmabuf == true), DMA the ring payload directly
into the MR page via dma_map_sg_attrs + per-fragment descriptors. The
bounce-buffer path is unchanged for CPU MRs.
Concrete changes:
- tbv_rx_reorder_msg gains zcopy_sgt / zcopy_mr / zcopy_dma_dev /
zcopy_sgt_nents / zcopy_mapped / refs (refcount_t). Per-fragment
zcopy DMAs bump refs so the msg stays alive until every completion
is processed; the SGL is dma_unmap_sg'd on last put.
- tbv_rx_buffer_write_fragment_locked() probes the destination MR with
tbv_rx_zcopy_try_lock(). If it is a dmabuf MR, the new path is taken:
tbv_rx_zcopy_prepare_msg() dma_map_sg's the destination SGL,
tbv_rx_zcopy_post_fragment() walks the SGL to the page that contains
dest_iova, and tbv_path_post_rx_zcopy_frame() programs one
descriptor that DMAs the wire payload into the MR page. Completion
runs on the QP workqueue via tbv_rx_zcopy_drain_work, marks the
fragment seen, and pushes the WC when the last fragment finishes.
- The per-frame struct tbv_data_frame gains zcopy_rx / zcopy_page /
zcopy_page_off so tbv_path_post_rx_zcopy_frame() can submit a
one-shot frame that uses no bounce buffer and is freed in
tbv_path_zcopy_rx_complete() after dma_unmap_page.
- Two new counters in tbv_state:
data_rx_dmabuf_zcopy (successes)
data_rx_dmabuf_zcopy_error (mapping / DMA failures)
Visible in /sys/kernel/debug/tbv/usb4_rdma*/summary.
- The UC oneway bench gains a trailing newline (cosmetic, no behavior
change).
Failure handling: dma_map_sg returning 0, dma_map_page errors, and
SGL walk failures all fall through to the existing drop+ack-error path,
the same as for bad rkey. No new error categories.
CPU-MR behavior is unchanged: tbv_rx_zcopy_try_lock() returns false
when mr->umem->is_dmabuf is unset, and the bounce-buffer path runs.
The implementation matches docs/RX_SIDE_DMA.md: per-fragment DMA,
no reorder/coalesce, no per-MR DMA context, NHI ring untouched.
The dmabuf_mr_probe.c file was never committed to the GDA branch and not wired into the bench-tools flake output. The RX-side DMA path needs this client to verify the kernel lands fragments directly into a dmabuf MR (data_rx_dmabuf_zcopy counter) instead of failing the bounce-buffer path on ZONE_DEVICE pages.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
origin/mainKICKpath so CREATE_QUEUE/doorbell/CQE behavior can be smoked without advertising transport SEND/WRITE capsValidation
nix build .#thunderbolt-ibverbs -Lnix build .#tbv-dv-caps-probe -L./result-dv-probes/bin/tbv-dv-caps-probe -ltools/tbv-target-module.sh strix-1 --booted-kernel --copyLive smoke note
Tried reloading this branch on
strix-1with the current module parameters preserved. The module loaded, but native HELLO failed with-EINVALbefore anyusb4_rdma*devices were published, so the live DVtbv-dv-caps-probe -qsmoke could not run. I restoredstrix-1to the booted-system module afterward and verified theusb4_rdma*devices came back.This PR is draft until that native HELLO reload regression is understood.