Skip to content

Add export support for shader constant buffers and VS input meshes #224

Closed

Description

@Misaka-Mikoto-Tech

Problem / Use case

rdc-cli currently has useful capture inspection and mesh export commands, but two common debugging workflows still require custom RenderDoc Python scripts:

  1. Exporting shader constant buffer contents for a specific event and stage.
    For example, dumping VS cbuffer1 at a given EID, including reflected variables and raw bytes.

  2. Exporting the VS input mesh / input assembler geometry.
    The current rdc mesh command exports post-transform mesh data (vs-out). That is useful, but when debugging original model geometry, vertex buffer layout, skinning inputs, or index buffer ranges, users often need the mesh before the vertex shader runs.

In practice, exporting vs-out can look like a projected or flattened surface, while the expected debugging target is the original VS input mesh.

Proposed solution

Add first-class CLI support for both workflows.

Constant buffer export

Add a command such as:

rdc cbuffer 1084 --stage vs --binding 1 --json
rdc cbuffer 1084 --stage vs --binding 1 --raw -o cbuffer1.bin

### Alternatives considered

This can currently be done with custom rdc script Python code that calls RenderDoc APIs directly, reads the constant buffers, decodes vertex/index buffers, resolves indirect draw arguments, and writes an OBJ manually.

That works, but it is fragile and not convenient for repeated capture analysis.

Activity

  1. BANANASJIM commented on May 15, 2026

    @BANANASJIM
    Owner

    Both workflows are implemented as two independent PRs:

    Both have full unit + Vulkan integration test coverage and pass CI (ruff/mypy strict, ~92.7% coverage).

    @Misaka-Mikoto-Tech — the dev environment is Linux, where RenderDoc replay is Vulkan-only, so I could not verify the D3D12 path (constant-buffer register spaces / root constants, D3D12 input-assembler layout) against a real capture. Could you test both against your D3D12 captures and report back?

    Useful checks:

    • rdc cbuffer <EID> --stage vs --binding 1 --json — do the reflected variable names/values match what you expect for that cbuffer? --raw -o cb.bin — is the byte size/content correct?
    • rdc mesh <EID> --stage vs-in -o vsin.obj — is this the original (pre-VS) model geometry rather than the projected vs-out surface?

    If cbuffer --raw errors on a buffer-backed cbuffer, please paste the error and, if possible, the output of rdc cbuffer <EID> --stage vs --binding 1 --json so I can check the reflection lookup against D3D12 register-space semantics. Thanks!

  2. BANANASJIM commented on May 15, 2026

    @BANANASJIM
    Owner

    Update — a deeper self-review (cross-checked against RenderDoc's official decode_mesh.py reference) found real correctness bugs that the Vulkan-mocked tests could not catch. Both PRs got hardening follow-up commits:

    #227 (vs-in mesh) — fixed:

    • baseVertex is now applied to indices (index + mesh.baseVertex, per RenderDoc reference) — without this, indexed D3D12 draws with a non-zero BaseVertexLocation produced silently scrambled triangles.
    • Position is now read at vertexByteOffset within the vertex stride (not offset 0) — fixes wrong positions when POSITION isn't the first IA element.
    • Non-triangle/patch topology now emits a clear stderr warning instead of a silently empty OBJ.
    • Known limitation (now documented): vs-in exports the position attribute + triangle connectivity only, not full per-attribute IA.

    #228 (cbuffer) — fixed:

    • Null/unbound cbuffer descriptor → clean -32001 "cbuffer not bound at this draw" (was: silent 0-byte file reported as success).
    • Descriptor byteSize == 0 (common for D3D12 root CBVs) → falls back to the reflected block size; never passes size 0 to GetBufferData (which would dump the entire upload heap as if it were the cbuffer). Clean error if size is truly unknown.
    • --set/--binding help now states the D3D12 meaning (register space / shader register bN).

    @Misaka-Mikoto-Tech — the highest-value checks on real D3D12 captures (a non-empty result is not sufficient — the prior failure modes were plausible-looking-but-wrong output):

    • vs-in: pick an indexed draw whose VS input is multi-attribute (e.g. POSITION+NORMAL+TEXCOORD, stride ≥ 32) and ideally has a non-zero BaseVertexLocation. rdc mesh <eid> --stage vs-in -o vsin.obj, open it next to RenderDoc's Mesh Viewer Input tab — do XYZ positions and triangle connectivity actually match?
    • cbuffer: pick a cbuffer at a known register(bN, spaceM) (ideally also a root CBV and a root-constants block). For each: rdc cbuffer <eid> --stage ps --set <M> --binding <N> --raw -o cb.bin and the decoded form (no --raw). Report: (a) does --set/--binding resolve the intended cbuffer? (b) cb.bin size vs the decoded variable footprint — equal, or 256-aligned/padded? (c) root CBV: is cb.bin the cbuffer or megabytes of heap? (d) root constants: clean -32602 not buffer-backed?

    (b)/(c) and the vs-in position/connectivity check are the load-bearing answers I can't determine without real D3D12 hardware.

  3. BANANASJIM commented on May 16, 2026

    @BANANASJIM
    Owner

    Both features are merged to master: rdc mesh --stage vs-in (squash 072cc49, #227) and rdc cbuffer (squash f23143a, #228). A deeper self-review added hardening for several silent-wrong-output cases (baseVertex/vertexByteOffset for vs-in; null-resource and byteSize==0 guards for cbuffer). @Misaka-Mikoto-Tech the D3D12-specific paths couldn't be verified on the Linux dev box — the verification asks above still stand (vs-in: compare positions+connectivity against RenderDoc's Mesh Viewer on a multi-attribute, non-zero-baseVertex draw; cbuffer: raw size vs decoded footprint, and root-CBV must not dump the whole heap). Please report back when you can; if anything's off we'll fix-forward. Leaving this issue open until you've had a chance to verify.

  4. BANANASJIM commented on Jun 23, 2026

    @BANANASJIM
    Owner

    Both requested workflows are implemented and merged:

    Both shipped against this issue and were archived in #229. Closing as delivered.

  5. BANANASJIM commented on Jun 23, 2026

    @BANANASJIM
    Owner

    Reopening — the close was premature on my part. The functionality is delivered (#227/#228), but leaving the close decision to the maintainer.

  6. Misaka-Mikoto-Tech commented on Jul 5, 2026

    @Misaka-Mikoto-Tech
    ContributorAuthor

    Thanks for implementing both workflows. I tested the current rdc 0.6.0 on Windows with real D3D12 and D3D11 captures. Short version: both features are partially implemented, but there are still correctness problems on Windows captures.

    Environment

    • Windows
    • RenderDoc replay module: 1.41
    • rdc --version: 0.6.0
    • GPU: NVIDIA GeForce RTX 3090
    • Tested APIs: D3D12 and D3D11

    1. rdc cbuffer

    D3D12: decoded JSON resolves the intended buffer, but uint variables are decoded as floats

    Test capture: D3D12

    EID: 7459

    Command:

    rdc cbuffer 7459 --stage ps --binding 1 --json

    The command resolves the intended ps cbuffer1, but the reflected variable has type: "4" and is still emitted from f32v.

    CLI output for first variable:

    {
      "name": "unknown.unknown[0]",
      "type": "4",
      "value": [1.0, 1.0, 1024.0, 1024.0]
    }

    Direct RenderDoc API inspection of the same variable shows it should be read from value.u32v:

    {
      "name": "unknown.unknown[0]",
      "type": 4,
      "value": [1065353216, 1065353216, 1149239296, 1149239296],
      "chosen_vector": "u32v"
    }

    So the decoded output is bit-identical but type-wrong. The implementation appears to always prefer f32v; it should choose u32v, s32v, f32v, etc. based on the reflected variable type.

    D3D12: --raw -o fails

    Command:

    rdc cbuffer 7459 --stage ps --binding 1 --raw -o cb.bin

    Result:

    error: not found: /draws/7459/cbuffer/0/1/data
    

    Same failure for --stage vs.

    I also noticed the raw VFS path does not include shader stage:

    /draws/<eid>/cbuffer/<set>/<binding>/data
    

    This is ambiguous on D3D12 when, for example, vs b1 and ps b1 both exist and point to different resources.

    For this draw, direct RenderDoc API extraction shows:

    VS cbuffer1: resource=ResourceId::458, offset=1048576, descriptor_size=3145728, reflected block size=6300
    PS cbuffer1: resource=ResourceId::299862, offset=3542528, descriptor_size=651776, reflected block size=492
    

    So stage is necessary to identify the intended cbuffer.

    D3D11: decoded JSON works, but --raw -o still fails

    Test capture: D3D11

    EID: 1130

    Commands:

    rdc cbuffer 1130 --stage vs --binding 1 --json
    rdc cbuffer 1130 --stage ps --binding 1 --json

    These decoded outputs look correct and match direct RenderDoc API typed values.

    Example:

    VS cbuffer1 first variable:
    cb1_v0 = [6.1338291168, 122.6765823364, 245.3531646729, 368.0297546387]
    
    PS cbuffer1 first variable:
    cb1_v0 = [6.1338291168, 122.6765823364, 245.3531646729, 368.0297546387]
    

    But raw export still fails:

    rdc cbuffer 1130 --stage ps --binding 1 --raw -o cb.bin

    Result:

    error: not found: /draws/1130/cbuffer/0/1/data
    

    Direct RenderDoc API extraction works:

    VS cbuffer1: resource=ResourceId::198, offset=0, descriptor_size=65536, GetBufferData wrote 192 bytes, reflected block size=112
    PS cbuffer1: resource=ResourceId::198, offset=0, descriptor_size=65536, GetBufferData wrote 192 bytes, reflected block size=128
    

    So cbuffer --raw appears to be broken at the CLI/VFS path layer, not because RenderDoc cannot provide the data.

    2. rdc mesh --stage vs-in

    D3D12: vs-in exports the same data as vs-out

    Test capture: D3D12

    EID: 7459

    Commands:

    rdc mesh 7459 --stage vs-in --json
    rdc mesh 7459 --stage vs-out --json

    The two outputs have identical vertices, indices, and faces.

    vs-in first vertex from rdc mesh:

    [-16788.84765625, -616.9601440429688, 10.0, 17140.150390625]
    

    But the actual IA input layout is:

    ATTRIBUTE0  slot 0 offset 0 float3
    ATTRIBUTE13 slot 1 offset 0 uint1 per-instance
    

    Direct IA vertex-buffer decode gives:

    ATTRIBUTE0 first vertex:
    [590.5662841796875, -24.96360969543457, 341.7481384277344]
    

    So on this D3D12 capture, rdc mesh --stage vs-in does not export the original input-assembler position geometry. It exports the same post-VS position data as vs-out.

    D3D11: vs-in fails even though RenderDoc GUI Mesh Viewer Input tab has valid data

    Test capture: D3D11

    EID: 1130

    Draw type:

    DrawIndexedInstancedIndirect(<170412, 1>)
    

    Command:

    rdc mesh 1130 --stage vs-in --json

    Result:

    {"error": {"message": "no PostVS data at this event"}}
    

    However, RenderDoc GUI Mesh Viewer Input tab shows valid IA data. Direct IA decode through GetVertexInputs() + GetVBuffers() also succeeds.

    Attributes:

    POSITION
    NORMAL
    COLOR
    TEXCOORD0
    TEXCOORD1
    TEXCOORD2
    TEXCOORD3
    TANGENT
    

    Example decoded IA data:

    POSITION[0] = [-0.1154867411, -0.1212715656, -0.1162633896]
    NORMAL[0]   = [0.9934030175, -0.0760263726, 0.0858583599]
    TEXCOORD0[0] = [0.9970703125, 0.970703125]
    

    I also tested a normal D3D11 DrawIndexed() draw.

    EID: 1067

    rdc mesh 1067 --stage vs-in --json

    Result:

    {"error": {"message": "no PostVS data at this event"}}
    

    But direct IA decode succeeds:

    POSITION[0]  = [0.0, 0.0, 1.0361328125, 1.0]
    TEXCOORD0[0] = [0.1883544921875, 0.9931640625]
    

    So this is not limited to indirect draws.

    Suggested implementation direction

    For mesh --stage vs-in, relying only on controller.GetPostVSData(..., MeshDataStage.VSIn) is not sufficient for these Windows captures.

    A robust implementation probably needs a fallback that mirrors what RenderDoc GUI Mesh Viewer Input can do:

    1. Read IA layout from pipe.GetVertexInputs().
    2. Read bound vertex buffers from pipe.GetVBuffers().
    3. Read index buffer from pipe.GetIBuffer().
    4. Decode the POSITION input semantic, respecting:
      • vertexBuffer
      • byteOffset
      • vertex buffer byteOffset
      • stride
      • index buffer format/offset
      • base vertex
      • per-instance inputs where relevant
    5. Use the decoded POSITION stream + decoded index buffer for the OBJ.

    For cbuffer --json, choose the value vector according to reflected variable type, e.g. u32v for uint variables instead of always using f32v.

    For cbuffer --raw, the raw export path should include stage, or otherwise resolve the exact stage-specific constant block before exporting. The current VFS path /draws/<eid>/cbuffer/<set>/<binding>/data is both missing from the populated VFS tree and ambiguous when multiple stages bind the same register.

  7. Misaka-Mikoto-Tech commented on Jul 6, 2026

    @Misaka-Mikoto-Tech
    ContributorAuthor

    I think there are two different use cases that should be separated:

    1. Full VS-IN / IA attribute export

    mesh --stage vs-in currently seems to export only one selected position-like attribute plus indices/faces. That is useful for OBJ-style geometry, but it is not equivalent to RenderDoc's Mesh Viewer Input tab.

    VS input is not always a conventional position-backed mesh. For example, UI/sprite/procedural meshes may store only indices, tile IDs, corner IDs, or instance data in VS inputs, and reconstruct the real position in the vertex shader from constants/textures. In those cases there may be no POSITION attribute at all.

    So a full VS-IN export should dump all input attributes per vertex/index and should not require any attribute to be named or inferred as POSITION.

    1. OBJ export from VS-IN

    For OBJ export, selecting a position attribute is still necessary because OBJ needs vertex positions. But the current name-based lookup is too fragile.

    A better position selection strategy would be:

    • Prefer an explicit semantic if present: POSITION, SV_Position, or maybe pos*.
    • Otherwise ignore per-instance attributes and infer from vertex-rate attributes.
    • Prefer float/vector formats, especially float3 or float4.
    • Prefer slot 0 and lower byte offsets.
    • If there are multiple plausible candidates, either choose the highest-ranked candidate with a warning or require an explicit user override.
    • Provide an override such as --position-attribute ATTRIBUTE0, --position-slot 0 --position-offset 0, or --position-index 0.
    • Include the selected attribute in JSON/metadata output, e.g. position_attribute, position_source: semantic|heuristic|user.

    For the D3D12 NTE capture I tested, the VS-IN attributes are ATTRIBUTE0 and ATTRIBUTE13; ATTRIBUTE0 is the actual position stream, and ATTRIBUTE13 is instance-rate uint data. A robust OBJ fallback should select ATTRIBUTE0 by heuristic or by user override, not fail just because no input semantic is named POSITION.

    Unreal/D3D12 captures often expose inputs as ATTRIBUTE0, ATTRIBUTE1, etc., where the names are not semantically meaningful.

    Originally posted by @Misaka-Mikoto-Tech in #265 (comment)

  8. Misaka-Mikoto-Tech commented on Jul 6, 2026

    @Misaka-Mikoto-Tech
    ContributorAuthor

    Thanks for the quick fixes and follow-up work!

    I reviewed the latest commits, especially 01527a8 and 73d1c65. The fixes look reasonable for the intended OBJ export workflow:

    • cbuffer --json now appears to handle typed values correctly instead of decoding uint data as floats.
    • cbuffer --raw now includes the shader stage in the VFS path, which should avoid the previous ambiguous/missing path issue.
    • mesh --stage vs-in now has a much better fallback path for OBJ export: semantic lookup first, then vertex-rate float/vector heuristic, with explicit overrides such as --position-attribute, --position-index, --position-slot, and --position-offset.
    • This should cover my D3D12/Unreal-style case where the VS input names are ATTRIBUTE0, ATTRIBUTE13, etc.; in that capture, ATTRIBUTE0 is the position stream and ATTRIBUTE13 is instance-rate uint data, so the new heuristic/override support should work.

    One clarification: this implements VS-IN OBJ export, not a full RenderDoc Mesh Viewer Input-tab export of all IA attributes. That distinction is fine for this issue; if I need full per-attribute VS-IN dumps later, I can handle that with a custom RenderDoc Python script.

    Thanks again for being responsive and for hardening the D3D paths.

  9. BANANASJIM commented on Jul 7, 2026

    @BANANASJIM
    Owner

    Closing based on the latest maintainer/user feedback. The D3D cbuffer fixes and VS-IN OBJ export hardening are delivered via #265 and #266: typed cbuffer values, stage-qualified raw cbuffer paths, IA fallback for VS-IN OBJ export, robust position selection, explicit overrides, and position metadata.

    Scope note: this closes the requested cbuffer + VS-IN OBJ export workflow. A full RenderDoc Mesh Viewer Input-tab / complete IA attribute-table export remains a separate feature if needed later, and should stay separate from OBJ export.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions