Skip to content

GC ratchet has broad deterministic fingerprint shifts that need attribution #8051

Description

@proggeramlug

Failure

The scheduled GC ratchet now produces deterministic counter fingerprints that differ substantially from the pinned baseline across roughly twenty rows. This is not a one-row/noise-shaped failure and must be attributed before any re-pin.

Exact evidence: run 31734917043 / gc-ratchet job 94564108190, head 3381e1b89048592cfc91f3d4a43349c6e7c16015. Duplicate samples in the artifact agree, making host jitter an inadequate explanation for the counter deltas.

Representative shifts from the job's pinned-baseline comparison:

workload counter pinned current delta
survivor-heavy copied objects 35,943 44,539 +23.92%
cross-gen copied objects 8,214 4,107 -50.00%
cross-gen copied bytes 656,256 262,592 -60.00%
shape-heavy heap-used bytes 12,419,456 21,516,800 +73.27%
async-heavy copied objects 13,013 241 -98.15%

Other rows shift copied/promoted/freed bytes in both directions. The current pinned file is therefore not being updated as part of freshness triage.

Required investigation

  • Compare the baseline capture SHA and current source/runtime configuration, then attribute each coherent shift cluster to a collector, allocation-layout, or workload change.
  • Classify attributed changes as regression or accepted mechanism evolution with source evidence and a local same-binary repeat.
  • Re-pin only the accepted rows, with the attribution recorded alongside the new capture.

Acceptance

  • No unexplained row is hidden by widening thresholds or wholesale baseline replacement.
  • Regressions are fixed and locally remeasured; accepted mechanism changes get a reviewed, reproducible baseline update.
  • Sabotage/self-tests still demonstrate that the ratchet rejects a meaningful counter regression.

Refs #7737 item 4 and #7966.

Activity

  1. proggeramlug commented on Aug 13, 2026

    @proggeramlug
    ContributorAuthor

    The pinned artifact is not a current-source baseline. Its provenance is:

    commit    98e9ecdb50075170e4be4469226b5379ebd67e73
    code_tree d55c2f44d230c2eb25a40a187d7a941ee0afbed6
    generated 2026-08-12T06:33:42Z
    

    The repin commit (#7921) explicitly accepted #7886/#7888/#7895. Between that capture and current head, several changes directly alter the counters that are now red:

    These are plausible attribution groups, not permission to re-pin. In particular, the current artifact came from a 3-vCPU virtual M1 runner (load_average 17.1/26.81/20.57) while the pinned capture is a physical M1 mini, but duplicated current samples agree; deterministic collector counters should be re-run on the pinned host before using host load to dismiss any row.

    Next evidence needed remains a same-binary repeat on the pinned mini plus per-change attribution of the coherent row groups. No baseline changes are included in the gate-freshness fixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions