Skip to content

repair: split-group completion is not bounded by FetchesPerCycle #751

Open

Description

@tdakkota

FetchesPerCycle caps how many targets one repair pass sends to peers, but split-group completion (repair.Pass.complete, added in #720 for #717) runs outside that cap. Each member round asks, in one FetchWants call, for every block a unit's split groups still lack, and the rounds repeat while any unit gains a newly discovered group. One target can therefore make a single pass copy every member of a wide split, and of every group nested in it, in one synchronous batch. Nothing bounds the network traffic, the memory for the answers held in answers/askedBy, or the wall time. u.asked makes the loop terminate for a finite lineage, but it does not bound the work.

Group width comes from a merge's output count, and the identity layer puts no ceiling on it. A small MaxPartBytes, a large flush, or repeated splits all produce wide groups legitimately.

#749 (the lineage catalog for #725) widens which groups completion reaches:

  • Index.Missing follows every group the catalog records that derives from a want's rows.
  • groupsNeeded includes groups the unit's rows derive through whose ancestry overlaps what the node holds, not only the groups the fetched parts carry.

It does not change the shape of the loop.

Suggested direction

  • Apply a per-pass member budget across the completion rounds.
  • Keep staged progress across cycles, the fetched members of an incomplete unit, so a pass that stops at the budget resumes rather than discarding the members and fetching them again forever.
  • Alternatively, send a group wider than the budget to wholesale adoption of the peer's index instead of member-by-member repair.

No rows are dropped: an incomplete unit commits nothing and is started over on the next pass.

Activity

  1. added
    performanceThroughput, latency, memory or I/O efficiency
    area/recordengineRecord engine (logs, traces, profiles, exemplars)
    area/clustercluster/: replication, partsync, EC, router, etcd
    on Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/clustercluster/: replication, partsync, EC, router, etcdarea/engineMetrics enginearea/recordengineRecord engine (logs, traces, profiles, exemplars)performanceThroughput, latency, memory or I/O efficiency

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions