Explore fixed-architecture partial parameter exchange

[holgerroth]

Context

PR #2 introduces participant-editable local_train() and aggregation while keeping the organizer-defined global model architecture fixed. The current contract requires every client to return the complete parameter mapping, and the locked validator rejects missing parameter keys.

For the open optimization tracks, participants may want to communicate only selected layers, tensors, sparse updates, masks, or low-rank representations. We should explore a supported protocol for this without allowing changes to the global model architecture or checkpoint schema.

Related work:

• Prototype FLModel participant hooks for open FeTS tracks #2 defines the participant local-training and aggregation interface.
• Add scoring-grade bidirectional communication accounting #3 will provide scoring-grade bidirectional communication accounting.

Goals

• Let participant algorithms exchange subsets or compressed representations of the parameters in the fixed organizer model.
• Preserve the organizer-defined parameter names, shapes, dtypes, initial checkpoint, and complete global checkpoint format.
• Define unambiguous behavior for omitted parameters.
• Keep full-model exchange as a backward-compatible baseline.
• Make the interface task-neutral so it can later support both segmentation and classification.
• Ensure all values needed to describe an update, including keys, masks, indices, metadata, and auxiliary tensors, are included in communication accounting.

Suggested staged design

Phase 1: partial client-to-server updates

• Continue broadcasting the full global model to each client.
• Allow local_train() to return a validated subset of parameter updates rather than requiring every key.
• Prefer an explicit update contract, such as a subset with ParamsType.DIFF, rather than overloading participant metadata with unvalidated payloads.
• Require every returned key to exist in the organizer model and retain its expected shape and dtype.
• Define an omitted parameter as "no update from this client" rather than a zero-valued model parameter.
• Define how the custom aggregator combines different subsets from different clients and how parameters with no contributors are preserved.
• Require the aggregator output used for checkpointing and evaluation to reconstruct a complete model state.

Phase 2: optional partial server-to-client exchange

• Evaluate whether clients may persist a synchronized full model and receive only selected global changes after initialization.
• Define initialization, resynchronization, missed-round, and periodic-full-broadcast behavior.
• Ensure hidden evaluation can reproduce the client state deterministically.

Validation and safety requirements

• Reject unknown parameter names and incompatible shapes or dtypes.
• Reject unsupported auxiliary communication paths that bypass the update contract or accounting ledger.
• Distinguish an intentionally empty update from malformed output and define whether empty updates are allowed.
• Preserve organizer-owned evaluation metrics and transport metadata.
• Produce a complete global checkpoint that can be loaded by the fixed organizer model after every round.
• Fail clearly when an aggregation algorithm does not support the submitted subset representation.

Communication-accounting integration

Coordinate this work with #3. Official accounting must include:

• transmitted tensor values
• parameter identifiers
• sparse indices or masks
• compression dictionaries or low-rank factors
• participant metadata and any other auxiliary payload

The accounting evidence should demonstrate an actual reduction in serialized bytes. Returning all parameters while logically updating only selected layers must remain classified as full communication.

Acceptance tests

• A valid client-to-server subset update completes a two-client, two-round simulation.
• Different clients can update different parameter subsets with documented per-parameter aggregation semantics.
• Omitted parameters remain unchanged when no client contributes an update for them.
• Unknown keys and incorrect shapes or dtypes fail before transport or aggregation.
• The existing full-parameter baseline remains backward compatible.
• The final checkpoint contains the complete fixed model state and passes organizer evaluation.
• A two-client, two-round H100 validation demonstrates successful reconstruction and aggregation.
• Once Add scoring-grade bidirectional communication accounting #3 is implemented, the recorded serialized byte count is lower than the equivalent full-exchange baseline and reconciles with NVFLARE transport statistics.

Non-goals

• Allowing participants to change the global model architecture.
• Implementing the classification task in this issue.
• Defining the final communication-aware ranking formula, which remains tracked by Add scoring-grade bidirectional communication accounting #3.

Open questions

• Should the first implementation support only named tensor subsets, or also sparse and low-rank representations?
• Should the participant aggregator receive the sparse/subset representation directly, or should organizer code reconstruct a standard update first?
• Is client-to-server partial exchange sufficient for the initial challenge, or should bidirectional partial exchange be required?
• Do we need periodic full synchronization rounds for robustness and reproducibility?