Skip to content

Allow data-cell families to span multiple data layers #81

Open

Description

@knoepfel

Motivation

The design currently defines a data-cell family as "a family of data cells that are in the same data layer" in doc/ch_conceptual_design/data_organization.rst (line 23). This is unnecessarily restrictive: a data-cell family need not correspond to only one layer.

In Phlex, data_cell_tracker tracks a family identified by a parent data-cell index and records child counts separately by layer. For example, the children of the Job cell can include both Run cells and calibration cells. These children belong to one parent-scoped family even though they occupy different layers. The tracker terminology is being changed from partition to family; this is a terminology change, not a change in grouping or execution behavior.

Requested Documentation Changes

  • Revise the data-cell family definition in doc/ch_conceptual_design/data_organization.rst to remove the same-layer requirement. Describe a data-cell family as an indexed collection of data cells that may span multiple layers, and explain the parent-index identification used for hierarchical child families.
  • Clarify that each individual data cell still belongs to its own data layer. Family membership does not collapse layers or require identical product contents across member cells.
  • Add a multi-layer example, such as one Job-scoped family containing Run and calibration children. Explain that counts and selections can remain layer-specific within that family.
  • Review the data-organization figure and caption (doc/ch_conceptual_design/data-organization.gv and its caption), including statements tying family formation exclusively to an unfold or a single layer. Keep the existing APA families as valid single-layer examples, but not as a restriction on all families.
  • Audit relevant glossary, conceptual-design, and technical-design references for the same-layer assumption and make their terminology consistent. The general indexed-family definition in doc/appendices/glossary.rst need not impose a layer restriction.
  • Clarify the distinction in doc/ch_conceptual_design/hofs/partitioned_folds.rst: partitioning specifies how an input family is divided into groups; the partition level identifies the data layer receiving one fold result per group. Each result folds over its corresponding input family. Preserve partition and partition level for this fold-specific grouping rule rather than globally replacing them with family.
  • Ensure the broader data-cell-family definition is not read as relaxing product-selector or fold input-type requirements. An operator can select the relevant layer/products from a family that spans multiple layers.

Acceptance Criteria

The documentation explicitly permits multi-layer data-cell families, explains a concrete parent-scoped example, and consistently distinguishes family membership from data-layer identity and fold partitioning. Existing single-layer families remain valid special cases.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions