Skip to content

Remove checkpoint migration playbook after W&B checkpoint migration is complete #73

Description

@SoheylM

docs/checkpoint_migration_playbook.md is still useful during the HF transition, but it should be removed once the historical W&B checkpoint migration is completed and no longer needed.

Done criteria:

  • historical checkpoint migration is complete
  • README no longer needs to point to the playbook
  • the file is deleted

Activity

  1. linear commented on Jun 24, 2026

    @linear
  2. mkeeler43 commented on Sep 2, 2026

    @mkeeler43
    Contributor

    Closing under the second of the two options @SoheylM offered in review of #75: record an
    explicit decision not to migrate, update the supported-generator documentation, and close
    or replace this issue.

    The decision: W&B-era checkpoints are not being migrated. It is a decision rather than
    a deferral. A migrated checkpoint arrives without the two things the current layout exists
    to provide — a config fingerprint identifying the hyperparameters that produced it, and a
    content hash tying a leaderboard row to exact bytes. It could be stored, but never
    canonically addressed and never verified, so no row could be ranked against it. Retraining
    a model costs less than a provenance story nobody can check.

    Written up in docs/checkpoint_layout.md § Historical checkpoints (963fc02).

    Against this issue's done criteria:

    • README no longer points to the playbook — done; no reference remains anywhere in the repo.
    • The file is deleted — done in Generator contract, shared evaluation layer, and HF-hosted leaderboard #75.
    • Historical checkpoint migration is complete — resolved by the decision above rather
      than by performing it. This is the criterion the issue was really tracking, and leaving
      it open would track work that is not going to happen.

    Also fixed the reporting that made the gap look larger than it is. --list-generators --check-availability queried every registered generator regardless of design kind, so on
    beams2d it reported 10 missing checkpoints and told the reader they could be trained and
    evaluated — false for the seven that cannot serve a 2D problem at all. It now separates
    does not fit the problem from nobody has published weights: 7 of 14 fit beams2d, 4
    have weights, 3 are actually owed — cgan_2d, gan_2d, pixel_cnn_pp_2d.

    Publishing those three, or narrowing the registry, is a separate project decision and not
    what this issue was about. Happy to reopen if the non-migration decision should be revisited.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions