feat(configs): hydra config groups for the batchflow stack

[ElmoPA]

data, experiment, evaluator and callbacks groups, plus the three model configs
any surviving config references. Per-arm model variants are deliberately not
included.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com

[ElmoPA]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat(configs): hydra config groups for the batchflow stack #558 : 2 dependent PRs (#560 , #564 ) 👈 (View in Graphite)
• test: batchflow test suite and retained tooling #557
• feat(eval): batchflow evaluation stack #556
• feat(algo): H-Net, diffusion and BC algorithms #555
• feat(models): H-Net and diffusion model zoo #554
• refactor(sim): give agents their own interface so the env stops branching on them #559
• sim: Tsimulation with sim_v1 and sim_v2 #553
• feat(infra): batchflow pipeline core, rldb unions, collapsed embodiment enum #552
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

Claude Code Review

Review: PR #558 — Hydra config groups for the batchflow stack

Summary

Large-scale reorganization of egomimic/hydra_configs/ into nested group directories (data/, callbacks/, model/, evaluator/, experiment/). Mostly file renames + new bundle configs; also folds in a couple of unrelated changes (Tsimulation obstacle levels, a docstring edit in eval_sim.py).

Key concerns

1. Stray edit in egomimic/eval/core/eval_sim.py — broken indentation in a comment

# cosmetic. Existing eval_hnet_sim.yaml and downstream callers keep working.
        # cosmetic. Existing evaluator/hnet/sim.yaml and downstream callers keep working.
HNetSimEval = PackedSimEval

The replaced line has 8 leading spaces at module scope. It's a comment so it parses, but it's clearly an accidental sed artifact. Fix to column 0.

2. Self-referential defaults in checkpoint variants

callbacks/checkpoint/ema.yaml, thin.yaml, transplant.yaml, transplant_router.yaml all do:

defaults:
  - checkpoint/base@_here_

Inside callbacks/checkpoint/, the relative group is just base, not checkpoint/base. Compare to callbacks/checkpoint/base.yaml which correctly uses /callbacks/base@_here_ (absolute). Almost certainly these need to be - base@_here_ or - /callbacks/checkpoint/base@_here_. Please instantiate each one via a smoke test (python -c "import hydra; hydra.compose(...)") before merging — Hydra defaults-list bugs surface only at compose time.

3. Missing newline at EOF in callbacks/base.yaml

Minor, but the diff shows \ No newline at end of file. Style-wise the rest of the repo terminates configs with a newline.

4. pl_utils.callbacks.* module paths — verify they exist

_target_s changed from egomimic.utils.timing_callback.WandbProfilerLogger (old) to egomimic.pl_utils.callbacks.timing_callback.WandbProfilerLogger (new). Also referenced: pl_utils.callbacks.random_attn_dropout, pl_utils.callbacks.chunker_residual_scheduler, pl_utils.callbacks.ratio_loss_scheduler. If this PR doesn't also move those files, every callback config is broken. Please confirm the module moves happened (or are in a companion PR that lands first).

5. Unrelated changes bundled in

• Tsimulation/sim_v2/pushshapes/obstacles.py: new levels 23–26, changed _collection_levels assertion 22 → 26. Substantive logic change, unrelated to config groups. Should be a separate PR — this is exactly the kind of change that gets missed in a "config rename" review.
• eval_sim.py docstring edit (see [FIX] use cv2.resize to fix OOM issue for image reading #1).

6. Docstring drift not verified

CONTRIBUTING_DATA.md, README.md, and latent_dataset.py update paths (e.g., aria.yaml → aria/base.yaml). But I don't see aria/base.yaml created in this diff — the renames go aria.yaml → debug/single_episode/human_cartesian.yaml. Either the docs are wrong or a file is missing. Grep the tree and reconcile.

7. Backward-compat / active experiments

Deleted files with no shim:

• data/cotrain_pickplace_qwen_objgen.yaml, cotrain_pickplace_qwen_wrist.yaml, aria_keypoints.yaml, aria_keypoints_wrist.yaml

Replacements exist at new paths (e.g., pick_place/cartesian/eva_human_qwen_objgen.yaml). Any in-flight sweep / submitit job / wandb resume referencing old data=cotrain_pickplace_qwen_objgen will break. Consider (a) a one-liner mapping table in the PR description, and (b) checking #experiments / recent launch scripts for stale references.

Suggestions

1. Fix the four self-referential defaults entries in callbacks/checkpoint/.
2. Split the Tsimulation obstacle changes into their own PR.
3. Fix the eval_sim.py indentation.
4. Add a compose-time smoke test in CI: iterate every top-level experiment config and call hydra.compose(...) — would have caught Main fixes move to roger's pipeline branch #2 and Merge latest progress from Main to UCSD branch #4.
5. Reconcile aria/base.yaml / scale/base.yaml doc references with the actual new tree.
6. Post the rename table in the PR body so people running long experiments can update their launch commands.

Verdict: Request Changes

Primarily for the checkpoint defaults-list bug (#2) and the unrelated Tsimulation changes (#5). Everything else is small. Once compose-tested and split, this is a nice cleanup.

---

Reviewed by Claude · Review workflow

[github-actions]

Claude Code Review

Review

Summary

Large-scale reorganization of Hydra config groups into nested directories (data/, callbacks/, evaluator/, etc.) with new bundle configs for callbacks. Also includes unrelated changes to Tsimulation/pushshapes/obstacles.py and a couple of doc comments.

Key concerns

1. Missing newline at EOF in egomimic/hydra_configs/callbacks/base.yaml (log_every_n_steps: 100 with no trailing newline). Minor but the old file had the same issue — worth fixing while touching it.

2. Broken comment indentation in eval_sim.py — the diff shows:

   # cosmetic. Existing eval_hnet_sim.yaml and downstream callers keep working.
           # cosmetic. Existing evaluator/hnet/sim.yaml and downstream callers keep working.

   The replacement comment is indented 8 spaces mid-module, which is a syntax-adjacent oddity (still a comment, so won't error, but looks like a botched sed). Please clean up.

3. WandbProfilerLogger module path change: base.yaml now points to egomimic.pl_utils.callbacks.timing_callback.WandbProfilerLogger, but the deleted defaults.yaml pointed to egomimic.utils.timing_callback.WandbProfilerLogger. Is that module move part of another PR? If not, this will fail to instantiate. Please verify import path exists on this branch.

4. Self-referential defaults in checkpoint/ema.yaml, checkpoint/thin.yaml, checkpoint/transplant.yaml, checkpoint/transplant_router.yaml:

   defaults:
     - checkpoint/base@_here_

   These files live under callbacks/checkpoint/. A relative checkpoint/base from inside checkpoint/ will resolve to callbacks/checkpoint/checkpoint/base.yaml — which doesn't exist. Compare with the bundles which correctly use /callbacks/checkpoint/base@_here_ (absolute). This is almost certainly broken. Should be either base@_here_ (relative to same dir) or /callbacks/checkpoint/base@_here_ (absolute).

5. Unrelated changes in Tsimulation/sim_v2/pushshapes/obstacles.py: adds levels 23–26 (radial rooms, plus, three-way), changes assert len(base_families) == 22 → 26. This is a substantive obstacle-level API change bundled into a "config groups" PR. Please split — anyone using level ranges/families downstream needs to review this separately, and it complicates rollback.

6. Massive scope: config rename affects nearly every training entry point. No mention in the description of:

   • How you validated existing configs still resolve (hydra --cfg job sweeps?)
   • Whether checkpoint resume paths embedded in run dirs still work
   • Norm stats caching — some caches key on config paths

7. bundle/attn_dropout.yaml references /callbacks/checkpoint/base@_here_ — that composes model_checkpoint from base, but doesn't set every_n_epochs overrides like other bundles. Confirm the base's every_n_epochs: 100 is what you want here (vs. chunker_dropout.yaml which overrides to 20).

Suggestions

1. Run hydra --cfg job --resolve against every canonical experiment config in hydra_configs/experiment/ and confirm each resolves. Attach the summary to the PR.
2. Fix the defaults: - checkpoint/base@_here_ bug (see Merge latest progress from Main to UCSD branch #4). Test by actually instantiating one of the transplant configs.
3. Split the obstacles.py change into a separate PR.
4. Fix the botched comment in eval_sim.py.
5. Confirm egomimic.pl_utils.callbacks.timing_callback exists — if not, update the target or land the module move first.
6. Add a mapping table (old→new) to the PR body so downstream users updating their local launch scripts have a reference. There are ~40 renames; a table saves everyone time.
7. Consider a shim script or data/aliases.yaml for one release to avoid breaking in-flight experiments that reference old paths.

Verdict: Request Changes

The checkpoint/* self-referential defaults look broken, and the unrelated obstacle changes should be split out. Once those are addressed and you've confirmed all downstream configs resolve, this is fine — the reorg itself is reasonable and overdue.

---

Reviewed by Claude · Review workflow