added mint files - #1
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a MinT-based (“mint” skeleton) data/processing path alongside the legacy Rajagopal/Nimble B3D pipeline, including MinT preprocessing (NPZ cache), retargeting (SMPL-H → SSM67 markers → OpenSim IK), updated training scripts/configs, and smoke tests/docs to support end-to-end training and generation on MinT.
Changes:
- Add MinT preprocessing + cache schema (NPZ), normalization, dataset loaders, and training configs for surrogate/SINDy/diffusion.
- Add MinT retargeting + OpenSim IK utilities (SSM67 marker extraction/TRC writing + setup generation).
- Update existing training/guidance code to be skeleton-aware via
common.skeleton_configandmotion_cache_dir.
Reviewed changes
Copilot reviewed 61 out of 62 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_mint_smoke.py | Adds smoke tests covering key MinT modules and cache roundtrips. |
| surrogate/model.py | Makes surrogate I/O dims skeleton-dependent (MinT vs Rajagopal). |
| surrogate/mint_dataset.py | Adds MinT NPZ activation dataset for surrogate training. |
| surrogate/guidance.py | Switches normalization lookup to skeleton-aware motion cache dir. |
| sindy/windows.py | Adds MinT window indexing + NPZ window extraction path. |
| sindy/train.py | Adds --skeleton plumbing and switches cache requirements to skeleton-aware checks. |
| sindy/targets.py | Switches target dimensionality + muscle names to skeleton-aware helpers; adds MinT bio path. |
| sindy/guidance.py | Makes SINDy guidance skeleton-aware (MinT feature/bio computation paths). |
| sindy/dataset.py | Adds skeleton-aware lazy data preparation (MinT vs B3D indexing). |
| scripts/train_surrogate.py | Adds skeleton support and MinT dataset selection for surrogate training. |
| scripts/train_diffusion.py | Makes diffusion training skeleton/dataset aware; supports dataset=mint. |
| scripts/preprocess_mint.py | New MinT preprocess entrypoint producing NPZ cache + optional normalization. |
| scripts/generate_motion.py | Uses skeleton-aware cache dir and forwards skeleton into guidance. |
| scripts/discover_mint.py | Adds MinT discovery/report script (layout + overlap + bundled models). |
| scripts/compute_normalization.py | Adds --skeleton mint path to compute MinT Mean/Std. |
| README.md | Documents MinT-default pipeline, env vars, and updated target dimensions. |
| models/mint/ssm67_smplh_markers.yaml | Adds SSM67 marker ↔ SMPL-H vertex index mapping. |
| models/mint/README.md | Documents bundled Lai model, Bruno download, retargeting assets, OpenSim version notes. |
| models/mint/opensim/Setup_IK_ssm67.xml | Adds IK setup file for SSM67 marker set (but currently malformed XML). |
| models/mint/opensim/Setup_IK_Mocap.xml | Adds/updates OpenCap mocap IK setup reference. |
| models/mint/opensim/LaiUhlrich2022_markers_ssm67.xml | Adds pre-generated Lai marker set with SSM67 names. |
| models/mint/opensim/LaiUhlrich2022_markers_mocap.xml | Adds OpenCap mocap marker template reference. |
| mint/init.py | Exposes MinT muscle constants/names. |
| mint/cache_schema.py | Defines MinT NPZ schema + read/write helpers + metadata. |
| mint/coord_map.py | Provides MinT OpenSim coordinate discovery + fallback names/count. |
| mint/discovery.py | Implements MinT dataset inspection + HumanML3D overlap reporting. |
| mint/features.py | Adds MinT proxy u/c feature computation (NumPy + Torch). |
| mint/label_lookup.py | Adds musint-based MinT muscle label lookup aligned to HumanML3D ids. |
| mint/model_loader.py | Adds OpenSim model loading + basic model introspection utilities. |
| mint/muscle_schema.py | Defines canonical 402-muscle naming and zero-placeholder detection. |
| mint/opensim_ik.py | Adds TRC→OpenSim IK pipeline and q extraction from .mot. |
| mint/physics.py | Adds MinT proxy bio-channel computation (finite-diff based). |
| mint/retarget.py | Adds MinT-faithful retarget (SMPL-H fit → markers → IK) + bootstrap fallback. |
| mint/retarget_bootstrap.py | Adds legacy bootstrap retargeting implementation. |
| mint/retarget_result.py | Adds retarget result dataclass. |
| mint/smplh_fit.py | Adds SMPL-H fitting routine for HumanML3D joints using smplx. |
| mint/ssm67_markers.py | Adds SSM67 definitions + marker/xml/IK-setup generation helpers. |
| mint/trc_io.py | Adds TRC writer for OpenSim marker trajectories. |
| mint/virtual_markers.py | Adds SSM67 marker extraction + validation utilities. |
| datasets/mint_motion_dataset.py | Adds diffusion dataset for MinT NPZ q windows + captions. |
| datasets/mint_cache_stats.py | Adds MinT cache normalization stats computation + metadata writing. |
| diffusion/config.py | Adds DatasetName.MINT. |
| diffusion/registry.py | Registers MintMotionDataset for diffusion training. |
| common/skeleton_config.py | Adds skeleton selection + dimension/name helpers (MinT vs Rajagopal). |
| common/run_setup.py | Adds skeleton-aware cache + normalization requirement helpers. |
| common/paths.py | Adds MinT cache dir helpers + unified motion_cache_dir. |
| configs/train_surrogate_mint.json | Adds MinT surrogate training config. |
| configs/train_sindy_mint.json | Adds MinT SINDy training config. |
| configs/train_diffusion_mint.json | Adds MinT diffusion training config. |
| env/environment.yaml | Adds dependencies needed for MinT path (e.g., musint, smplx, pandas). |
| deploy/scripts/run-preprocess-mint.sh | Adds deploy helper to run MinT preprocessing. |
| deploy/README.md | Documents new MinT Kubernetes jobs/paths. |
| deploy/jobs/preprocess-mint/kustomization.yaml | Adds kustomize manifest for MinT preprocess job. |
| deploy/jobs/preprocess-mint/job.yaml | Adds MinT preprocess Kubernetes job. |
| deploy/jobs/train-surrogate-mint/kustomization.yaml | Adds kustomize manifest for MinT surrogate job. |
| deploy/jobs/train-surrogate-mint/job.yaml | Adds MinT surrogate training Kubernetes job. |
| deploy/jobs/train-sindy-mint/kustomization.yaml | Adds kustomize manifest for MinT SINDy job. |
| deploy/jobs/train-sindy-mint/job.yaml | Adds MinT SINDy training Kubernetes job. |
| deploy/jobs/train-diffusion-mint/sindy/kustomization.yaml | Adds kustomize manifest for MinT diffusion+sindy job. |
| deploy/jobs/train-diffusion-mint/sindy/job.yaml | Adds MinT diffusion+sindy Kubernetes job. |
Comments suppressed due to low confidence (2)
surrogate/mint_dataset.py:87
validate_activation_matrix()is called with keyword argumentatol, but its signature isvalidate_activation_matrix(arr, *, atol_zero=...)(mint/muscle_schema.py). This will raise aTypeErrorat runtime when indexing the dataset.
surrogate/model.py:23- The class docstring says this surrogate maps "Rajagopal q" windows, but the defaults now depend on the active skeleton (Rajagopal vs MinT). This is misleading for MinT training/inference.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
429
to
431
| parser.add_argument("--num_workers", type=int, default=0) | ||
| parser.add_argument("--seed", type=int, default=42) | ||
| parser.add_argument("--skeleton", default="", help="rajagopal|mint (default: SINDYFFUSE_SKELETON env)") | ||
| add_run_log_cli_args(parser) |
| continue | ||
| if skip_zero_placeholders: | ||
| act = data["muscle_activations"] | ||
| if validate_activation_matrix(act, atol=float(zero_atol)): |
| "\t\t<accuracy>1e-5</accuracy>", | ||
| "\t\t<IKTaskSet>", | ||
| "\t\t\t<objects>", | ||
| ",\n".join(tasks), |
Comment on lines
+350
to
+351
| out_path.write_text(body, encoding="utf-8") | ||
| return out_path |
Comment on lines
+10
to
+17
| <IKMarkerTask name="C7"> | ||
| <apply>true</apply> | ||
| <weight>5</weight> | ||
| </IKMarkerTask>, | ||
| <IKMarkerTask name="CLAV"> | ||
| <apply>true</apply> | ||
| <weight>5</weight> | ||
| </IKMarkerTask>, |
Comment on lines
+10
to
11
| from common.skeleton_config import motion_ndof, muscle_activation_rows, resolve_skeleton | ||
| from nimble.rajagopal_coord_map import RAJAGOPAL_NIMBLE_DOF_NAMES |
| from common.paths import mint_cache_dir | ||
| from common.skeleton_config import MINT_MUSCLE_COUNT | ||
| from datasets.splits import load_split_ids | ||
| from mint.cache_schema import KEY_HAS_LABELS, KEY_MUSCLE, KEY_Q, cache_has_labels, read_motion_cache |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.