Skip to content

Stuck at batch = next(data_iter) during training #21

Description

@tututu-7

When running the train_camera_control.py script, the training process is getting stuck at the line batch = next(data_iter) within the training loop. The code does not progress past this point, and the data loader does not return any batch.
[01/12 16:48:24 AnimateDiff:752]: encoder trainable scale: 199.262 M
[01/12 16:48:24 AnimateDiff:753]: attention processor trainable scale: 18.962 M
[01/12 16:48:24 AnimateDiff:754]: trainable parameter scale: 218.224 M
[01/12 16:48:25 AnimateDiff:768]: Building training datasets
[01/12 16:48:25 AnimateDiff:790]: Building validation datasets
[01/12 16:48:26 AnimateDiff:841]: ***** Running training *****
[01/12 16:48:26 AnimateDiff:842]: Num examples = 321
[01/12 16:48:26 AnimateDiff:843]: Num Epochs = 78
[01/12 16:48:26 AnimateDiff:844]: Instantaneous batch size per device = 1
[01/12 16:48:26 AnimateDiff:845]: Total train batch size (w. parallel, distributed & accumulation) = 1
[01/12 16:48:26 AnimateDiff:846]: Gradient Accumulation steps = 1
[01/12 16:48:26 AnimateDiff:847]: Total optimization steps = 25000

Changes I made: During the video_clips processing, the generated videos were placed in a newly created folder. However, when running the generate_realestate_json.py script to generate the JSON file, it did not execute successfully. So, I moved the MP4 files directly into the video_clips folder.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions