Skip to content

Leading-slash object keys cause cache lock files to be created outside the configured cache directory #210

Closed

Description

@bbrianxiao

When reading a prepared WebDataset through Energon, object keys can begin with /. With disk caching enabled, MSC constructs the cache lock path for these keys outside the configured cache directory.

This can cause permission errors and prevent the dataloader from reading the dataset.

Versions

  • Multi-Storage Client 1.2.0
  • Megatron-Energon 7.4.1

Steps to reproduce

  1. Configure an MSC profile pointing to S3-compatible object storage.
  2. Enable disk caching with a writable local cache directory.
  3. Read a prepared WebDataset through Energon using an msc:// path.
  4. When Energon passes a leading-slash object key, observe where MSC attempts to create its cache lock.

Expected behavior

Cache lock files remain inside the configured cache directory, including for object keys beginning with /.

Actual behavior

The leading slash causes the lock path to resolve outside the configured cache directory.

Activity

  1. self-assigned this
    on Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions