Skip to content

[FEA]: Add ordering support to MultiStoragePath for deterministic sorting #62

Closed

Description

@sagostinho-nvidia

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

Low (would be nice)

Please provide a clear description of problem this feature solves

The following pattern is common to see when globing contents in a folder using python Path objects:

from pathlib import Path

directory = Path("some_path")
paths = list(sorted(directory.glob("*.parquet")))

Sorting the glob results gives a deterministic path order. This is useful because glob() iteration order can vary across machines/filesystems, while sorting should return the same list as long as the directory contents are the same.

With MultiStoragePath, the same pattern currently fails:

TypeError: '<' not supported between instances of 'MultiStoragePath' and 'MultiStoragePath'

It would be useful for MultiStoragePath to implement ordering operators, likely based on the normalized string/path representation, so that common Python ordering utilities such as sorted(...), min(...), and max(...) work consistently with pathlib.Path.

Feature Description

.

Describe your ideal solution

.

Describe any alternatives you have considered

No response

Additional context

No response

Code of Conduct

  • I agree to follow the Code of Conduct
  • I have searched the open feature requests and have found no duplicates for this feature request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions