BUG: Categorical.map() sort categories for unordered categoricals (#58153)#65286
Open
tinezivic wants to merge 1 commit intopandas-dev:mainfrom
Open
BUG: Categorical.map() sort categories for unordered categoricals (#58153)#65286tinezivic wants to merge 1 commit intopandas-dev:mainfrom
tinezivic wants to merge 1 commit intopandas-dev:mainfrom
Conversation
33e3402 to
6a44ddb
Compare
…#58153) Categorical.map() preserved the positional order of categories from the original (pre-mapped) categorical. For unordered categoricals, this meant the mapped values inherited an arbitrary category ordering, causing sort_values(key=...) to ignore custom sort orders. Fix: In Categorical.map(), when the categorical is unordered, sort the mapped categories and remap codes accordingly. For ordered categoricals, preserve the existing category order (since the ordering is user-defined). Mixed-type categories (e.g. str and float) that cannot be compared gracefully fall back to preserving the original category order. Closes pandas-dev#58153 Generated-by: GitHub Copilot
6a44ddb to
f8b7b35
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
keynot working on categorical column #58153doc/source/whatsnew/v3.1.0.rstfileBug
DataFrame.sort_values(key=...)on aCategoricalcolumn where the key maps values to a custom order ignored the key, sorting alphabetically instead of by the mapped values.Minimal reproduction:
Root Cause
Categorical.map()preserved the positional category order from the original categories. For unordered categoricals the mapped values inherited an arbitrary ordering, sosort_valuessorted by old category position instead of the mapped values.Fix: In
Categorical.map(), when the categorical is unordered, sort the mapped categories and remap codes accordingly. For ordered categoricals the existing category order is preserved. Mixed-type categories that cannot be compared (e.g.strandfloat) fall back to original order viaTypeErrorcatch.AI Disclosure
This fix was developed with the assistance of GitHub Copilot (Claude Sonnet 4.6). The AI assisted with code generation and diff review. The contributor verified the fix locally and confirmed all 49 tests pass.