Skip to content

Add value counts aggregation to select.group_by - #1114

Merged
ebhills merged 3 commits into
devfrom
recovery/select-group-by-counts
Aug 14, 2026
Merged

Add value counts aggregation to select.group_by#1114
ebhills merged 3 commits into
devfrom
recovery/select-group-by-counts

Conversation

@ebhills

@ebhills ebhills commented Aug 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a counts aggregation to select.group_by
  • return each distinct value and its frequency as a dictionary
  • convert dictionary keys to JSON-safe strings
  • represent missing values as null and booleans as lowercase true or false
  • support existing per-column output renaming behavior

Why

Grouped recipes sometimes need the distribution of values within each group rather than only a scalar count or a list of the original values. Without this aggregation, recipes need additional transformations to construct a value-to-frequency mapping.

Impact

Recipes can now request value counts directly:

wrangles:
  - select.group_by:
      by: Group
      counts:
        - Selection: Selection Counts
      auto_rename_columns: false

The resulting Selection Counts value is a JSON-safe dictionary such as {"Primary": 2, "None": 1}.

Validation

  • complete TestGroupBy class: 30 passed
  • grouped smoke coverage for strings, nulls, and booleans passed
  • git diff --check passed
  • two existing Python invalid-escape SyntaxWarning warnings were emitted; no test failures

@ebhills ebhills self-assigned this Aug 3, 2026
@ebhills
ebhills requested a review from Copilot August 3, 2026 02:22
@ebhills
ebhills marked this pull request as ready for review August 3, 2026 02:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new counts aggregation option to select.group_by so grouped recipes can return per-value frequency distributions (as JSON-safe dictionaries) instead of only scalar aggregates.

Changes:

  • Extend select.group_by to support a counts aggregation that returns {value_as_string: frequency} per group.
  • Normalize keys for JSON-safety (string conversion, lowercase boolean strings, and a "null" key for missing values).
  • Add a unit test validating the new aggregation and per-column output renaming with auto_rename_columns: false.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
wrangles/recipe_wrangles/select.py Adds schema documentation and implements the counts aggregation using value_counts(dropna=False) with JSON-safe key normalization.
tests/recipes/wrangles/test_select.py Adds a test covering counts output structure and column renaming behavior.

Comment thread wrangles/recipe_wrangles/select.py Outdated
Comment thread tests/recipes/wrangles/test_select.py
ebhills and others added 2 commits August 14, 2026 15:03
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@ebhills
ebhills merged commit 8c155e5 into dev Aug 14, 2026
1 of 2 checks passed
@ebhills
ebhills deleted the recovery/select-group-by-counts branch August 14, 2026 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants