Exclude deprecated labels from training and test datasets

[jeffhandley]

Summary

• add excluded_labels inputs to training flow and reusable download/test actions
• add --excluded-labels parsing in Downloader and Tester, normalizing labels during arg parsing
• enforce downloader-side filtering for issues, pulls, and discussions so items with excluded labels are skipped from datasets
• add test-report caution output using summary alert API when excluded labels are detected
• add LabelUtils.NormalizeLabels and unit tests for label normalization

Validation

• dotnet test IssueLabeler/IssueLabeler.sln -c Release --nologo
• dotnet build IssueLabeler/IssueLabeler.sln -c Release --nologo

Closes #132.

[Copilot]

Pull request overview

This PR adds end-to-end support for excluding deprecated labels from both dataset generation (Downloader) and evaluation (Tester), with normalization utilities and reporting to ensure excluded labels don’t leak into training/test flows.

Changes:

• Add excluded_labels inputs to reusable download/test actions and propagate through the training workflow.
• Add --excluded-labels argument parsing (with normalization) and enforce downloader-side filtering for issues/PRs/discussions.
• Enhance test reporting to surface excluded-label usage and emit a CAUTION-style summary when excluded labels appear in results.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/action.yml	Adds excluded_labels input and forwards it to the Tester CLI.
download/action.yml	Adds excluded_labels input and forwards it to the Downloader CLI.
.github/workflows/labeler-train.yml	Wires excluded_labels through the training workflow to download/test steps.
IssueLabeler/src/Downloader/Args.cs	Parses --excluded-labels and normalizes the label list.
IssueLabeler/src/Downloader/Downloader.cs	Reports excluded labels and passes them into GitHub downloads.
IssueLabeler/src/Tester/Args.cs	Parses --excluded-labels and normalizes the label list for testing.
IssueLabeler/src/Tester/Tester.cs	Adds excluded-label reporting and CAUTION summary output when excluded labels are detected.
IssueLabeler/src/GitHubClient/GitHubApi.cs	Implements downloader-side filtering to skip items containing excluded labels.
IssueLabeler/src/Common/LabelUtils.cs	Introduces shared label normalization helper.
IssueLabeler/tests/Common.Tests/LabelUtilsTests.cs	Adds unit tests validating label normalization behavior.

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (7)

IssueLabeler/src/GitHubClient/GitHubApi.cs:362

• Same comparer concern as above: use StringComparer.OrdinalIgnoreCase for GitHub label name matching to keep behavior consistent and avoid culture-sensitive casing differences.

        pageSize = Math.Min(pageSize, 100);
        HashSet<string> excludedLabelSet = new(LabelUtils.NormalizeLabels(excludedLabels), StringComparer.InvariantCultureIgnoreCase);

IssueLabeler/src/GitHubClient/GitHubApi.cs:440

• Same perf issue as the issues/pulls path: if excludedLabelSet is empty, this per-item Any(...) scan is redundant. Add a Count > 0 guard to avoid extra label iteration when no excluded labels were provided.

                if (item.LabelNames.Any(excludedLabelSet.Contains))

IssueLabeler/src/Tester/Tester.cs:30

• HashSet enumeration order is non-deterministic, so the summary list ordering can vary run-to-run. Sorting the labels makes the summary stable (and easier to test/read) while preserving the same content.

        summary.AddMarkdownHeading("Excluded Labels Supplied", 2);
        summary.AddMarkdownList([.. excludedLabels.Select(label => $"`{label}`")]);
    });

download/action.yml:84

• In GitHub Actions, unset inputs are typically empty strings (not null). With the current != null checks, this can emit --excluded-labels "" / --excluded-authors , and ArgUtils.GetInputString treats whitespace/empty as null, causing TryGetStringArray to fail and the downloader to exit. Gate on truthiness (inputs.excluded_labels / inputs.excluded_authors) and quote the value so the flag is only passed when a non-empty string is supplied.

          ${{ (inputs.excluded_authors != null && format('--excluded-authors {0}', inputs.excluded_authors)) || '' }} \
          ${{ (inputs.excluded_labels != null && format('--excluded-labels "{0}"', inputs.excluded_labels)) || '' }} \

IssueLabeler/src/GitHubClient/GitHubApi.cs:210

• excludedLabelSet is created with StringComparer.InvariantCultureIgnoreCase, while label comparisons elsewhere use ordinal semantics. For GitHub label names (identifier-like), StringComparer.OrdinalIgnoreCase is a safer, consistent choice and avoids culture-sensitive casing edge cases.

This issue also appears on line 360 of the same file.

        pageSize = Math.Min(pageSize, 100);
        HashSet<string> excludedLabelSet = new(LabelUtils.NormalizeLabels(excludedLabels), StringComparer.InvariantCultureIgnoreCase);

IssueLabeler/src/GitHubClient/GitHubApi.cs:297

• When no excluded labels are supplied, this check still iterates every item's labels and calls HashSet.Contains (always false). Guarding on excludedLabelSet.Count avoids unnecessary work in the common case and keeps the hot path cheaper for large downloads.

This issue also appears on line 440 of the same file.

                if (item.LabelNames.Any(excludedLabelSet.Contains))

IssueLabeler/src/Tester/Tester.cs:67

• This PR is marked as closing #132, but the linked issue’s acceptance criteria requires tests covering excluded-label filtering and caution/reporting behavior. In the current repo, tests appear limited to IssueLabeler/tests/Common.Tests/* and there are no tests exercising GitHubApi downloader filtering or Tester excluded-label detection/report output, so the acceptance criteria around test coverage is not met yet.

if (excludedLabelDetections.ExistingCount > 0 || excludedLabelDetections.PredictedCount > 0)
{
    action.Summary.AddPersistent(summary =>
    {
        summary.AddAlert(
            $"Excluded labels were detected in test results: **{excludedLabelDetections.ExistingCount:N0}** existing-label matches and **{excludedLabelDetections.PredictedCount:N0}** predictions.",
            AlertType.Caution);

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (3)

download/action.yml:84

• The condition inputs.excluded_labels != null is effectively always true for GitHub Action inputs (unset inputs are typically an empty string), so this will pass --excluded-labels "" even when the user didn't provide anything. Use a truthy/empty-string check (consistent with test/action.yml) so the flag is only added when non-empty.

          ${{ (inputs.excluded_labels && format('--excluded-labels "{0}"', inputs.excluded_labels)) || '' }} \

IssueLabeler/src/Tester/Tester.cs:269

• excludedLabels.Contains(...) is executed for every item tested, making excluded-label detection O(n) per prediction (n = number of excluded labels). Precompute a HashSet<string> once (case-insensitive) and use excludedLabelSet.Contains(...) to keep per-item checks O(1).

    if (result.Label is not null && excludedLabels.Contains(result.Label, StringComparer.OrdinalIgnoreCase))
    {
        stats.ExcludedExistingCount++;
        RecordExcludedLabel(stats.ExcludedExistingByLabel, result.Label);
    }

    if (predictedLabel is not null && excludedLabels.Contains(predictedLabel, StringComparer.OrdinalIgnoreCase))
    {

IssueLabeler/src/GitHubClient/GitHubApi.cs:301

• Excluded-label filtering is currently only applied after downloading pages (client-side item.LabelNames.Any(excludedLabelSet.Contains)), which can still pull large volumes of items and consume more GraphQL quota/time than necessary. The linked issue #132 requests query-level filtering where available; consider switching to a search-based query (e.g., search(query:"repo:org/repo is:issue -label:...", type: ISSUE) / equivalent) and add tests to cover both query construction and the software-side fallback.

                if (item.LabelNames.Any(excludedLabelSet.Contains))
                {
                    if (verbose) action.WriteInfo($"{typeName} {org}/{repo}#{item.Number} - Excluded from output. Contains excluded label.");
                    continue;
                }

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (4)

IssueLabeler/src/GitHubClient/GitHubApi.cs:362

• Same as above: use StringComparer.OrdinalIgnoreCase for excluded label comparisons to avoid culture-dependent behavior and keep comparisons consistent with other label handling.

        pageSize = Math.Min(pageSize, 100);
        HashSet<string> excludedLabelSet = new(LabelUtils.NormalizeLabels(excludedLabels), StringComparer.InvariantCultureIgnoreCase);

IssueLabeler/src/GitHubClient/GitHubApi.cs:210

• Use StringComparer.OrdinalIgnoreCase for label-name comparisons. InvariantCultureIgnoreCase can behave unexpectedly for identifier-like strings (e.g., labels) under certain cultures, and it’s inconsistent with LabelUtils.NormalizeLabels (OrdinalIgnoreCase).

This issue also appears on line 360 of the same file.

        pageSize = Math.Min(pageSize, 100);
        HashSet<string> excludedLabelSet = new(LabelUtils.NormalizeLabels(excludedLabels), StringComparer.InvariantCultureIgnoreCase);

test/action.yml:22

• The excluded_labels input description mentions "training/test data", but this action is the test action. Consider narrowing the wording to the downloaded test dataset to avoid confusion about what this action affects.

  excluded_authors:
    description: "A comma-separated list of authors to exclude."
  excluded_labels:
    description: "A comma-separated list of labels that should be excluded from training/test data."
  limit:

IssueLabeler/src/GitHubClient/GitHubApi.cs:301

• Excluded-label filtering is a significant behavior change (items with excluded labels are skipped). The linked requirements for #132 call for tests covering the software-side exclusion fallback and the reporting/caution output, but this PR only adds unit tests for label normalization. Please add automated tests that assert items with excluded labels are not yielded and that the tester emits the CAUTION alert when excluded labels are detected in results.

                if (item.LabelNames.Any(excludedLabelSet.Contains))
                {
                    if (verbose) action.WriteInfo($"{typeName} {org}/{repo}#{item.Number} - Excluded from output. Contains excluded label.");
                    continue;
                }

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (5)

IssueLabeler/src/GitHubClient/GitHubApi.cs:362

• Same as above: for label identifiers, prefer StringComparer.OrdinalIgnoreCase over InvariantCultureIgnoreCase when building the excluded-label set.

        pageSize = Math.Min(pageSize, 100);
        HashSet<string> excludedLabelSet = new(LabelUtils.NormalizeLabels(excludedLabels), StringComparer.InvariantCultureIgnoreCase);

test/action.yml:21

• The excluded_labels input description here mentions "training/test data", but this composite action only runs the Tester. Consider updating the wording to avoid implying this action affects training datasets.

  excluded_labels:
    description: "A comma-separated list of labels that should be excluded from training/test data."

IssueLabeler/src/GitHubClient/GitHubApi.cs:210

• Label names are identifiers; using InvariantCultureIgnoreCase can cause subtle culture-related edge cases and is slower than ordinal comparisons. Prefer StringComparer.OrdinalIgnoreCase for label set membership checks.

This issue also appears on line 360 of the same file.

        pageSize = Math.Min(pageSize, 100);
        HashSet<string> excludedLabelSet = new(LabelUtils.NormalizeLabels(excludedLabels), StringComparer.InvariantCultureIgnoreCase);

IssueLabeler/src/GitHubClient/GitHubApi.cs:301

• Issue #132 acceptance criteria calls for tests covering downloader-side exclusion fallback and reporting/caution output. This new excluded-label skip logic is currently untested in this repo (only Common utilities have unit tests), so regressions in filtering behavior and reporting would be hard to catch.

                if (item.LabelNames.Any(excludedLabelSet.Contains))
                {
                    if (verbose) action.WriteInfo($"{typeName} {org}/{repo}#{item.Number} - Excluded from output. Contains excluded label.");
                    continue;
                }

test/action.yml:78

• inputs.retries is a comma-separated string that may include spaces (e.g. "30, 30, 300"). Without quoting, bash will split it into multiple arguments and break --retries parsing. The download action already quotes this input; this action should do the same for consistency and correctness.

          ${{ (inputs.limit && format('--{0}-limit {1}', inputs.type, inputs.limit)) || '' }} \
          ${{ (inputs.page_size && format('--page-size {0}', inputs.page_size)) || '' }} \
          ${{ (inputs.page_limit && format('--page-limit {0}', inputs.page_limit)) || '' }} \
          ${{ (inputs.retries && format('--retries {0}', inputs.retries)) || '' }} \
          ${{ inputs.verbose && '--verbose' || '' }}