Skip to content

feat(doc|cli|review|scan|prompt): add structured category and severity fields to review findings#299

Closed
ScarletCarpet wants to merge 1 commit into
alibaba:mainfrom
ScarletCarpet:feat/support-category
Closed

feat(doc|cli|review|scan|prompt): add structured category and severity fields to review findings#299
ScarletCarpet wants to merge 1 commit into
alibaba:mainfrom
ScarletCarpet:feat/support-category

Conversation

@ScarletCarpet

Copy link
Copy Markdown
Contributor

add category and serverity feature which forked from pr-29 and add or fix some extra change @mvanhorn:

  • Use fun call instead of modify system prompt
  • Add colored severity badge to terminal text output for quick visual scanning
  • Add --level and --category CLI flags to filter findings at client-side
  • Normalize severity values: only high, medium, low
  • Remove test category and add imporvement category
  • Sort findings by severity (high -> medium -> low) then by category
  • Update README (en/zh-CN) with filter flag documentation and examples
  • Add comprehensive tests for normalization, filtering, sorting, badges

Description

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactoring (no functional changes)
  • Documentation update
  • CI / Build / Tooling

How Has This Been Tested?

  • make test passes locally
  • Manual testing (describe below)

Checklist

  • My code follows the project's coding style (go fmt, go vet)
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective or my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the documentation accordingly (if applicable)
  • I have signed the CLA

Related Issues

issue refer to:
#16
#96

original PR refer to:
#29

Comment thread cmd/opencodereview/review_cmd.go
Comment thread cmd/opencodereview/flags.go
Comment thread cmd/opencodereview/output.go
Comment thread cmd/opencodereview/output.go
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🔍 OpenCodeReview found 6 issue(s) in this PR.

  • ✅ 4 posted as inline comment(s)
  • 📝 2 posted as summary

📄 internal/model/review.go

Potential issue: When a severity or category filter is active, comments with an empty Severity or Category field are silently excluded. Since these fields are omitempty and depend on LLM output, the LLM may not always populate them. This means valid review comments could be dropped from the output without any indication to the user.

Consider allowing comments with empty severity/category to pass through filters, so that only comments that have a severity/category are checked against the filter. For example:

if len(fc.levels) > 0 && c.Severity != "" && !fc.levels[c.Severity] {
    continue
}
if len(fc.categories) > 0 && c.Category != "" && !fc.categories[c.Category] {
    continue
}

This way, comments without a severity/category are always included, while comments that do have one are properly filtered.


📊 Posting Statistics:

  • ✅ Successfully posted: 4 comment(s)
  • ❌ Failed to post: 1 comment(s)

⚠️ Inline comments shown in summary


📄 cmd/opencodereview/shared.go (L273-L277)

⚠️ GitHub could not post this as an inline comment: Unprocessable Entity: "Line could not be resolved"

Inconsistent comment count when filters are active. The telemetry recording (line 260) and PrintTraceSummary (line 274) both use the unfiltered len(comments), but the downstream output functions (outputTextWithWarnings, outputJSONWithWarnings) apply filterComments before rendering. When --level or --category filters are active, the user will see e.g. "10 comment(s)" in the trace summary but only 3 comments in the actual output, which is confusing.

Consider applying filtering once here in emitRunResult before both telemetry reporting and output, or pass the filtered count to PrintTraceSummary. For telemetry (RecordCommentsGenerated), you may want to keep the unfiltered count (to track LLM output volume), but the user-facing PrintTraceSummary should reflect what the user actually sees.

@ScarletCarpet

ScarletCarpet commented Jul 4, 2026

Copy link
Copy Markdown
Contributor Author

something like this:
image

@ScarletCarpet

Copy link
Copy Markdown
Contributor Author

feature more, may need support customlized level and category(including definition) in future.

@lizhengfeng101

Copy link
Copy Markdown
Collaborator

Hi @ScarletCarpet, thank you so much for this contribution — and for taking the time to fork, extend, and polish this feature. The colored badges, client-side filters, normalization, and the accompanying tests are all genuinely appreciated.

I want to be transparent about where things stand. There are currently three PRs implementing this same capability — #29, #79, and yours (#299) — and #29 came first. Out of fairness to the original author, we'd like to give #29 a chance to land first. If it doesn't get a response by next week, our team will step in and drive this feature to completion ourselves.

I also owe you an apology for the slow turnaround. Please know it isn't a reflection of the quality of your work. The real bottleneck for this feature isn't the coding — it's the evaluation. Getting the severity/category classification to behave consistently and measurably well across real-world reviews requires many rounds of evaluation and iteration. Because of that, it will likely be more efficient for our team to carry it forward directly — it saves a lot of back-and-forth communication overhead during those evaluation cycles. We hope you understand.

Thank you again for your patience and for your contribution. 🙏

@ScarletCarpet

Copy link
Copy Markdown
Contributor Author

Hi @ScarletCarpet, thank you so much for this contribution — and for taking the time to fork, extend, and polish this feature. The colored badges, client-side filters, normalization, and the accompanying tests are all genuinely appreciated.

I want to be transparent about where things stand. There are currently three PRs implementing this same capability — #29, #79, and yours (#299) — and #29 came first. Out of fairness to the original author, we'd like to give #29 a chance to land first. If it doesn't get a response by next week, our team will step in and drive this feature to completion ourselves.

I also owe you an apology for the slow turnaround. Please know it isn't a reflection of the quality of your work. The real bottleneck for this feature isn't the coding — it's the evaluation. Getting the severity/category classification to behave consistently and measurably well across real-world reviews requires many rounds of evaluation and iteration. Because of that, it will likely be more efficient for our team to carry it forward directly — it saves a lot of back-and-forth communication overhead during those evaluation cycles. We hope you understand.

Thank you again for your patience and for your contribution. 🙏

Fully understand. Quality First., just follow your own step. The main goal is to build a more powerfull OCR.
The AI age, code is cheap, idea and data are what matter most.

…y fields to review findings

add category and serverity feature which forked from [pr-29](alibaba#29)
and add or fix some extra change @mvanhorn:

- Use fun call instead of modify system prompt
- Add colored severity badge to terminal text output for quick visual scanning
- Add --level and --category CLI flags to filter findings at client-side
- Normalize severity values: only high, medium, low
- Remove `test` category and add `imporvement` category
- Sort findings by severity (high -> medium -> low) then by category
- Update README (en/zh-CN) with filter flag documentation and examples
- Add comprehensive tests for normalization, filtering, sorting, badges
@ScarletCarpet ScarletCarpet force-pushed the feat/support-category branch from f7e6fa6 to 282279d Compare July 5, 2026 12:12
@ScarletCarpet

Copy link
Copy Markdown
Contributor Author

all reviews had been fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants