Repository navigation
fix(coderabbit-config): rank findings under the cap, index key nodes once per mapping (BE-7786) - #192
Conversation
…once per mapping (BE-7786) Round-2 review findings. The first two are regressions in the round-1 fix itself, not in the original change. * HIGH — the round-1 cap truncated in error-sort order, and the root `additionalProperties` error has an EMPTY `absolute_path`, so it sorts first and fans out to one finding per unknown key. 150 unknown root keys plus one `maxLength` violation therefore sliced the file-rejecting error away and left 100 warnings: exit 0 on a config CodeRabbit rejects WHOLE. Findings now carry a rank and `_apply_cap` drops the lowest first, so a file-rejecting error is never crowded out. The compensating note could not have covered this — a note does not reach the exit code, the same argument that turned the bounded-walk report into a finding. * HIGH — the cap bounded the output but not the work: `_extra_keys` returns every extra key in the mapping, and each finding rescanned all of `parent.value` constructing candidate keys. ~50k unknown root keys fit inside MAX_CONFIG_BYTES and cost ~1e9 constructions, so the job burned to its timeout instead of printing 100 annotations. The fan-out now stops at the cap, and the resolved-key -> node map is built once per mapping and cached on the composed node. 20k keys: 1.6s. * The incomplete-scan report was appended after both truncation sites, so a run that filled the cap emitted MAX_FINDINGS + 1 alongside a note saying only the first MAX_FINDINGS are reported. It goes through the same cap now, ranked above unknown keys so it is not the one dropped. * `_suggest_home` compared `_format_path`'s `[0]` against `_index_schema_paths`' `[]`, so neither the parent preference nor the self-exclusion could ever match for a key under a list element. Both vocabularies are normalized before the comparison. * Two keys still fell back to `str()`, reproducing the defect round 1 fixed: a `.nan:` key (NaN is not equal to itself, so it missed its own entry) and a merge-keyed mapping (`safe_load` flattens `<<: *anchor`, but the composed node keeps only `<<`). The index normalizes NaN and expands merge sources, with explicit keys keeping YAML's own precedence over merged ones and a cycle guard for `&x [*x]`. * `bounded` was set on ENTRY to `walk`, so spending the last budget unit on a known key whose value is a scalar flagged the trailing visit as a partial walk — a fully walked document reported as stopped early, which is a hard error under `strict_unknown_keys`. A node with nothing to inspect now returns before the bail-out. * The cutoff message said "after MAX_WALK_KEYS keys" while the counter also charges node visits, and the docs claimed the five explicitly closed objects were exempt from the vendored-schema staleness window. They are not: a key upstream adds at the root is missing from the vendored `properties` and its `additionalProperties: false` rejects it just the same. Both corrected. 87 tests (up from 72); every case above is pinned.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 5 minutes Limit details: You’ve used the included review currently available. Your 95 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Comment |
There was a problem hiding this comment.
🔍 Cursor Review — Consolidated panel
Triggered by @mattmillerai.
Found 6 finding(s).
| Severity | Count |
|---|---|
| 🟠 High | 1 |
| 🟡 Medium | 1 |
| 🟢 Low | 3 |
| ⚪ Nit | 1 |
Panel: 8/8 reviewers contributed findings.
…rges transitively (BE-7786)
Round-3 review findings on this PR's own fixes.
Exempting a leaf visit from the bail-out flag also exempted it from the
BUDGET, and the list branch recurses once per element with no budget check
of its own — so `MAX_WALK_KEYS` stopped bounding `walk()` calls and started
bounding only non-empty containers. A 1.6 KB aliased document over a
nested-`items` schema took 1.0 s and reported `bounded=False` ("I saw the
whole thing"); it is 0.03 s now. Only the FLAG is gated on there being
something to inspect; the visit is charged either way, which is what
`test_list_traversal_is_charged_against_the_walk_budget` was written to hold.
`_merge_entries` stopped at one level while PyYAML's `flatten_mapping` is
transitive, so `a: &a {x: 1}` / `b: &b {<<: *a}` / `reviews: {<<: *b}` put
`x` under `reviews` in the loaded document and a literal `"<<"` in the index
— the `str()` fallback with no line number, one level deeper. And `_descend`
compared raw key text, so an ancestor arriving through a merge ended the
descent at `<<`: same defect one level up. The index now carries the value
node too and `_descend` routes through it, which also stops `on:`/`true:`
being read as a list index (a bool IS an int).
The truncation note still said "only the first N are reported" after ranking
made the survivors not a prefix, and `_index_key`'s summary line named a
`None` sentinel the function never returns.
`OpennessTest` gains the premise behind the one gap left open by design: the
`islice` over `iter_errors` is unranked FIFO and runs a stage before
`_apply_cap`, so it could in principle drop a file-rejecting error — it
cannot against this schema because every self-closing object is either
unmultiplied or under a `maxItems`-bounded array, and a refresh that changes
that now fails here.
93 tests, up from 87.
There was a problem hiding this comment.
🔍 Cursor Review — Consolidated panel
Triggered by @mattmillerai.
Round 2 — ledger: 6 prior finding(s) across 1 round(s) (0 never answered).
Found 6 finding(s).
| Severity | Count |
|---|---|
| 🟡 Medium | 2 |
| 🟢 Low | 3 |
| ⚪ Nit | 1 |
Panel: 8/8 reviewers contributed findings.
… the node (BE-7786) Round-4 review findings. The transitive merge expansion had no depth bound. A `<<:` chain of ~1500 links is ~37 KB — far inside MAX_CONFIG_BYTES — and `safe_load` flattens it fine while this expansion, costing more frames per link, blew the recursion limit; `RecursionError` is not a `yaml.YAMLError`, so it escaped `validate` and killed an advisory check with a traceback. Bounded at `_MAX_MERGE_DEPTH`, degrading to the documented `str()` fallback. PyYAML's own `flatten_mapping` can lose the same race first (anchors in a lazily constructed sequence, so `reviews` flattens before them) — that now raises `ConfigError` and exits 2, like an oversized config: "I could not check this" must not read as "this is fine". `_descend` typed the PATH PART instead of the node, so an integer mapping key went down the sequence branch and died there. Reachable today: `reviews.mutually_exclusive_groups` takes arbitrary property names, so `1:` is legal and a rejecting error below it lost its line and rendered as `...[1]` — a list position the file does not have. `_descend` now dispatches on the node and `_format_path` is given the document so it can tell an integer key from a list index. The index was first-wins throughout, but `construct_mapping` assigns per pair, so `safe_load` keeps the LAST: a duplicate key annotated its earlier twin's line, and where a mapping carries two `<<:` keys the earlier one won here and the later one won in the document. Explicit keys now assign unconditionally and merge groups are consumed in reverse. `_apply_cap`'s docstring claimed a document-order tie-break; the two halves are concatenated, so it is emission order — corrected, with why the walk's overflow finding is deliberately the first dropped. `test_a_rejecting_error_cannot_be_crowded_out_before_ranking` modelled `maxItems` as a multiplicity bound, which it is not — jsonschema keeps validating elements past it. Reformulated to assert the four facts that actually hold, including that `maxItems` precedes `items` in key order so its REJECTING error is emitted ahead of the element errors. Verified by mutation: all four clauses fail when broken. 99 tests, up from 93.
Follow-up to #188, which merged while its round-2 review was still running. Two of the eight findings that round raised are regressions in #188's own review-round fix, and both are now live on
main— they affect every consumer repo runningcoderabbit-config-validate.yml.The two that matter
A file-rejecting error could be capped away, flipping the exit code to 0. #188 re-applied
MAX_FINDINGSby slicing the finding list, which is in error-sort order — and the rootadditionalPropertieserror has an emptyabsolute_path, so it sorts first and fans out to one finding per unknown key. A config with ~150 unknown root keys plus onemaxLengthviolation therefore sliced the file-rejecting error away, leaving 100 warnings and exit 0 on a config CodeRabbit rejects whole. Findings now carry a rank and_apply_capdrops the lowest first, so a file-rejecting error is never crowded out by warnings that happened to sort ahead of it. The truncation note could not have covered this: a note does not reach the exit code — the same argument #188 used to turn the bounded-walk report into a finding.Verified end to end: that input now exits 1, with the
maxLengthfinding present among the 100.The cap bounded the output but not the work.
_extra_keysreturns every extra key in the mapping, and each finding rescanned all ofparent.value, constructing every candidate key node on the way.k00000: 1is ten bytes, so ~50k unknown root keys fit insideMAX_CONFIG_BYTESat ~1e9 constructor invocations — the job burns runner minutes to its timeout instead of printing 100 annotations. #188 made this materially worse: it replaced a cheap string compare with aSafeConstructor+construct_object(deep=True)per candidate. The fan-out now stops at the cap, and the resolved-key → node map is built once per mapping and cached on the composed node. 20k unknown keys: 1.6s.The rest
MAX_FINDINGS + 1findings alongside a note saying only the firstMAX_FINDINGSare reported. It goes through the same cap now, ranked above unknown keys so it is not the one dropped._suggest_homecompared_format_path's[0]against_index_schema_paths'[], so for a key under a list element neither the parent preference nor the self-exclusion could ever match. Both vocabularies are normalized before comparing.str(), reproducing the "path naming a key that appears nowhere in the file" defect feat(coderabbit-config): flag unknown keys inside schema-open objects (BE-7786) #188 set out to fix: a.nan:key (NaN is not equal to itself, so it missed its own index entry) and a merge-keyed mapping (safe_loadflattens<<: *anchor, but the composed node keeps only<<). The index normalizes NaN and expands merge sources, with explicit keys keeping YAML's own precedence over merged ones, and a cycle guard for&x [*x].boundedwas set on entry towalk, so spending the last budget unit on a known key whose value is a scalar (reviews.profile: chill) flagged the trailing visit as a partial walk — a fully walked document reported as stopped early, which is a hard error understrict_unknown_keys. A node with nothing to inspect now returns before the bail-out.MAX_WALK_KEYSkeys" while the counter also charges node visits; and the docs claimed the five explicitly-closed objects were exempt from the vendored-schema staleness window. They are not — a key upstream adds at the root is missing from the vendoredpropertieslist and the vendoredadditionalProperties: falserejects it just the same. Both corrected.Round 3 — the review panel's findings on the fixes above
All six threads on this PR are addressed in
e257047.MAX_WALK_KEYSbounded non-empty containers, notwalk()calls. A 1.6 KB aliased document over a nested-itemsschema took 1.01 s and reportedbounded=False("I saw the whole thing"); it is 0.03 s now. Only the flag is gated on there being something to inspect; the visit is charged either way, which is the invarianttest_list_traversal_is_charged_against_the_walk_budgetwas written to hold. (The panel's 1e9 figure overstates it for this schema, which has no nesteditems— the vendored-schema worst case measures 1.71 s. The bound is still worth restoring:refresh-coderabbit-schema.ymlmoves the schema automatically and the walk is written to be schema-independent.)flatten_mappingrecurses before splicing, soa: &a {x: 1}/b: &b {<<: *a}/reviews: {<<: *b}putsxunderreviewsin the loaded document while the index got a literal"<<"— thestr()fallback with no line number, one level deeper._merge_entriesnow recurses, with a merged mapping's own keys ahead of what it merged in._descendcompared raw key text, so an ancestor arriving through a merge ended the walk at<<: the same defect one level up. The index now carries the value node and_descendroutes through it — which also stopson:/true:being read as a list index (a bool IS an int in Python).isliceruns a stage before ranking. Unranked FIFO overiter_errors, so it could in principle drop a file-rejecting error before_apply_capcan protect it. Not reachable against this schema — checked, and for a different reason than the panel gave:maxItemsdoes not stop element validation (150 badlinked_repositoriesreally do yield 150 errors), but any such document also tripsmaxItems: 20, which is itself file-rejecting. The real premise is the bound: only a literalfalseyields a stripped-keyword error, and every self-closing object is either unmultiplied or under amaxItems-bounded array (4 + 20 = 24). Now pinned astest_a_rejecting_error_cannot_be_crowded_out_before_ranking, which computes that bound from the schema._index_key's summary line named aNonesentinel the function never returns. Both corrected.Round 4 — the panel's findings on the round-3 fixes
All six threads addressed in
b88e7f2.<<:chain of ~1500 links is ~37 KB, far insideMAX_CONFIG_BYTES;safe_loadflattens it fine while this expansion, costing more frames per link, blew the recursion limit — andRecursionErroris not ayaml.YAMLError, so it escapedvalidateand killed an advisory check with a traceback. Bounded at_MAX_MERGE_DEPTH, degrading to the documentedstr()fallback. PyYAML's ownflatten_mappingcan lose the same race first (anchors in a lazily-constructed sequence, soreviewsflattens before them) — that path was already unguarded onmain, and now raisesConfigErrorand exits 2, like an oversized config: "I could not check this" must not read as "this is fine"._descendtyped the path PART instead of the node, so an integer mapping key went down the sequence branch and died there. Reachable today:reviews.mutually_exclusive_groupstakes arbitrary property names, so1:is legal —reviews.mutually_exclusive_groups[1]with no line becamereviews.mutually_exclusive_groups.1at line 4._format_pathis given the document too, so it renders an integer as[n]only when the container really is a sequence.construct_mappingassigns per pair, sosafe_loadkeeps the LAST. A duplicate key annotated its earlier twin's line; two<<:keys in one mapping resolved to the earlier one while the document kept the later. Explicit keys assign unconditionally now, and merge groups are consumed in reverse.maxItemsas a multiplicity bound, which it is not — jsonschema keeps validating elements past it. Reformulated to assert the four facts that actually hold, including thatmaxItemsprecedesitemsin key order so its rejecting error is emitted ahead of the element errors. Mutation-tested: all four clauses fail when broken; the old form was silent on all four._apply_cap's docstring claimed a document-order tie-break; the halves are concatenated, so it is emission order. Corrected, with why the walk's overflow finding is deliberately the first dropped._NAN_KEYslot. Distinguishing.nan:from.NaN:would need positional matching between the loaded dict and the composed node — the constructed float cannot be bridged by identity or by equality, which is why the sentinel exists. Cosmetic, on pathological input.Verification
99 tests, up from 72 — every case above is pinned by a test named for the defect, including the two High ones (
test_a_file_rejecting_error_is_never_crowded_out_by_unknown_keys,test_the_unknown_key_fan_out_stops_working_at_the_cap,test_the_key_index_is_built_once_per_mapping), the round-3 High (test_scalar_elements_are_charged_against_the_walk_budget) and the round-4 Mediums (test_a_merge_chain_the_loader_itself_cannot_build_exits_two,test_an_integer_mapping_key_is_not_read_as_a_list_index). Every behavioural round-3 and round-4 test was re-run against the pre-fix checker and fails there. The test workflow's CLI smoke steps were re-run verbatim, plus an end-to-end assertion that the flood-plus-maxLengthinput exits 1. Also green: agents-md-integrity (46), cursor-review (194), groom (365, 1 skipped), refresh-reviewers (48), workflow-pins (78 + lint).Refs BE-7786.
Provenance
jsonschema==4.26.0+PyYAML==6.0.3, on this branch level with currentmain(0 behind); the test workflow's 11 CLI smoke steps executed verbatim, all pass; every behavioural round-3 and round-4 test re-run against the pre-fix checker and confirmed failing there (5 of 5 for round 4); the reformulated schema-premise test mutation-tested against four broken-schema variants, all four caught; walk cost measured 1.01 s → 0.03 s on the nested-itemsshape, 512 KiB worst cases 1.74–1.83 s, 50k-key index build 0.03 s; the merge-chain recursion window measured (1500 links crash before the fix, 900 do not) and both crash paths — this file's and PyYAML's own — confirmed handled; agents-md-integrity 46 passed; cursor-review 194 passed; groom 365 passed (1 skipped); refresh-reviewers 48 passed; workflow-pins 78 passed + lint clean_resolved_key_index's merged-entries loop lives in_merge_entriesinstead — the single point that expands merges — somergedcannot carry a<<entry by construction rather than by an unreachable branch. (2)_apply_cap's emission-order tie-break is documented rather than changed: the walk's overflow finding exists to settruncated, and tagging walk findings with document positions would cost the stable per-half ordering the concatenation buys. (3) The NaN-key collapse is declined, not deferred — cosmetic on pathological input, below the follow-up bar. Separately:AGENTS.mdis 273 lines and over the checker's 200-line ceiling — pre-existing onmain, untouched here, and path-filtered so it does not gate this PR.