diff --git a/.github/assign-reviewers/README.md b/.github/assign-reviewers/README.md index 7967b4d2..df5f0426 100644 --- a/.github/assign-reviewers/README.md +++ b/.github/assign-reviewers/README.md @@ -78,7 +78,21 @@ front rather than only the ones a given run routes through, since the rot belong file and not to the diff — the sweep warns, it does not enqueue, so an unmatched rule's owners are still never candidates; a line whose only content is non-s-white whitespace is **no longer blank**, and since indentation counts spaces it reads as column 0 and terminates the -block above it, which inside `rules:` drops every later rule; and the `setKey` regex +block above it, which inside `rules:` drops every later rule — so a column-0 line that +**actually breaks something** is now reported by a warning on both ports, naming its line +number and rendering its content codepoint-escaped (`line 3 is not a recognised top-level +key (\u00a0)`), since nothing about that truncation is visible in an editor and a +misspelled key truncates identically. Two arms, and the *silence* between them is as +pinned as the text: a line that ENDED an open `default_pool:`/`rules:` block, and a +near miss that names a supported key without opening it (`\u0085rules:`, whose stray +character is not indentation; `rules:v2:`, a key YAML reads as `rules:v2`; and +`default_pool:[alice]`, which YAML reads as a plain scalar — the last two used to be +silently HONOURED as the supported key, which is why both ports now require s-white or +a line end after a key's colon). A `---`/`...` document marker, a `version:` key before +the first block and a stray key between two complete blocks break nothing and stay +silent, as does every INDENTED fallthrough line — one annotation per thing actually +broken, so a tab-indented or zero-indented config cannot flood GitHub's ~10-annotation +budget and bury the one that matters. Finally, the `setKey` regex spells its class out as `[^\n]*` rather than `.`, because Python's `.` excludes only LF while JS's also excludes CR, U+2028 and U+2029 — with `.` the Python port matched a rule line ending in a bare CR and the JS port did not, so the runtime dropped the key while diff --git a/.github/assign-reviewers/parser-corpus.json b/.github/assign-reviewers/parser-corpus.json index 3ead717e..97c26663 100644 --- a/.github/assign-reviewers/parser-corpus.json +++ b/.github/assign-reviewers/parser-corpus.json @@ -360,7 +360,7 @@ }, { "name": "a U+00A0-only line at column 0 ends a default_pool block on both ports", - "why": "narrowing the blank-line test to s-white means a line whose only content is non-s-white whitespace is no longer blank, and `_indent_of`/`indentOf` count SPACES only, so it reads as column 0 and terminates the block. Both ports moved together, so this is a behaviour change rather than a parity break — pinned because nothing else covers a whitespace-only line and the truncation is otherwise silent", + "why": "narrowing the blank-line test to s-white means a line whose only content is non-s-white whitespace is no longer blank, and `_indent_of`/`indentOf` count SPACES only, so it reads as column 0 and terminates the block. Both ports moved together, so this is a behaviour change rather than a parity break — pinned because nothing else covers a whitespace-only line. Both ports now also WARN on the terminating line, naming its number and rendering the character codepoint-escaped, so the truncation is no longer silent; the warning text is asserted per-suite (each port has its own channel), never here", "text": "default_pool:\n - alice\n\u00a0\n - bob\n", "expected": { "default_pool": ["alice"], @@ -369,7 +369,7 @@ }, { "name": "a U+00A0-only line at column 0 drops every later rule on both ports", - "why": "same mechanism as the default_pool case, but inside the `rules:` loop the `break` discards every rule AFTER the stray line, not just the tail of one list. Pinned so the blast radius of one invisible character is executable on both ports rather than discovered in a consumer repo", + "why": "same mechanism as the default_pool case, but inside the `rules:` loop the `break` discards every rule AFTER the stray line, not just the tail of one list. Pinned so the blast radius of one invisible character is executable on both ports rather than discovered in a consumer repo. Both ports now also WARN on the terminating line — one warning for the line, not one per discarded rule — asserted per-suite rather than here", "text": "rules:\n - paths: ['a/**']\n reviewers: [alice]\n\u00a0\n - paths: ['b/**']\n reviewers: [bob]\n", "expected": { "default_pool": [], @@ -380,12 +380,50 @@ }, { "name": "an INDENTED U+00A0-only line does not end the block", - "why": "the negative control for the two cases above: `_indent_of`/`indentOf` count the leading spaces, so a padded blank line that is INDENTED reads as column 2, never reaches the column-0 `break`, and is simply not a `- ` item. Bounds the truncation to a stray character at column 0 specifically", + "why": "the negative control for the two cases above: `_indent_of`/`indentOf` count the leading spaces, so a padded blank line that is INDENTED reads as column 2, never reaches the column-0 `break`, and is simply not a `- ` item. Bounds the truncation to a stray character at column 0 specifically — and, since the unrecognised-key warning fires only at column 0, it is equally the negative control for that: both ports stay SILENT here, which is what keeps the orphaned `- bob` tail below a real truncation from burying the one warning that matters", "text": "default_pool:\n - alice\n \u00a0\n - bob\n", "expected": { "default_pool": ["alice", "bob"], "rules": [] } + }, + { + "name": "an unknown top-level key at column 0 ends the block", + "why": "the key-SHAPED variant of the U+00A0 cases above, and the reason the warning is phrased around top-level keys rather than invisible characters: a plain misspelling or an unsupported key truncates the block exactly the same way, because only `default_pool:` and `rules:` are matched and everything else falls through to the same column-0 terminator. Pinned so a future port that starts skipping unknown keys instead of ending the block cannot land green", + "text": "default_pool:\n - alice\nowners:\n - bob\n", + "expected": { + "default_pool": ["alice"], + "rules": [] + } + }, + { + "name": "a key sharing a prefix with `rules:` is not the `rules:` key", + "why": "`rules:v2:` is valid YAML for a key NAMED `rules:v2` — `yaml.safe_load` returns `{'rules:v2': [...]}`, not a `rules` mapping — but a bare `startsWith('rules:')`/`startswith(\"rules:\")` claimed it and parsed its children as live routing rules, silently HONOURING an unsupported key. Both ports now require s-white or the end of the line after a supported key's colon, so this falls through to the unrecognised-key warning instead. Pinned on both ports because the prefix test was written twice", + "text": "rules:v2:\n - paths: ['a/**']\n reviewers: [alice]\n", + "expected": { + "default_pool": [], + "rules": [] + } + }, + { + "name": "no space after the colon is a plain scalar, not a key", + "why": "the other half of the same tightening, and the reason it is YAML-correct rather than merely convenient: `yaml.safe_load(\"default_pool:[alice]\")` returns the STRING `default_pool:[alice]`, because YAML reads `key:` as a mapping only with s-white or a line end after the colon. Both ports used to accept it as a flow pool; both now fall it through, and the warning names the exact rule so a caller who wrote it is not left guessing why the pool went empty", + "text": "default_pool:[alice]\n", + "expected": { + "default_pool": [], + "rules": [] + } + }, + { + "name": "YAML document markers are not unrecognised keys", + "why": "`---` and `...` fall through both ports (and `...` after a `rules:` block even terminates it), but nothing is lost — the outer loop keeps scanning and every block still parses. yamllint's default `document-start` rule REQUIRES the leading `---`, so a perfectly conformant reviewers.yml has one; warning about it would fire on every scheduled drift run of a well-formed config and train people to ignore the annotation. Pinned for the PARSE, with the silence asserted per-suite", + "text": "---\ndefault_pool:\n - alice\nrules:\n - paths: ['a/**']\n reviewers: [bob]\n...\n", + "expected": { + "default_pool": ["alice"], + "rules": [ + {"paths": ["a/**"], "reviewers": ["bob"]} + ] + } } ], "globs": [ diff --git a/.github/assign-reviewers/tests/assignment.test.cjs b/.github/assign-reviewers/tests/assignment.test.cjs index 4776b92d..9d5eba08 100644 --- a/.github/assign-reviewers/tests/assignment.test.cjs +++ b/.github/assign-reviewers/tests/assignment.test.cjs @@ -384,11 +384,48 @@ test('the shared corpus is loaded and non-empty', () => { assert(corpus.globs.length > 0, 'parser-corpus.json has no glob cases'); assert(corpus.globs.every(entry => entry.cases.length > 0), 'a corpus glob entry has no path cases'); }); +// Every corpus case that emits a warning, with its count — the same table test_generate.py +// carries for the Python channel, so a diagnostic that fires on one port and not the other is +// caught by the suites even though the warning TEXT cannot live in the shared fixture. +// The point of an EXHAUSTIVE table rather than per-case assertions is the silence: a regression +// that made the unrecognised-key warning fire on ordinary blank lines, comment lines or indented +// list items would warn on nearly every case here and nothing else would notice. +// Read it in both directions. The entries that are PRESENT and non-obvious are the second-BOM and +// leading-NEL cases: each is a document whose FIRST line is an unrecognised key precisely because +// the stray character is not indentation, which is the failure those two cases exist to pin, now +// with an annotation on it (they reach the near-miss arm, not the terminator arm — no block is +// open on line 1). The entries that are ABSENT matter just as much: `indented decoys and unknown +// top-level keys are ignored` is silent because a `version:`/`notes:` key before the first block +// ends nothing and drops nothing, and `YAML document markers are not unrecognised keys` is silent +// because `---`/`...` are syntax. Both used to warn, on documents whose expected parse is complete. +const CORPUS_WARNINGS = { + 'duplicate default_pool, flow then block': 1, + 'duplicate default_pool, block then flow': 1, + 'duplicate default_pool, second one empty': 1, + 'two leading BOMs: only one is stripped': 1, + 'U+0085 before a top-level key is not indentation': 1, + 'a U+00A0-only line at column 0 ends a default_pool block on both ports': 1, + 'a U+00A0-only line at column 0 drops every later rule on both ports': 1, + 'an unknown top-level key at column 0 ends the block': 1, + 'a key sharing a prefix with `rules:` is not the `rules:` key': 1, + 'no space after the colon is a plain scalar, not a key': 1, +}; for (const {name, text, expected} of corpus.configs) { test(`corpus config — ${name}`, () => { + helperWarnings.length = 0; assert.deepEqual(helpers.parseReviewerConfig(text), expected); + assert.equal(helperWarnings.length, CORPUS_WARNINGS[name] ?? 0, `warnings for ${JSON.stringify(name)}: ${JSON.stringify(helperWarnings)}`); }); } +// `CORPUS_WARNINGS[name] ?? 0` above only checks corpus case -> table, so a table entry whose +// corpus case was renamed or deleted would silently go dead and that case's count would stop being +// asserted while the suite stayed green. The Python side compares both directions in one +// `assertEqual(counts, self.CORPUS_WARNINGS)`; this is that other direction, so the two copies of +// the "same table" cannot drift apart through a rename. +test('every CORPUS_WARNINGS key names a corpus case that still exists', () => { + const names = new Set(corpus.configs.map(c => c.name)); + assert.deepEqual(Object.keys(CORPUS_WARNINGS).filter(n => !names.has(n)), []); +}); // The corpus compares parsed CONFIGS, which is deliberately silent about the duplicate-key warning // — the Python port prints its own `::warning::` line instead, so the text cannot live in the shared // fixture. Each side therefore asserts its own channel; last-wins itself stays corpus-pinned above. @@ -422,6 +459,165 @@ test('an empty first default_pool: still warns on the duplicate', () => { assert.deepEqual(helpers.parseReviewerConfig('default_pool: []\ndefault_pool: [bob]\n').default_pool, ['bob']); assert.equal(helperWarnings.length, 1); }); +// A column-0 line that is neither key ENDS the block above it, and since both ports narrowed to +// trimming s-white it need not look like a key: one U+00A0 is enough. The truncation is otherwise +// completely silent, so the parser names the line and renders the character codepoint-escaped. +// Parsed results stay corpus-pinned (three `U+00A0-only line` cases); the warning text cannot be, +// because each port emits on its own channel, so each suite asserts its own — as with duplicates. +test('a U+00A0-only line ending a default_pool block warns once, naming the line and the character', () => { + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('default_pool:\n - alice\n\u00a0\n - bob\n').default_pool, ['alice']); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /^line 3 is not a recognised top-level key \(\\u00a0\)/); + // The literal `U+00A0` in the message's own prose is what makes the character searchable for a + // reader who does not think in `\u` escapes; both are asserted so neither can quietly drop. + assert.match(helperWarnings[0], /U\+00A0/); +}); + +test('a U+00A0-only line dropping every later rule warns once, naming its line', () => { + helperWarnings.length = 0; + const out = helpers.parseReviewerConfig("rules:\n - paths: ['a/**']\n reviewers: [alice]\n\u00a0\n - paths: ['b/**']\n reviewers: [bob]\n"); + assert.equal(out.rules.length, 1); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /^line 4 is not a recognised top-level key \(\\u00a0\)/); +}); + +test('an INDENTED U+00A0-only line is silent — it truncates nothing', () => { + // The negative control, and the one that keeps the warning useful: the orphaned `- bob` tail + // below a real truncation also falls through, and warning per orphan would bury the terminator. + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('default_pool:\n - alice\n \u00a0\n - bob\n').default_pool, ['alice', 'bob']); + assert.deepEqual(helperWarnings, []); +}); + +test('an unknown top-level key at column 0 warns the same way, verbatim', () => { + // The key-shaped variant of the same truncation: nothing invisible, still silent before this. + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('default_pool:\n - alice\nowners:\n - bob\n').default_pool, ['alice']); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /^line 3 is not a recognised top-level key \(owners:\)/); +}); + +// --- what the warning deliberately stays SILENT about ------------------------------------- +// Warning on EVERY column-0 fallthrough both overclaimed and flooded, so these are the negative +// half of the diagnostic and each one is a shape a real config has. Mirrored in test_generate.py. + +test('YAML document markers are silent', () => { + // yamllint's default `document-start` rule REQUIRES the leading `---`, so a conformant + // reviewers.yml has one; `...` even TERMINATES the `rules:` block, so without the carve-out a + // well-formed config warns on every run. + helperWarnings.length = 0; + const out = helpers.parseReviewerConfig("---\ndefault_pool:\n - alice\nrules:\n - paths: ['a/**']\n reviewers: [bob]\n...\n"); + assert.deepEqual(out.default_pool, ['alice']); + assert.equal(out.rules.length, 1); + assert.deepEqual(helperWarnings, []); +}); + +test('a metadata key before any block is silent', () => { + // No block is open yet, so nothing is ended and nothing is dropped — the old message asserted + // both. The nested `default_pool:` decoy is still ignored. + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('version: 1\nnotes:\n default_pool: [mallory]\ndefault_pool: [alice]\n').default_pool, ['alice']); + assert.deepEqual(helperWarnings, []); +}); + +test('a zero-indented block sequence warns once, not once per item', () => { + // `default_pool:` followed by a COLUMN-0 sequence is valid YAML and common, and every item used + // to draw its own annotation — five here, ten in a real pool, against GitHub's ~10-per-step + // budget. Only the first ends the block; the rest end nothing, so the one warning that explains + // the empty pool is not buried. + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('default_pool:\n- alice\n- bob\n- carol\n- dave\n- eve\n').default_pool, []); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /^line 2 is not a recognised top-level key \(- alice\)/); +}); + +test('a non-YAML file is silent', () => { + // A `reviewer_config_path` aimed at prose opens no block, so it warns not at all rather than + // once per line. The empty parse is the diagnostic there. + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('Some prose here.\nAnother line.\nAnd another.\n'), {default_pool: [], rules: []}); + assert.deepEqual(helperWarnings, []); +}); + +// --- near misses: a line that NAMES a supported key without opening it ---------------------- + +test('an invisible prefix before a key warns as a near miss', () => { + // U+0085, not U+FEFF: a SINGLE leading BOM is legal YAML and the parser strips it, so a one-BOM + // document parses fine and must stay silent. The stray character is not indentation, so the key + // reads as column 0, falls through, and its whole block is never read — while the line looks + // perfect in an editor. No block was open, so the terminator arm would have stayed silent here. + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('\u0085default_pool: [alice]\nrules:\n').default_pool, []); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /^line 1 is not a recognised top-level key \(\\u0085default_pool: \[alice\]\) — it is not the supported `default_pool:` key/); + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('\ufeffdefault_pool: [alice]\n').default_pool, ['alice']); + assert.deepEqual(helperWarnings, []); +}); + +test('a key sharing a prefix with a supported one is not honoured, and warns', () => { + // `yaml.safe_load("rules:v2:\\n - paths: [a]")` is `{"rules:v2": [...]}` — a key NAMED + // `rules:v2`, not `rules`. The old `startsWith` claimed it and parsed its children as live + // routing rules, so an unsupported key was silently HONOURED. + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig("rules:v2:\n - paths: ['a/**']\n reviewers: [alice]\n").rules, []); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /it is not the supported `rules:` key/); +}); + +test('no space after the colon is a plain scalar, not a key', () => { + // `yaml.safe_load("default_pool:[alice]")` is the STRING `default_pool:[alice]`: YAML reads + // `key:` as a mapping only with s-white or a line end after the colon. + helperWarnings.length = 0; + assert.deepEqual(helpers.parseReviewerConfig('default_pool:[alice]\n').default_pool, []); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /it is not the supported `default_pool:` key/); +}); + +test('showInvisible escapes an astral character as a UTF-16 surrogate pair', () => { + // The regex has no `u` flag, so it walks code units — which is the REFERENCE the Python port's + // `_show_invisible` was corrected to match (a plain `"\\u%04x" % ord(c)` emitted `\u1f600` + // there, five digits and not a valid escape). Asserted so a later `u` flag here cannot silently + // reopen the divergence on the other side. + helperWarnings.length = 0; + helpers.parseReviewerConfig('default_pool:\n - alice\n\u{1f600}x\n - bob\n'); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /\(\\ud83d\\ude00x\)/); +}); + +test('the unrecognised-key warning carries the configured path when one is given', () => { + helperWarnings.length = 0; + helpers.parseReviewerConfig('default_pool:\n - alice\nowners:\n', '.github/owners.yml'); + assert.equal(helperWarnings.length, 1); + assert.match(helperWarnings[0], /^\.github\/owners\.yml: line 3 is not a recognised/); +}); + +// The two tests above run the EXTRACTED region, where `core` is the `helperCore` stub. These two +// run the WHOLE shipped script instead, which is the only thing that proves `showInvisible` resolves +// at BOTH of its call sites: it is a `const`, so it is in the temporal dead zone until its own line +// executes, and the parser is CALLED further up the script than the invalid-login warning that used +// to declare it. Declared at the wrong point, one of these two paths throws ReferenceError at run +// time while every region-scoped test above stays green. +test('the full shipped script emits the unrecognised-key warning, naming the configured path', async () => { + // The stray key lands INSIDE the `rules:` block, which is where the truncation actually bites: + // the rule below it is orphaned, so nothing matches `src/api/**` and routing falls back to the + // pool. (A stray key BETWEEN two top-level blocks truncates only its own orphaned tail — the + // outer loop keeps scanning and still finds a later `rules:`.) + const result = await run({config: 'default_pool: [generalist]\nrules:\nowners:\n - paths: ["src/api/**"]\n reviewers: [alice]\n'}); + assert.equal(result.logs.filter(m => /is not a recognised top-level key/.test(m)).length, 1); + assert.match(result.logs.find(m => /is not a recognised top-level key/.test(m)), /^\.github\/reviewers\.yml: line 3 .*\(owners:\)/); + assert.deepEqual(result.selected, ['generalist']); +}); + +test('the full shipped script still warns about an invalid configured login', async () => { + // Same helper, its OTHER caller. Pinned here because nothing else executed this path. + const result = await run({config: 'rules:\n - paths: ["src/api/**"]\n reviewers: ["\u00a0alice"]\n'}); + const warning = result.logs.find(m => /is not a valid GitHub login/.test(m)); + assert(warning, `no invalid-login warning in ${JSON.stringify(result.logs)}`); + assert.match(warning, /configured reviewer "\\u00a0alice"/); +}); + test('a single default_pool: is silent, however it is written', () => { helperWarnings.length = 0; for (const text of ['default_pool: [alice]\n', 'default_pool:\n - alice\n', 'rules:\n - reviewers: [a]\n', ' default_pool: [indented]\ndefault_pool: [alice]\n']) { diff --git a/.github/refresh-reviewers/generate.py b/.github/refresh-reviewers/generate.py index 803ba6a8..aae95753 100644 --- a/.github/refresh-reviewers/generate.py +++ b/.github/refresh-reviewers/generate.py @@ -275,7 +275,108 @@ def _indent_of(line): return len(line) - len(line.lstrip(" ")) -def parse_reviewer_config(text): +def _show_invisible(s): + r"""Render a token so a non-printing character in it is findable in a log. + + The JS port's `showInvisible` (assign-reviewers.yml, inside the + `PARITY-HARNESS:config-parser` region) character for character: everything + outside printable ASCII becomes a lower-case `\uXXXX` escape. Both ports + render the same escape for the same token so a reader comparing a + `::warning::` annotation from the drift generator against a `core.warning` + from the runtime sees one string, not two spellings of it. + + That includes ABOVE the BMP, which is why an astral code point is emitted as + a SURROGATE PAIR rather than as itself: JS's regex has no `u` flag, so it + walks UTF-16 code units and renders U+1F600 as `\ud83d\ude00`, while a plain + `"\\u%04x" % ord(c)` renders `\u1f600` — five digits, which is not a valid + `\u` escape at all and reads ambiguously as `\u1f60` followed by `0`. The + characters this exists for (U+00A0, U+FEFF, U+0085, U+2028) are all BMP, but + the helper escapes the WHOLE token, so an emoji anywhere in a stray line or a + configured login used to render differently on the two ports. + """ + out = [] + for c in s: + cp = ord(c) + if " " <= c <= "~": + out.append(c) + elif cp > 0xFFFF: + v = cp - 0x10000 + out.append("\\u%04x\\u%04x" % (0xD800 + (v >> 10), 0xDC00 + (v & 0x3FF))) + else: + out.append("\\u%04x" % cp) + return "".join(out) + + +def _wc_escape(s): + """Escape a message for a `::warning::` workflow command. + + The runner decodes `%25`/`%0D`/`%0A` out of a workflow command's message and + treats a raw newline as the end of the command, so three things could mangle + an annotation here: a `%` in a config token (`_show_invisible` leaves it + alone — it is printable ASCII), and a CR or LF in `source_path`, which comes + from the caller-configurable `reviewer_config_path` and is interpolated raw. + A newline there splits the physical stdout line and can start a SECOND + runner command. `core.warning` escapes exactly these three for the JS port, + so doing it here is what keeps the two annotations one string rather than + two spellings of it. `%` first, or the escapes escape each other. + """ + return s.replace("%", "%25").replace("\r", "%0D").replace("\n", "%0A") + + +def _is_key(line, key): + """True when an s-white-trimmed `line` opens the top-level block `key`. + + A bare `startswith` also claims a DIFFERENT key that merely shares the + prefix: `rules:v2:` is valid YAML for a key named `rules:v2`, and the prefix + test parsed its children as live routing rules — silently HONOURING an + unsupported key instead of falling through to the warning below. YAML reads + `key:` as a mapping only when s-white or the end of the line follows the + colon, so require that; `default_pool:[alice]` is a plain scalar to YAML and + is now a fallthrough here too. The JS port's `isKey` is the same test. + """ + return line.startswith(key) and line[len(key):len(key) + 1] in ("", " ", "\t") + + +def _near_miss_key(line): + """The supported key a fallthrough `line` names but does not open. + + Two shapes reach this, and both are silent breakage worth an annotation: + + * `\ufeffdefault_pool:`, `\u0085rules:` — the stray character is not + indentation (neither `_trim` nor `_indent_of` sees it), so the line reads + as column 0, falls through, and the block it meant to open is never read. + Stripping the leading non-printable-ASCII run recognises the intent. + * `rules:v2:`, `default_pool:[alice]` — a DIFFERENT key that merely shares + the prefix, or a colon YAML would not read as a mapping. `_is_key` + (rightly) declines these; without this they would fall through in + silence, which is how `rules:v2:` used to be honoured as live routing. + + Only called from the fallthrough, where `_is_key(line, key)` is already + known false, so a bare prefix match here always means one of the two. The + JS port's `nearMissKey` is identical. + """ + head = re.sub(r"^[^\x21-\x7e]+", "", line) + for key in ("default_pool:", "rules:"): + if head.startswith(key): + return key + return None + + +def _is_doc_marker(line): + """True for a YAML document marker, which is syntax rather than a key. + + yamllint's default `document-start` rule REQUIRES a leading `---`, so a + perfectly conformant reviewers.yml opens with one (and may close with + `...`). Both fall through this parser, and `...` after a `rules:` block even + terminates it — but nothing is lost, so warning about them would fire on + every scheduled run of a well-formed config and train people to ignore the + annotation. A SECOND document in the file is a separate, pre-existing + limitation (its keys merge into the first); this only silences the marker. + """ + return line in ("---", "...") or line.startswith("--- ") or line.startswith("---\t") + + +def parse_reviewer_config(text, source_path=None): """Return (config, locations). config = {"default_pool": [...], "rules": [{"paths": [...], @@ -301,12 +402,24 @@ def parse_reviewer_config(text): # data; that would have traded the old divergence for a new one. Splitting on # the two-character break produces the same number of lines as `split("\n")` # on a CRLF document, so every line index is left alone. + # `source_path` only ever NAMES the file in a diagnostic, and is optional for + # the same reason the JS port's is: the unit tests (and the shared corpus) hand + # the parser text with no path, and a missing one must drop the prefix rather + # than print a literal `None:` or hardcode a `reviewers.yml` the caller may + # have configured away via `reviewer_config_path`. + where = f"{source_path}: " if source_path else "" raw_lines = re.split(r"\r?\n", text.removeprefix("\ufeff")) lines = [_strip_comment(l) for l in raw_lines] config = {"default_pool": [], "rules": []} locs = {"default_pool": None, "rules": [], "rule_paths": []} seen_default_pool = False + # The block a column-0 line has just TERMINATED, or None. Both scans below + # skip blank lines and `break` only on a non-blank column-0 line WITHOUT + # advancing `i`, so the very next iteration of this loop is guaranteed to be + # the terminating line itself — which is what makes a one-iteration handoff + # sound rather than a flag that could go stale. Every branch consumes it. + ended_block = None i = 0 n = len(lines) while i < n: @@ -315,7 +428,8 @@ def parse_reviewer_config(text): if not line: i += 1 continue - if _indent_of(raw) == 0 and line.startswith("default_pool:"): + terminated, ended_block = ended_block, None + if _indent_of(raw) == 0 and _is_key(line, "default_pool:"): # YAML requires unique keys; PyYAML's de-facto last-wins is the # reference. Warn, never reject — the drift generator must not # hard-fail on a malformed map. Both the flow and the block arm below @@ -323,7 +437,8 @@ def parse_reviewer_config(text): # further work here; each arm also REPOINTS `locs["default_pool"]`, # so the rewrite can never target an occurrence the parse discarded. if seen_default_pool: - print("::warning::duplicate top-level default_pool: key — last one wins") + print("::warning::" + _wc_escape( + f"{where}duplicate top-level default_pool: key — last one wins")) seen_default_pool = True rest = _trim(line[len("default_pool:"):]) flow = _parse_flow(rest) @@ -345,6 +460,7 @@ def parse_reviewer_config(text): i += 1 continue if _indent_of(r) == 0: + ended_block = "default_pool:" break t = _trim(r) if t.startswith("- "): @@ -362,7 +478,7 @@ def parse_reviewer_config(text): # means "no rewritable list here", which `rewrite_config` skips. locs["default_pool"] = ("block", item_lines, indent) if item_lines else None continue - if _indent_of(raw) == 0 and line.startswith("rules:"): + if _indent_of(raw) == 0 and _is_key(line, "rules:"): i += 1 current = None cur_loc = None @@ -419,6 +535,7 @@ def set_key(seg, line_idx): i += 1 continue if _indent_of(r) == 0: + ended_block = "rules:" break ind = _indent_of(r) t = _trim(r) @@ -445,6 +562,54 @@ def set_key(seg, line_idx): set_key(t, i) i += 1 continue + # Fallthrough: a non-empty line that is neither `default_pool:` nor `rules:`. + # At column 0 such a line can do real, INVISIBLE damage, and the two shapes it + # takes are the two arms below. It need not look like a key at all: since both + # ports narrowed to trimming s-white, a line whose only content is ONE non-s-white + # whitespace character (U+00A0, U+0085, U+000B, U+000C, U+001C-U+001F, U+FEFF, + # U+2028, U+2029, U+200B) is no longer blank, and `_indent_of` counts SPACES, so it + # reads as column 0. Real YAML rejects most such documents outright (PyYAML raises + # on NBSP and FF), so widening the blank test would have diverged from YAML; making + # the damage visible does not change what parses. Warn, never reject — the drift + # generator must not hard-fail on a malformed map, the same posture as the + # duplicate-key warning above. + # + # What this deliberately does NOT warn about is an unrecognised column-0 line that + # broke nothing, because warning on EVERY fallthrough both overclaimed and flooded: + # a leading `---` or trailing `...` document marker (valid YAML, and what yamllint's + # default `document-start` rule REQUIRES), a `version:`/`notes:` metadata key before + # the first block, and a stray key between two complete blocks all leave the outer + # loop scanning and every later `default_pool:`/`rules:` parsing — yet each drew an + # annotation asserting the block above was ended and the rules below were dropped, + # on a document whose parse is complete. Silence there is also what caps the volume: + # a zero-indented block sequence, a tab-indented config (`_indent_of` counts spaces, + # so EVERY item reads as column 0) and a `reviewer_config_path` aimed at a non-YAML + # file each used to warn once per line and bury the annotation that mattered inside + # GitHub's ~10-per-step budget; now they warn once per block actually truncated. + # + # INDENTED fallthrough lines stay silent for the same reason: they are the orphaned + # tail of an already-reported truncation (` - bob` after the stray line), and one + # warning per terminator is the signal — one per orphaned item would bury it. + if _indent_of(raw) == 0 and not _is_doc_marker(line): + near = _near_miss_key(line) + if near: + # Checked BEFORE the terminator arm because it names the actual cause and + # the larger blast radius: `\ufeffrules:` may also end a (complete) + # `default_pool:` above it, but what the reader needs to know is that every + # rule below is gone because of one character they cannot see. + print("::warning::" + _wc_escape( + f"{where}line {i + 1} is not a recognised top-level key " + f"({_show_invisible(line)}) — it is not the supported `{near}` key, so " + "the whole block it opens is ignored; that key is read only at column 0 " + "with nothing but spaces before it and a space, a tab or the end of the " + "line after its colon")) + elif terminated: + print("::warning::" + _wc_escape( + f"{where}line {i + 1} is not a recognised top-level key " + f"({_show_invisible(line)}) — it ends the `{terminated}` block above " + "it, and every list item or rule indented below it is dropped; only " + "`default_pool:` and `rules:` are read, and at column 0 one invisible " + "character (e.g. U+00A0) ends a block exactly like a misspelled key does")) i += 1 # locs["rules"] holds dicts internally; expose the reviewers loc as # locs["rules"] and the (skip-only) paths loc alongside it. @@ -1113,7 +1278,7 @@ def main(): return _noop_exit( f"{config_path} on origin/{branch} uses CR-only line endings, " "which the byte-faithful reader does not split on") - config, locs = parse_reviewer_config(committed_text) + config, locs = parse_reviewer_config(committed_text, config_path) skipped_unterminated = [{"key": key, "line": idx + 1} for key, idx in unterminated_locations(locs)] for entry in skipped_unterminated: diff --git a/.github/refresh-reviewers/tests/test_generate.py b/.github/refresh-reviewers/tests/test_generate.py index 4183eb63..e1b89791 100644 --- a/.github/refresh-reviewers/tests/test_generate.py +++ b/.github/refresh-reviewers/tests/test_generate.py @@ -507,12 +507,38 @@ def test_bom_flow_default_pool_rewrite_keeps_the_bom(self): self.assertEqual(out.split("\n")[1:], cfg.split("\n")[1:]) -class TestDuplicateDefaultPool(unittest.TestCase): - """Last-wins plus a warning, never a rejection. +class TestShowInvisible(unittest.TestCase): + """`_show_invisible` has to render a token the way the JS `showInvisible` does. - The parsed result is pinned language-neutrally by the shared corpus; the - warning cannot be, because each port emits on its own channel (`::warning::` - here, `core.warning` in the JS step), so each side asserts its own. + Both warning channels quote the offending token, and a reader comparing a + `::warning::` here against a `core.warning` there must see one string. + """ + + def test_bmp_characters_escape_to_four_lower_case_hex_digits(self): + self.assertEqual(gen._show_invisible("a\u00a0b\ufeff"), "a\\u00a0b\\ufeff") + + def test_printable_ascii_is_left_alone(self): + self.assertEqual(gen._show_invisible(" !~%"), " !~%") + + def test_astral_characters_escape_as_a_surrogate_pair(self): + # JS's regex has no `u` flag, so it walks UTF-16 code units and renders U+1F600 + # as `\ud83d\ude00`. A plain `"\\u%04x" % ord(c)` renders `\u1f600` — five + # digits, which is not a valid `\u` escape at all and reads ambiguously as + # `\u1f60` followed by `0`. Emoji reach this via a stray line or a configured + # login, both of which the helper escapes whole. + self.assertEqual(gen._show_invisible("\U0001f600"), "\\ud83d\\ude00") + self.assertEqual(gen._show_invisible("\U0010ffff"), "\\udbff\\udfff") + + +class TestParserWarnings(unittest.TestCase): + """Both parser diagnostics: a duplicate `default_pool:` (last-wins) and a + column-0 line that is neither key (it ends the block above it). Warnings, + never rejections — a drift generator must not hard-fail on a malformed map. + + Every parsed result here is pinned language-neutrally by the shared corpus; + the warnings cannot be, because each port emits on its own channel + (`::warning::` here, `core.warning` in the JS step), so each side asserts + its own. assignment.test.cjs mirrors each case below. """ def parse(self, text): @@ -527,6 +553,168 @@ def test_duplicate_warns_once_and_last_wins(self): self.assertEqual(out.count("::warning::"), 1) self.assertIn("duplicate top-level default_pool: key", out) + def test_nbsp_line_ending_default_pool_warns_once(self): + # U+00A0 stays ESCAPED here, as everywhere in this repo: a literal would be + # invisible in review, and an editor or lint that normalised it away would + # make this test silently vacuous rather than red. + config, _locs, out = self.parse("default_pool:\n - alice\n\u00a0\n - bob\n") + self.assertEqual(config["default_pool"], ["alice"]) + self.assertEqual(out.count("::warning::"), 1) + self.assertIn("line 3 is not a recognised top-level key (\\u00a0)", out) + # The literal `U+00A0` in the message's own prose is what makes the character + # searchable for a reader who does not think in `\u` escapes. + self.assertIn("U+00A0", out) + + def test_nbsp_line_ending_rules_warns_once(self): + config, _locs, out = self.parse( + "rules:\n - paths: ['a/**']\n reviewers: [alice]\n" + "\u00a0\n - paths: ['b/**']\n reviewers: [bob]\n") + self.assertEqual(len(config["rules"]), 1) + self.assertEqual(out.count("::warning::"), 1) + self.assertIn("line 4 is not a recognised top-level key (\\u00a0)", out) + + def test_indented_nbsp_line_is_silent(self): + # The negative control, and the one that keeps the warning useful: the orphaned + # `- bob` tail below a REAL truncation falls through the same way, and warning + # once per orphan would bury the single terminator that explains them all. + config, _locs, out = self.parse("default_pool:\n - alice\n \u00a0\n - bob\n") + self.assertEqual(config["default_pool"], ["alice", "bob"]) + self.assertNotIn("::warning::", out) + + def test_unknown_top_level_key_warns_once(self): + # The key-SHAPED variant: nothing invisible, same silent truncation before this. + config, _locs, out = self.parse("default_pool:\n - alice\nowners:\n - bob\n") + self.assertEqual(config["default_pool"], ["alice"]) + self.assertEqual(out.count("::warning::"), 1) + self.assertIn("line 3 is not a recognised top-level key (owners:)", out) + + def test_warnings_name_the_configured_path_when_given(self): + # `reviewer_config_path` is a caller input, so neither warning may hardcode + # `reviewers.yml` — a caller that configured another name would be sent to look + # at a file its repo does not have. Without a path the prefix is simply absent, + # never a literal `None:`; the JS port asserts the same two shapes. + # The second `default_pool:` is the BLOCK form on purpose: `owners:` only warns + # because it terminates an open block, so the flow form would make this test + # assert one warning while claiming to assert two. + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + gen.parse_reviewer_config( + "default_pool: [alice]\ndefault_pool:\n - bob\nowners:\n", + ".github/owners.yml") + lines = [l for l in buf.getvalue().splitlines() if l] + self.assertEqual(len(lines), 2) + for line in lines: + self.assertTrue(line.startswith("::warning::.github/owners.yml: "), line) + self.assertNotIn("None", self.parse("default_pool:\n - alice\nowners:\n - bob\n")[2]) + + # --- what the warning deliberately stays SILENT about ------------------------- + # Warning on EVERY column-0 fallthrough both overclaimed and flooded, so these six + # are the negative half of the diagnostic and each one is a shape a real config has. + + def test_document_markers_are_silent(self): + # yamllint's default `document-start` rule REQUIRES the leading `---`, so a + # conformant reviewers.yml has one; `...` even TERMINATES the `rules:` block, so + # without the carve-out a well-formed config warns on every scheduled run. + config, _locs, out = self.parse( + "---\ndefault_pool:\n - alice\nrules:\n - paths: ['a/**']\n" + " reviewers: [bob]\n...\n") + self.assertEqual(config["default_pool"], ["alice"]) + self.assertEqual(len(config["rules"]), 1) + self.assertNotIn("::warning::", out) + + def test_metadata_key_before_any_block_is_silent(self): + # No block is open yet, so nothing is ended and nothing is dropped — the old + # message asserted both. The nested `default_pool:` decoy is still ignored. + config, _locs, out = self.parse( + "version: 1\nnotes:\n default_pool: [mallory]\ndefault_pool: [alice]\n") + self.assertEqual(config["default_pool"], ["alice"]) + self.assertNotIn("::warning::", out) + + def test_zero_indented_block_sequence_warns_once_not_per_item(self): + # `default_pool:` followed by a COLUMN-0 sequence is valid YAML and common, and + # every item used to draw its own annotation — five here, ten in a real pool, + # against GitHub's ~10-per-step budget. Only the first ends the block; the rest + # end nothing, so they are silent and the one warning that explains the empty + # pool is not buried. + config, _locs, out = self.parse( + "default_pool:\n- alice\n- bob\n- carol\n- dave\n- eve\n") + self.assertEqual(config["default_pool"], []) + self.assertEqual(out.count("::warning::"), 1) + self.assertIn("line 2 is not a recognised top-level key (- alice)", out) + + def test_non_yaml_file_is_silent(self): + # A `reviewer_config_path` aimed at prose opens no block, so it warns not at all + # rather than once per line. The empty parse is the diagnostic there. + config, _locs, out = self.parse( + "Some prose here.\nAnother line.\nAnd another.\nYet more.\n") + self.assertEqual(config, {"default_pool": [], "rules": []}) + self.assertNotIn("::warning::", out) + + # --- near misses: a line that NAMES a supported key without opening it --------- + + def test_invisible_prefix_before_a_key_warns_as_a_near_miss(self): + # The stray character is not indentation, so the key reads as column 0, falls + # through, and its whole block is never read — while the line looks perfect in an + # editor. No block was open, so the terminator arm would have stayed silent here; + # this arm is what keeps the two BOM/NEL corpus cases annotated. + # U+0085, not U+FEFF: a SINGLE leading BOM is legal YAML and the parser strips + # it, so a one-BOM document parses fine and must stay silent. It takes a second + # BOM (the corpus case) or a character nothing strips to reach this arm. + config, _locs, out = self.parse("\u0085default_pool: [alice]\nrules:\n") + self.assertEqual(config["default_pool"], []) + self.assertEqual(out.count("::warning::"), 1) + self.assertIn( + "line 1 is not a recognised top-level key (\\u0085default_pool: [alice]) " + "— it is not the supported `default_pool:` key", out) + # The negative control the line above depends on. + self.assertNotIn("::warning::", self.parse("\ufeffdefault_pool: [alice]\n")[2]) + + def test_prefix_sharing_key_is_not_honoured_and_warns(self): + # `yaml.safe_load("rules:v2:\\n - paths: [a]")` is `{"rules:v2": [...]}` — a key + # NAMED `rules:v2`, not `rules`. The old `startswith` claimed it and parsed its + # children as live routing rules, so an unsupported key was silently HONOURED. + config, _locs, out = self.parse("rules:v2:\n - paths: ['a/**']\n reviewers: [alice]\n") + self.assertEqual(config["rules"], []) + self.assertEqual(out.count("::warning::"), 1) + self.assertIn("it is not the supported `rules:` key", out) + + def test_no_space_after_the_colon_is_not_a_key(self): + # `yaml.safe_load("default_pool:[alice]")` is the STRING `default_pool:[alice]`: + # YAML reads `key:` as a mapping only with s-white or a line end after the colon. + config, _locs, out = self.parse("default_pool:[alice]\n") + self.assertEqual(config["default_pool"], []) + self.assertEqual(out.count("::warning::"), 1) + self.assertIn("it is not the supported `default_pool:` key", out) + + # --- the annotation has to survive the runner --------------------------------- + + def test_workflow_command_characters_are_escaped(self): + # `::warning::` is built by hand here, and the runner DECODES `%25`/`%0D`/`%0A` + # out of the message: an unescaped `%` in a config token (printable ASCII, so + # `_show_invisible` leaves it) is decoded into a mangled multi-line annotation, + # and a newline in the caller-configured path can start a SECOND runner command. + # `core.warning` does exactly this for the JS port, so this is also the parity. + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + gen.parse_reviewer_config( + "default_pool:\n - alice\n100%0Adone\n - bob\n", "a\nb.yml") + out = buf.getvalue() + self.assertEqual(len(out.splitlines()), 1) + self.assertIn("a%0Ab.yml: ", out) + self.assertIn("(100%250Adone)", out) + + def test_live_consumer_config_shape_is_silent(self): + # The acceptance shape every live consumer config has: only `default_pool:`, + # `rules:`, comments and blank lines at column 0. A warning here would fire on + # every scheduled drift run and train people to ignore the annotation. + _config, _locs, out = self.parse( + "# a leading comment\n\n" + "default_pool:\n - alice\n - bob\n\n" + "# a comment between blocks\n\n" + "rules:\n - paths: ['a/**']\n reviewers: [alice]\n\n" + " # an indented comment\n - paths: ['b/**']\n reviewers: [bob]\n") + self.assertNotIn("::warning::", out) + def test_empty_first_list_still_warns(self): # Keyed on "the key was seen", not on "the list is non-empty" — an # emptiness test would make this port silent where the JS one warns. @@ -1338,11 +1526,51 @@ def test_config_cases(self): with self.subTest(case["name"]): # stdout is swallowed only so the duplicate-key cases do not # emit `::warning::` annotations from a passing test job; the - # warning itself is asserted by TestDuplicateDefaultPool. + # warning itself is asserted by TestParserWarnings. with contextlib.redirect_stdout(io.StringIO()): config, _locations = gen.parse_reviewer_config(case["text"]) self.assertEqual(config, case["expected"]) + # Every corpus case that emits a `::warning::`, with its count. The point of an + # EXHAUSTIVE table rather than per-case assertions is the silence: a regression + # that made the unrecognised-key warning fire on ordinary blank lines, comment + # lines or indented list items would warn on nearly every case here and nothing + # else in either suite would notice. The JS port carries the same table. + # + # Read it in both directions. The entries that are PRESENT and non-obvious are the + # second-BOM and leading-NEL cases: each is a document whose FIRST line is an + # unrecognised key precisely because the stray character is not indentation, which + # is the failure those two cases exist to pin, now with an annotation on it (they + # reach the near-miss arm, not the terminator arm — no block is open on line 1). + # The entries that are ABSENT matter just as much: `indented decoys and unknown + # top-level keys are ignored` is silent because a `version:`/`notes:` key before + # the first block ends nothing and drops nothing, and `YAML document markers are + # not unrecognised keys` is silent because `---`/`...` are syntax. Both used to + # warn, on documents whose expected parse right here is complete. + CORPUS_WARNINGS = { + "duplicate default_pool, flow then block": 1, + "duplicate default_pool, block then flow": 1, + "duplicate default_pool, second one empty": 1, + "two leading BOMs: only one is stripped": 1, + "U+0085 before a top-level key is not indentation": 1, + "a U+00A0-only line at column 0 ends a default_pool block on both ports": 1, + "a U+00A0-only line at column 0 drops every later rule on both ports": 1, + "an unknown top-level key at column 0 ends the block": 1, + "a key sharing a prefix with `rules:` is not the `rules:` key": 1, + "no space after the colon is a plain scalar, not a key": 1, + } + + def test_config_case_warnings_are_exactly_the_expected_set(self): + counts = {} + for case in self.corpus["configs"]: + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + gen.parse_reviewer_config(case["text"]) + n = buf.getvalue().count("::warning::") + if n: + counts[case["name"]] = n + self.assertEqual(counts, self.CORPUS_WARNINGS) + def test_glob_cases(self): for entry in self.corpus["globs"]: compiled = gen.glob_to_regexp(entry["glob"]) diff --git a/.github/workflows/assign-reviewers.yml b/.github/workflows/assign-reviewers.yml index ef159885..5f0b29cc 100644 --- a/.github/workflows/assign-reviewers.yml +++ b/.github/workflows/assign-reviewers.yml @@ -457,6 +457,16 @@ jobs: // js-yaml is not bundled with github-script; the schema is small and // fixed, so a focused parser is safer than an unavailable dependency. // PARITY-HARNESS:config-parser:begin + // Render a token so a non-printing character in it is findable in a log: everything + // outside printable ASCII becomes a lower-case `\uXXXX` escape. Two callers — the + // unrecognised-top-level-key warning below, and the invalid-configured-login warning + // further down the script — so it is declared ONCE, and it lives INSIDE this region + // because the parity harness executes the region on its own: a helper the parser calls + // but the extracted span does not contain is a ReferenceError in the tests, and a + // `const` declared BELOW the parser's own call site is one in the shipped script. + // `_show_invisible` in refresh-reviewers/generate.py is the port, so both sides render + // the same character the same way on their own warning channel. + const showInvisible = (s) => s.replace(/[^\x20-\x7e]/g, (c) => '\\u' + c.codePointAt(0).toString(16).padStart(4, '0')); // `sourcePath` is only ever used to NAME the file in a diagnostic, and is // optional because the parity harness extracts this region on its own and // has no path to hand it — an omitted one just drops the prefix rather than @@ -507,6 +517,33 @@ jobs: return inner.split(',').map((x) => unquote(x)).filter((x) => x.length > 0); }; const indentOf = (l) => l.match(/^ */)[0].length; + // A bare `startsWith` also claims a DIFFERENT key that merely shares the prefix: + // `rules:v2:` is valid YAML for a key named `rules:v2`, and the prefix test parsed + // its children as live routing rules — silently HONOURING an unsupported key + // instead of falling through to the warning below. YAML reads `key:` as a mapping + // only when s-white or the end of the line follows the colon, so require that; + // `default_pool:[alice]` is a plain scalar to YAML and is a fallthrough here too. + // `_is_key` in refresh-reviewers/generate.py is the port. + const isKey = (l, key) => l.startsWith(key) && (l.length === key.length || l[key.length] === ' ' || l[key.length] === '\t'); + // The supported key a FALLTHROUGH line names but does not open. Two shapes reach + // this, both silent breakage worth an annotation: `\ufeffdefault_pool:`/`\u0085rules:`, + // where the stray character is not indentation (neither `trimSWhite` nor `indentOf` + // sees it) so the line reads as column 0 and the block it meant to open is never + // read; and `rules:v2:`/`default_pool:[alice]`, which `isKey` rightly declines. + // Only called where `isKey` is already known false, so a bare prefix match on the + // padding-stripped head always means one of the two. Ported as `_near_miss_key`. + const nearMissKey = (l) => { + const head = l.replace(/^[^\x21-\x7e]+/, ''); + for (const key of ['default_pool:', 'rules:']) if (head.startsWith(key)) return key; + return null; + }; + // A YAML document marker is syntax, not a key. yamllint's default `document-start` + // rule REQUIRES a leading `---`, so a perfectly conformant reviewers.yml opens with + // one (and may close with `...`); both fall through here, and `...` after a `rules:` + // block even terminates it — but nothing is lost, so warning would fire on every run + // of a well-formed config and train people to ignore the annotation. A SECOND + // document in the file is a separate pre-existing limitation, untouched by this. + const isDocMarker = (l) => l === '---' || l === '...' || l.startsWith('--- ') || l.startsWith('---\t'); // A single leading U+FEFF is legal YAML and must not become part of the first // key. `trim()` used to absorb it here incidentally and the Python port's // `strip()` did not; now that both sides trim s-white only, NEITHER absorbs it @@ -515,11 +552,19 @@ jobs: const lines = text.replace(/^\uFEFF/, '').split(/\r?\n/).map(stripComment); let i = 0; let seenDefaultPool = false; + // The block a column-0 line has just TERMINATED, or null. Both scans below skip + // blank lines and `break` only on a non-blank column-0 line WITHOUT advancing `i`, + // so the very next iteration of this loop is guaranteed to be the terminating line + // itself — which is what makes a one-iteration handoff sound rather than a flag that + // could go stale. Every branch consumes it. Ported as `ended_block`. + let endedBlock = null; while (i < lines.length) { const raw = lines[i]; if (!trimSWhite(raw)) { i++; continue; } const line = trimSWhite(raw); - if (indentOf(raw) === 0 && line.startsWith('default_pool:')) { + const terminated = endedBlock; + endedBlock = null; + if (indentOf(raw) === 0 && isKey(line, 'default_pool:')) { // YAML requires unique keys; PyYAML's de-facto last-wins is the reference. // Warn, never reject — routing must not hard-fail on a malformed map — and // reset rather than accumulate, so the block form replaces like the flow @@ -535,14 +580,14 @@ jobs: while (i < lines.length) { const r = lines[i]; if (!trimSWhite(r)) { i++; continue; } - if (indentOf(r) === 0) break; + if (indentOf(r) === 0) { endedBlock = 'default_pool:'; break; } const t = trimSWhite(r); if (t.startsWith('- ')) out.default_pool.push(unquote(t.slice(2))); i++; } continue; } - if (indentOf(raw) === 0 && line.startsWith('rules:')) { + if (indentOf(raw) === 0 && isKey(line, 'rules:')) { i++; let current = null, listKey = null, ruleIndent = -1; const setKey = (seg) => { @@ -568,7 +613,7 @@ jobs: while (i < lines.length) { const r = lines[i]; if (!trimSWhite(r)) { i++; continue; } - if (indentOf(r) === 0) break; + if (indentOf(r) === 0) { endedBlock = 'rules:'; break; } const ind = indentOf(r); const t = trimSWhite(r); const isDash = t === '-' || t.startsWith('- '); @@ -588,6 +633,45 @@ jobs: } continue; } + // Fallthrough: a non-empty line that is neither `default_pool:` nor `rules:`. + // At column 0 such a line can do real, INVISIBLE damage, and the two shapes it + // takes are the two arms below. It need not look like a key at all: since both + // ports narrowed to trimming s-white, a line whose only content is ONE non-s-white + // whitespace character (U+00A0, U+0085, U+000B, U+000C, U+001C-U+001F, U+FEFF, + // U+2028, U+2029, U+200B) is no longer blank, and `indentOf` counts SPACES, so it + // reads as column 0. Real YAML rejects most such documents outright, so widening + // the blank test would have diverged from YAML; making the damage visible does not + // change what parses. Warn, never reject — routing must not hard-fail on a + // malformed map, the same posture as the duplicate-key warning above. + // + // What this deliberately does NOT warn about is an unrecognised column-0 line that + // broke nothing, because warning on EVERY fallthrough both overclaimed and flooded: + // a `---`/`...` document marker, a `version:`/`notes:` metadata key before the + // first block, and a stray key between two complete blocks all leave the outer loop + // scanning and every later `default_pool:`/`rules:` parsing — yet each drew an + // annotation asserting the block above was ended and the rules below were dropped, + // on a document whose parse is complete. Silence there is also what caps the + // volume: a zero-indented block sequence, a tab-indented config (`indentOf` counts + // spaces, so EVERY item reads as column 0) and a `reviewer_config_path` aimed at a + // non-YAML file each used to warn once per line and bury the annotation that + // mattered inside GitHub's ~10-per-step budget; now they warn once per block + // actually truncated, or not at all. + // + // INDENTED fallthrough lines stay silent for the same reason: they are the orphaned + // tail of an already-reported truncation, and one warning per terminator is the + // signal — one per orphaned item would bury it. + if (indentOf(raw) === 0 && !isDocMarker(line)) { + const near = nearMissKey(line); + if (near) { + // Checked BEFORE the terminator arm because it names the actual cause and the + // larger blast radius: `\ufeffrules:` may also end a (complete) `default_pool:` + // above it, but what the reader needs to know is that every rule below is gone + // because of one character they cannot see. + core.warning(`${where}line ${i + 1} is not a recognised top-level key (${showInvisible(line)}) — it is not the supported \`${near}\` key, so the whole block it opens is ignored; that key is read only at column 0 with nothing but spaces before it and a space, a tab or the end of the line after its colon`); + } else if (terminated) { + core.warning(`${where}line ${i + 1} is not a recognised top-level key (${showInvisible(line)}) — it ends the \`${terminated}\` block above it, and every list item or rule indented below it is dropped; only \`default_pool:\` and \`rules:\` are read, and at column 0 one invisible character (e.g. U+00A0) ends a block exactly like a misspelled key does`); + } + } i++; } return out; @@ -654,11 +738,12 @@ jobs: // set legitimately holds the PR author and `EXCLUDE`, and warning about a // correctly-configured owner who happens to have opened the PR would fire on // essentially every run and train people to ignore it. The token is rendered - // codepoint-escaped because the whole point is that it looks fine verbatim; - // note this is NOT the `escape` declared further down (that one is an HTML - // escaper for the job summary, and `const` — referencing it from here would - // hit the temporal dead zone and throw). - const showInvisible = (s) => s.replace(/[^\x20-\x7e]/g, (c) => '\\u' + c.codePointAt(0).toString(16).padStart(4, '0')); + // codepoint-escaped by `showInvisible`, declared UP in the config-parser region, + // which warns about an unrecognised top-level key and needs it too: a `const` sits + // in the temporal dead zone until its own line runs, and the parser is CALLED above + // here, so declaring it at this point would throw the moment the parser reached for + // it — the same trap that puts the `escape` further down (an HTML escaper for the + // job summary) out of reach here. const warnedLogins = new Set(); const warnConfiguredLogin = (raw) => { const login = normalizeLogin(raw); diff --git a/docs/callers/assign-reviewers.md b/docs/callers/assign-reviewers.md index d2846c0c..ccae5e6e 100644 --- a/docs/callers/assign-reviewers.md +++ b/docs/callers/assign-reviewers.md @@ -221,10 +221,37 @@ since that is normal and would otherwise fire on nearly every run. One consequence of trimming spaces and tabs only: a line whose sole content is some *other* invisible character is no longer a blank line, and indentation counts spaces, so -at column 0 it ends the block above it — inside `rules:` that silently discards every -rule after it. Both ports behave identically here and the corpus pins it, but a stray -non-breaking space at the start of a line is worth ruling out if owners stop being -assigned. Indented, such a line is harmless. +at column 0 it ends the block above it — inside `rules:` that discards every rule after +it. Both ports behave identically here and the corpus pins it. The run no longer leaves +you to guess, though: it **warns**, naming the file and the line number and rendering the +line's content codepoint-escaped so the character is findable — + +``` +::warning::reviewers.yml: line 3 is not a recognised top-level key (\u00a0) — it ends the `default_pool:` block above it, and every list item or rule indented below it is dropped; only `default_pool:` and `rules:` are read, and at column 0 one invisible character (e.g. U+00A0) ends a block exactly like a misspelled key does +``` + +A misspelled or unsupported top-level key truncates the block identically and gets the +same warning. A **near miss** — a line that names a supported key without opening one — +gets its own: + +``` +::warning::reviewers.yml: line 1 is not a recognised top-level key (\u0085default_pool: [alice]) — it is not the supported `default_pool:` key, so the whole block it opens is ignored; that key is read only at column 0 with nothing but spaces before it and a space, a tab or the end of the line after its colon +``` + +That covers a stray character before the key (which is not indentation, so the key reads +as column 0 and falls through) as well as `rules:v2:` — valid YAML for a key *named* +`rules:v2` — and `default_pool:[alice]`, which YAML reads as a plain scalar rather than a +mapping. Both of the latter used to be silently accepted **as** the supported key. + +What does *not* warn is as deliberate. A column-0 line that breaks nothing is silent: a +`---` or `...` document marker (yamllint's default `document-start` rule requires the +former, so a conformant config has one), a `version:`-style key before the first block, +and a stray key between two complete blocks all leave every later `default_pool:`/`rules:` +parsing. So are the orphaned items *below* a stray line, and every indented fallthrough +line. One annotation per thing actually broken is the signal — otherwise a tab-indented +config (indentation counts spaces, so every item reads as column 0) or a zero-indented +block sequence would spend GitHub's ~10-annotation-per-step budget on the symptoms and +bury the cause. ## Gotchas diff --git a/docs/callers/refresh-reviewers.md b/docs/callers/refresh-reviewers.md index 7919e34d..d816ccc9 100644 --- a/docs/callers/refresh-reviewers.md +++ b/docs/callers/refresh-reviewers.md @@ -45,6 +45,18 @@ resets it from the default branch, so only push to it while its bot PR is open. A no-drift run closes a stale still-bot-authored drift PR so an obsolete proposal cannot linger mergeable. +The generator reads `.github/reviewers.yml` with the same parser +[`assign-reviewers.yml`](assign-reviewers.md) uses, so it emits the same warnings about an +unrecognised **top-level key** — a misspelled key, or a line whose only content is one +invisible character such as U+00A0 at column 0, both of which end the block above them; +and a near miss such as `\u0085rules:` or `rules:v2:`, which names a supported key without +opening one. That matters more here than at PR time: the generator rewrites the +*truncated* list it parsed and leaves the orphaned items below the stray line where they +are, so a drift PR that seems to drop reviewers for no reason is worth reading alongside +the run's annotations. A column-0 line that breaks nothing — a `---`/`...` document +marker, a metadata key before the first block — is silent, so the annotations that do +appear all point at something real. + ## Prerequisites | | |