Skip to content

Commit 9754e8c

Browse files
yinlianghuiclaude
andauthored
fix(devx): the driver-conformance coverage scan reads code, not prose (#12321)
`coveringFiles` answered "does this driver drive this case-set" from raw file text, so a marker named only in a comment satisfied it. Route the read through `scripts/js-comment-mask.mjs` (`stripComments`, on the module's documented discriminator: this axis reports FILES, never a line or an offset) and pin the hole with a self-test case whose fixture is first proven to be one the unmasked rule accepts. Measured over 5 drivers x 9 case-sets: no cell changes hands and the headline stays 45 covered / 0 DEBT / 0 exempt. Two per-file answers flip, both driver implementations that name FILTER_LOGIC_CASES in prose only — driver-sql/src/sql-driver.ts and driver-mongodb/src/mongodb-filter.ts. Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6 Co-authored-by: Claude <noreply@anthropic.com>
1 parent 3baf1ea commit 9754e8c

1 file changed

Lines changed: 121 additions & 5 deletions

File tree

‎scripts/check-driver-conformance.mjs‎

Lines changed: 121 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,18 @@
9090
// the failure mode that actually happened: three drivers silently not running a
9191
// standard three changesets said they were held to.
9292
//
93+
// A MENTION is not a reference (#12135). The scan reads CODE: every file is
94+
// passed through `scripts/js-comment-mask.mjs` before the import-and-reference
95+
// rule is applied, so a docblock that names a marker cannot make a cell green.
96+
// It used to be able to, and the shape was live on this tree -- `sql-driver.ts`
97+
// and `mongodb-filter.ts` are driver IMPLEMENTATIONS that name
98+
// `FILTER_LOGIC_CASES` only in prose, and both were scored as covering files.
99+
// It changed no cell's verdict, because both cells are genuinely covered by a
100+
// real suite as well; that is precisely why it had to be fixed on a day it was
101+
// harmless. Delete a driver's only suite for a case-set and leave the sentence
102+
// that describes it, and the cell stayed green -- a coverage claim satisfied by
103+
// prose, which is the declared-not-enforced shape this gate exists to close.
104+
//
93105
// ## DEBT is frozen debt, not a permission slip
94106
//
95107
// Every entry was measured against `main`. To clear one: write the suite, then
@@ -982,9 +994,14 @@ function walkTsInto(dir, out) {
982994
* behind. Extracted so the dialect axis asserts a stance the same way this
983995
* script asserts a case-set, rather than inventing a second idiom next to it.
984996
*
985-
* @param {string} src file text — pass it through `stripComments` when a
986-
* comment mentioning the symbol must not count. (`consumes` deliberately does
987-
* not, so its verdicts stay byte-identical to what they were.)
997+
* @param {string} src file text. This function reads whatever TEXT it is given
998+
* and has no opinion about comments — both callers strip first, because on
999+
* both axes a comment mentioning the symbol must not count
1000+
* ({@link coveringFiles} via {@link codeOf}, `dialectStance` via its own
1001+
* `stripComments`). Kept comment-agnostic rather than stripping in here so
1002+
* that `dialectStance`, which already holds stripped text and applies this
1003+
* rule to several symbols in a row, does not re-mask the same text per
1004+
* symbol.
9881005
*/
9891006
function drivenFrom(src, symbol, specifier) {
9901007
if (!src.includes(symbol)) return false;
@@ -1005,15 +1022,51 @@ function drivenFrom(src, symbol, specifier) {
10051022
* different dialect stances, and scoring only the first would let an
10061023
* undeclared one hide behind a matrix-routed sibling — which is exactly the
10071024
* arrangement FILTER_TEXT_CASES is in on this tree.
1025+
*
1026+
* Read through {@link codeOf}, so a marker named only in PROSE is not coverage
1027+
* (#12135). Measured when that was added, over 5 drivers x 9 case-sets: the
1028+
* covered-cell count did not move (45 -> 45) and no cell changed hands, because
1029+
* the two files whose answer flipped are driver implementations whose cells are
1030+
* covered by a real suite too. What moved is WHICH files this returns, which
1031+
* the dialect axis scores and which `consumes` quotes back to the author.
10081032
*/
10091033
function coveringFiles(driverDir, marker) {
10101034
const hits = [];
10111035
for (const file of walkTs(join(driverDir, 'src'))) {
1012-
if (drivenFrom(readFileSync(file, 'utf8'), marker, '@objectstack/spec/data')) hits.push(file);
1036+
if (drivenFrom(codeOf(file), marker, '@objectstack/spec/data')) hits.push(file);
10131037
}
10141038
return hits;
10151039
}
10161040

1041+
/**
1042+
* This file's text with its COMMENT spans gone — the only form the coverage
1043+
* scan is allowed to read.
1044+
*
1045+
* `stripComments` rather than `maskComments`, on the discriminator
1046+
* `js-comment-mask.mjs` documents rather than on which one was already
1047+
* imported: pick `maskComments` when the caller reports a LINE or a byte offset
1048+
* into the original text, `stripComments` when it feeds a scanner and reports
1049+
* neither. This axis reports FILES — `consumes` returns a path, the RECONCILED
1050+
* and CONSUMED messages quote a path — so nothing downstream can be made wrong
1051+
* by the offsets moving, and the deleting form is the one that stays cheap
1052+
* (the same module records a gate that took 51x longer on `maskComments`,
1053+
* because a scan over the whitespace runs blanking leaves is quadratic in the
1054+
* comment bytes).
1055+
*
1056+
* Deliberately NOT memoised per path, though the walk is per (driver x marker)
1057+
* and so re-reads each file once per marker. A cache here is a correctness
1058+
* hazard for the only caller that rewrites a file between two calls — the
1059+
* self-test, which writes three different bodies to one `a.test.ts` path and
1060+
* asserts a different verdict for each. A stale entry would make those three
1061+
* assertions read the first body and pass in a way no refactor could disturb,
1062+
* which is the failure this file's own #4930 note is about: a guard reporting a
1063+
* verdict it did not measure. Measured cost of not caching: the whole gate runs
1064+
* in ~0.4s over this tree.
1065+
*/
1066+
function codeOf(file) {
1067+
return stripComments(readFileSync(file, 'utf8'));
1068+
}
1069+
10171070
/**
10181071
* Does this driver package drive `marker`? The covering file, or `null`.
10191072
*
@@ -1436,6 +1489,67 @@ function selfTest() {
14361489
'local re-declaration must not count as coverage',
14371490
consumes(tmp, 'PAGINATION_CASES') === null,
14381491
);
1492+
1493+
// #12135 — a marker named only in PROSE must not count as coverage.
1494+
//
1495+
// The fixture is the shape that was LIVE on this tree, not an invented one:
1496+
// real imports from the case-set module at the top, the marker mentioned in
1497+
// a line comment among them and again in a docblock far below, and no
1498+
// reference to it anywhere in the code. `sql-driver.ts` and
1499+
// `mongodb-filter.ts` — two driver implementations, neither a suite — both
1500+
// scored as covering files for FILTER_LOGIC_CASES this way.
1501+
//
1502+
// The first assertion is what keeps the second one falsifiable. A fixture
1503+
// the old detector would ALSO have rejected pins nothing: it would go green
1504+
// the day someone reverts `codeOf`, and the case would read as a guard
1505+
// while guarding nothing. So the raw text is asserted to be a case the
1506+
// unmasked rule ACCEPTS, and only then is the real detector asserted to
1507+
// refuse it. Revert the masking and this pair fails on the second line.
1508+
const proseOnly = [
1509+
"import { TEMPORAL_CASES } from '@objectstack/spec/data';",
1510+
"// The `PAGINATION_CASES` table this driver's conformance suite runs; this",
1511+
'// file only compiles the paging clause it exercises.',
1512+
"import { FILTER_LOGIC_CASES } from '@objectstack/spec/data';",
1513+
'',
1514+
'export function compile() { return [TEMPORAL_CASES, FILTER_LOGIC_CASES]; }',
1515+
'',
1516+
'/**',
1517+
' * Kept in step with the `PAGINATION_CASES` table above.',
1518+
' */',
1519+
'export const VERSION = 1;',
1520+
'',
1521+
].join('\n');
1522+
writeFileSync(join(tmp, 'src', 'a.test.ts'), proseOnly);
1523+
expect(
1524+
'#12135 — the prose-only fixture IS accepted by the unmasked rule (else the case below '
1525+
+ 'pins nothing and would survive a revert)',
1526+
drivenFrom(proseOnly, 'PAGINATION_CASES', '@objectstack/spec/data'),
1527+
);
1528+
expect(
1529+
'#12135 — a marker mentioned only in a comment must not count as coverage',
1530+
consumes(tmp, 'PAGINATION_CASES') === null,
1531+
);
1532+
expect(
1533+
'#12135 — and the file is not returned as a covering file either (the dialect axis scores '
1534+
+ 'these, and `consumes` quotes one back to the author)',
1535+
coveringFiles(tmp, 'PAGINATION_CASES').length === 0,
1536+
);
1537+
1538+
// The mirror: prose about the marker does not DISARM a genuine reference.
1539+
// Over-masking would be the quiet direction to fail in here — every cell
1540+
// still green, coverage now judged by a rule that cannot see a suite whose
1541+
// author documented it.
1542+
writeFileSync(
1543+
join(tmp, 'src', 'a.test.ts'),
1544+
proseOnly
1545+
+ "import { PAGINATION_CASES } from '@objectstack/spec/data';\n"
1546+
+ 'for (const c of PAGINATION_CASES) {}\n',
1547+
);
1548+
expect(
1549+
'#12135 — a real driving reference still counts when the same file also discusses the '
1550+
+ 'marker in prose',
1551+
consumes(tmp, 'PAGINATION_CASES') !== null,
1552+
);
14391553
} finally {
14401554
rmSync(tmp, { recursive: true, force: true });
14411555
}
@@ -1915,7 +2029,9 @@ function selfTest() {
19152029
process.exit(1);
19162030
}
19172031
console.log(
1918-
'OK self-test: detects driven / unused / re-declared fixtures, discovers both axes, accounts for '
2032+
'OK self-test: detects driven / unused / re-declared / prose-only fixtures (#12135 — a marker '
2033+
+ 'named only in a comment is not coverage, proven against a fixture the unmasked rule '
2034+
+ 'accepts, and a documented suite is still coverage), discovers both axes, accounts for '
19192035
+ 'every entry under DRIVERS_DIR (a dropped or manifestless row is red, not a smaller matrix), '
19202036
+ 'holds the dead-root hard error (red when a scan root is renamed, green when restored), and '
19212037
+ 'keeps CONSUMED\'s ledger offer marked maintainer-only (#8435). It also declares the driver '

0 commit comments

Comments
 (0)