Repository navigation
Pin the two unpinned widening tie rules in css() aggregation, and correct the roxygen that misdescribes a third (#203) - #208
Conversation
css()'s aggregation resolves several ties by returning more, so a removal at
the per-subsample fitting level can widen what the exported selection
functions return. Two of those widening rules were unpinned by measurement,
and the roxygen sentence meant to disclose a third described a different rule.
Tests (both call getSelectedClusters() directly, on hand-built cssr fixtures):
- the max loop's Exit B -- the `else { break }` arm that fires when clusters
tie at a count strictly below B. That is the branch a removal reaches, since
a removal lowers a proportion rather than raising it to 1.0. The only
existing coverage of this loop reaches it through plot.cssr() at proportion
exactly 1.0, which is the other exit.
- the min loop's silent overshoot, plus the counter-case showing why
max_num_clusts = NA is load-bearing there.
Documentation:
- "divided equally among all the clusters that are tied for the top selection
proportion" now says cluster members, at all eight roxygen sites. It is
members of one cluster that tie; as written the sentence described the
between-cluster cutoff rule instead of the within-cluster weighting rule.
- cssSelect() states, on selected_feats, that sparse weighting lets a selected
cluster contribute more than one feature.
- cssPredict() states the weighted_avg scheme it passes, with the
sum(sel_props) == 0 exception clause getCssSelections() already published,
adapted because cssPredict() returns predictions rather than features.
- getCssDesign() and getCssPreds() carry that same clause, so ?cssPredict and
?getCssPreds no longer disagree about the fallback.
No behavior changes and no condition added.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The new block's comment said Exit A -- `if(thresh > B) break` -- was "the case covered by" the `(#42)` block. It is not. Measured by instrumenting both break arms: the `(#42)` fixture reaches neither break, because its clusters sit at counts B and B - 1 and so are not tied; the threshold separates them and the `while` condition goes false. Exit A is reached by the `(#159c)` plot.cssr block, which is what the plan and its mutation table said all along. The root cause was the comment's opening cardinality, "the max loop has TWO exits". The loop has two `break` statements and three ways out, and counting exits as two forced `(#42)` to be assigned to one of the breaks. The comment now names the block that reaches each way out instead of counting them, per execplan's "name the predicate, not the cardinality". The comment renders onto the published literate-program site, so this was a false claim on a public page rather than an internal note. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review — post-execution pass + drift sentinel, independent contexts, at
|
| fixture | shape | weights |
|---|---|---|
| within-cluster tie | one cluster 1:3, counts 6, 6, 2 |
0.5 0.5 0 |
| three-way within-cluster tie | one cluster 1:4, counts 5, 5, 5, 1 |
1/3 1/3 1/3 0 |
| between-cluster tie | three singletons, counts 6, 6, 2 | c1 → 1, c2 → 1 |
The last is decisive: each tied cluster keeps a full weight vector, so the old wording's "0.5 each" described something the code never does. The fix reaches 8 source sites with 0 survivors of the old wording, and uniq -c over the eight copies returns one distinct string — no drift between them. Generated cssr/man (3/3) and docs/reference (6/6) agree.
"Cluster members" is also the whole story: the all-zero "sparse" path returns 1/n each, which is the tie rule with everything tied at max 0 — which is why the zero-sum exception clause correctly attaches to "weighted_avg" only.
The scope rationale holds
The PR's claim is that measuring first changed the answer — the sparse tie #203 names is already pinned, and the max_num_clusts loop is pinned only on the arm a removal doesn't reach. Both verified: the sparse-tie mutant (which(maxes)[1]) reddens exactly two assertions, in getClustWeights works and formCssDesign works — the two claimed. Exit A truncation reddens one block, the (#159c) plot.cssr one, as claimed. And base-tree instrumentation shows the min_num_clusts overshoot never fires, so "nothing in the suite reached it" is true.
Zero existing blocks moved, and this is stronger than a count: the generated tests.R diff is a single contiguous 109-line insertion with no deletions or modifications, so no pre-existing assertion count can have changed. 182 → 184 blocks, 4298 + 8 = 4306.
1. The Exit B block's comment makes a false reachability claim — the second in those two lines
index.Rmd:10688-10689:
# The max loop can leave more than one way, and this block pins the way
# nothing else reaches.
I instrumented Exit B to record the enclosing test_that() name and ran the full suite (FAIL 0 | PASS 4306, behaviour-neutral). Exit B is reached by three blocks:
getSelectedClusters returns the whole tied set when raising the threshold would empty it (#203)
duplicate clusters are removed to match the docs (#156)
cssSelect works
The comment's other two claims are correct — Exit A's sole reacher is the (#159c) block, and (#42) reaches neither break, exiting on the while condition. What is true of this block is detection, not reachability: it is the only one whose assertions go red when Exit B is broken. The evidence was already in hand — both plan.md § 17 and issue #207 record that cssSelect works loses five executing assertions under the Exit B mutant.
This is a published comment, and it is the second false claim in the two lines that 0553999 exists to correct. One word — "reaches" → "detects" — fixes it.
2. The body's red-side claim names a mutant that doesn't discriminate
"truncating the
else { break }arm gives FAIL 3"
The two reviewers resolved this differently, which is the tell. Read as removing the arm, it is an equivalent mutant: any(counts >= thresh) is monotone, so once it goes false the loop simply spins to the thresh > B exit with selected_clusts unchanged. I measured it — deleting the arm leaves the suite byte-identical at 184 blocks / 0 failed / 4306 passed. Read as truncating the selection at that point, it does give FAIL 3 at tests.R:3379/3385/3391, all inside the new block.
So the block's power is real and the body's description of it is not reproducible as written. Name the mutant the way post_execution_review.md:222 does. This is the same reachability-versus-detection blur as finding 1, one level up.
3. "Every exported page now agrees" is not quite true — a vignette still carries the old understatement
source-files/prediction.Rmd:82 still says "sparse" means "use a single member (the most-selected one) … discards the other members" — exactly what this PR corrects everywhere else, and wrong in the same direction on a tie. Pre-existing, and it is a vignette rather than a help page, but the body's claim covers it. (plot.cssr and summary.cssr are clean — they delegate rather than restate.) Profile gotcha 11 predicts this class: source-files/ holds live prose that the roxygen sweep doesn't reach.
4. Both new expect_warning literals are loose
They end on a bare number, so fixed = TRUE prefix-matches:
grepl("... max_num_clusts = 1", "... max_num_clusts = 12 clusters ...", fixed = TRUE) # TRUE
The assertion would pass if the code returned 12 clusters where it should return 1. Appending " clusters" closes it — verified FALSE with the suffix.
5. The overshoot block pins the silence #206 asks about, without naming it
#206 — "checkSelectedClusters() warns on min_num_clusts undershoot but is silent on overshoot: is the asymmetry intended?" — is an open question about the exact rule the new overshoot block now pins. Pinning is still right: it makes any future change visible rather than silent, which is the point. But the block should name #206, because if that asymmetry is resolved the other way the change would redden five blocks, and whoever does it should know the pin was written with the question already open.
Notes
- The zero-sum exception clause was added at 3 of the 8 weighting copies; the five
@noRdcopies still assert that only subsample-selected members get nonzero weight, which is false in the fallback — andindex.Rmd:3124sits directly above thesum(sel_props) == 0branch that contradicts it. Not published, and better than the base state (1 of 8), but it is the repo's signature shape. - The
do.call(cbind, lapply(props, …))fixture builder is now on its third copy (:10669,:10719,:10761). plan.md§ 17's Exit B line numbers are stale by 4.
What else held up
Gate against a git archive copy, repo never dirtied: build.R exit 0 — so the weave passes at edition 2, and both new blocks were confirmed green at edition 2 as well as 3 — diff -r of committed versus rebuilt cssr/ empty including LitrId, check Status: OK 0/0/0 with no timestamp NOTE, PASS 4306, spell_check 0 rows, url_check four 403s on the three recorded DOIs. cssSelect() documents "sparse" and hardcodes "sparse"; cssPredict() documents and hardcodes "weighted_avg". The weighted_avg zero-sum fallback returns 1/3 1/3 1/3, keeping all members, as the new clause says. #205, #206, #207 all exist, are open, and say what the body implies — worth noting given how often that has been the gap here.
CI's 9m58s ubuntu-devel job is environmental and explained: Cache not found for input keys: … R version 4.7.0 — R-devel rolled, the cache key missed, and dependencies rebuilt from source, putting 7m27s into setup-r-dependencies. check-r-package itself was 1m48s against 1m15s/1m26s on prior devel runs. Not the apt-get variance from #200, and not this change.
Action items
index.Rmd:10688-10689— "reaches" → "detects" (finding 1). Needs a rebuild.- Name the mutant precisely in the body (finding 2). No rebuild.
- Fix
source-files/prediction.Rmd:82, or narrow the body's "every exported page" claim (finding 3). - Append
" clusters"to bothexpect_warningliterals (finding 4); name checkSelectedClusters() warns on min_num_clusts undershoot but is silent on overshoot: is the asymmetry intended? #206 in the overshoot block (finding 5). Same rebuild as (1). - Optional: carry the exception clause to the five
@noRdcopies.
Only (1) is more than editorial, and it is one word in a published comment.
Two independent subagent passes plus orchestrator-run verification: an instrumented Exit B reachability run, the else { break } equivalence mutation, and the fixed = TRUE prefix-match check. Full reports in .plans/issue-203-tie-rule-characterisation/post_execution_review_v2.md and sentinel_feedback_v3.md.
The comment said this block pins the way out of the max loop "nothing else reaches". Measured by instrumenting the arm to record the enclosing test_that name and running the full suite (behaviour-neutral, FAIL 0 | PASS 4306): Exit B is reached by three blocks -- this one, "duplicate clusters are removed to match the docs (#156)", and "cssSelect works". What is true of this block is detection, not reachability: only its assertions go red when Exit B is broken, which is what the FAIL 3 red side measures. The other two lose executing assertions without failing. Replaced in place rather than expanded, so the correction does not lengthen the comment; the measurement belongs in the PR discussion, not on the published literate-program page. This is the second correction to these two lines. Both errors had the same shape -- asserting a relationship between this block and others without instrumenting it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both literals stopped on a bare number, so under fixed = TRUE they prefix-match
any longer number in the same position: grepl("...max_num_clusts = 1",
"...max_num_clusts = 12 clusters...", fixed = TRUE) is TRUE. Appending
" clusters" makes each exact, verified FALSE against the =12 form.
Recorded as cheap hardening rather than as closing a live hole. The review that
raised it described the failure as "would pass if the code returned 12 clusters
where it should return 1", and that is not reachable: the number in the message
is the max_num_clusts ARGUMENT, interpolated by checkSelectedClusters(), not the
count returned -- and both blocks hardcode their arguments at 1 and 2. No return
value can produce the message the loose literal would wrongly accept. The
assertions that discriminate the count are expect_gt() and the exact-names
expect_identical(), which is what the block's own comment already says.
The change still earns its place: the literal is now exact against a future edit
that changes what the fixture passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The block asserts expect_length(overshoot_warns, 0) -- that the min_num_clusts overshoot raises nothing -- and #206 is the open question of whether that silence is intended. Anyone resolving #206 the other way turns this assertion red, and should know the pin was written with the question already open rather than as a statement that the current behaviour is settled. An addition rather than a correction, so it legitimately lengthens the comment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The weighting text stands in eight copies. Before this PR one carried the "(Exception: if no member of a selected cluster was selected on any subsample ...)" clause; the PR took that to three, all exported. The five without it were all @nord, and each asserted -- flatly, and falsely -- that only members selected on at least one subsample get nonzero weight. getClustWeights' own copy sat directly above the sum(sel_props) == 0 branch that contradicts it. All eight now carry it, with the final clause adapted to what each function describes, as the exported copies already were: "are returned" where the subject is the returned features (getSelectedClusters, getAllClustWeights, getClustWeights, matching getCssSelections), "contribute to the representative" where it is a cluster representative (formCssDesign, checkGetCssPredsInputs, matching getCssDesign and getCssPreds). Not published -- @nord generates no man page -- so this is not a gotcha 9 fix. It is a false claim removed from the text maintainers and agents read, at three lines a site, on a rebuild that was happening anyway. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
source-files/prediction.Rmd said "sparse" means "use a single member (the most-selected one) ... discards the other members" -- the same understatement this PR corrects in every roxygen copy, and wrong in the same direction: on a tie at the top selection proportion, every tied member is kept and they share the weight, so a cluster can contribute more than one column. Pre-existing, and a vignette rather than a help page -- but it is executed by R CMD check and published to the pkgdown site, so the sentence is user-facing in substance. Profile gotcha 11 predicts exactly this class: source-files/ holds live prose that a roxygen sweep does not reach. The PR body's "every exported page now agrees" was defensible on a narrow reading of "page" and is now true on a wide one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…wice-corrected one (#203) Two corrections, both found by a narrow review round briefed on the suspicion that this block's comment keeps going wrong the same way. **The vignette sentence was wrong, and I wrote it in this review round.** Fixing the old understatement ("sparse discards the other members") I wrote "so a cluster can contribute more than one column" -- false in the prediction vignette's own context. formCssDesign() enforces `ncol(X_clus_reps) == n_sel_clusts`, so a selected cluster contributes exactly one column; on a sparse tie that column is the AVERAGE of the tied members rather than a copy of one. Measured: a two-member cluster tied at count 6 gives an 8x2 design whose c1 column equals rowMeans of both members. The vignette contradicted the new sentence seventeen lines later, at "# One column per selected cluster". The sentence now says the column is their average rather than a copy of one member. The "more than one" I had in mind is true of selected_feats, which is what cssSelect() returns -- not of a design matrix, which is what this vignette is about. **And the block comment's coverage claim is reformed rather than re-corrected.** It said simplifying the block down to the warning "would delete the pin without failing anything"; measured, the expect_warning is itself one of the three assertions the Exit B mutant reddens, so that is false. This is the second time this sentence has needed correcting, so per the rule for a coverage claim corrected twice it changes FORM rather than content: it now states positively what the three assertions pin, once, with no enumeration of mutation outcomes left to go stale. +6/-6 comment lines, no lengthening. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Thanks — all five findings applied, each verified before I took it. Eight commits now; gate re-run clean at the tip ( 1 — reachability vs detection. Confirmed and fixed in 2 — the equivalent mutant. Confirmed: deleting the 3 — the vignette. Fixed in 4 — I made the change, but the stated mechanism does not hold. The literals do prefix-match, and appending 5 — #206 named, in Optional item taken: all eight weighting copies now carry the exception clause, One more from the narrow round, since it is the same class you were tracking: my comment's claim that "simplifying the block down to the warning would delete the pin without failing anything" was also false — the Also noting Ready for another look. CI is running on |
Closes #203.
css()'s aggregation resolves several ties by returning more, so a removal at theper-subsample level can widen what the exported functions return. #203 asked whether that
surface is characterised. Measuring first changed the answer: the
sparsetie the issue namesis already pinned by two assertions, and the
max_num_clustsloop is pinned only on thebranch a removal does not reach. So this PR pins what nothing pins.
What changed
Two woven test blocks in
index.Rmd, both callinggetSelectedClusters()directly:max_num_clustsloop'selse { break }arm, which fires when clusters tie belowproportion 1.0 — the case a removal produces. The only existing coverage of that loop
exercises the
thresh > Barm instead, throughplot.cssr().min_num_clustsloop's overshoot, which returns more than asked for and raises nothing.Nothing in the suite reached it.
Roxygen: the
@param weightingsentence said thesparsetie divides weight "among all theclusters that are tied" — it is cluster members, so the sentence meant to disclose the
within-cluster rule described the between-cluster one instead. Corrected at every site.
cssSelect()andcssPredict(), which hardcode a weighting, now say which. Every copy gainsthe zero-sum exception clause
getCssSelections()already published, andprediction.Rmd'saccount of
"sparse"is corrected the same way, so no surviving prose says a tie keeps onemember.
No behaviour changes and no condition is added. Both rules do what they are specified to
do; they are now observable to a test and to a reader.
Evidence
Suite
[ FAIL 0 | WARN 0 | SKIP 0 | PASS 4306 ], from 4298. GateStatus: OK—0 errors | 0 warnings | 0 notes, no timestamp NOTE.spell_checknrow 0.url_checkfour403s on the three recorded publisher DOIs, nothing else.
Red side, control
PASS 4306. The mutant is named precisely because a nearby variant does notdiscriminate: truncating
selected_cluststomax_num_clustsinside theelsearm givesFAIL 3, all in the new Exit B block, whereas deleting the arm is an equivalent mutant —
the guard is monotone, so the loop spins to
thresh > Bunchanged (measuredPASS 4306).Truncating the min loop gives FAIL 4, all in the new overshoot block. 182 → 184 blocks,
zero existing blocks moved, 4298 + 8 = 4306.
Filed out of this cycle: #205 (a fourth widening route, on the default
cssPredict()path),#206 (whether the warn/silence asymmetry is intended), #207 (the full tie-surface inventory).
🤖 Generated with Claude Code