fix(campaign): export composite proposer#333
Conversation
|
@tangletools review now |
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 8a802d32
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-09T21:03:43Z
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 2 (1 medium-concern, 1 weak-concern) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 128.6s (2 bridge agents) |
| Total | 128.6s |
💰 Value — sound
Exports the already-implemented, already-tested compositeProposer from the campaign barrel — an obvious missing-export fix, fully in-grain, with no existing equivalent to reuse.
- What it does: Adds
export { type CompositeProposerOptions, compositeProposer } from './proposers/composite'to src/campaign/index.ts:195-198, plus a pack/subpath-import survival check in scripts/verify-package-exports.mjs:80-84, and a lockstep 0.110.0→0.110.1 bump (package.json:3, clients/python/.../init.py:61). The symbol itself was shipped in PR #312 (git log: 107dfbd) with 142 lines of implementation ( - Goals it achieves: Makes a budget-fan-out primitive reachable: consumers can now split one population budget across GEPA/SkillOpt/FAPO/memory/trace-analyst/etc. into a single merged generation pool with per-member provenance labels and failure isolation (composite.ts:2-31), instead of copying the portfolio implementation. In-grain: every sibling proposer is exported via the identical `export { type XOptions, xPropos
- Assessment: Good on its merits. It is the minimal, correct change: the implementation and tests already exist and pass, the export line matches the established barrel pattern exactly, the verify script addition mirrors the adjacent analyzeBeliefPolicy survival check (verify-package-exports.mjs:78-79), and the version trio is bumped consistently. Coherent, proportionate, no grain-fighting.
- Better / existing approach: None — this is the right approach. Checked the one candidate the docs point to as an alternative: fapoProposer (src/campaign/proposers/fapo.ts:127, options at :83) is a level-escalation orchestrator (prompt→parameters→structure, one scoped change per cycle, escalate when cheaper level exhausts), a fundamentally different abstraction than compositeProposer's same-surface N-way budget split with mer
- Model: opencode/kimi-for-coding/k2p7
- Bridge attempts: 1
🎯 Usefulness — sound-with-nits
Exports a tested, distinct portfolio-composition proposer that consumers were copying; the only material gap is that the public docs still say in three places this primitive deliberately does not exist.
- Integration: Correctly wired and reachable. The symbol is re-exported from src/campaign/index.ts:195-198, the
./campaignsubpath is declared in package.json:87, and scripts/verify-package-exports.mjs:81-84 now asserts the symbol survivespnpm pack+ subpath import in a real unpacked tarball. Version lock is consistent across package.json, pyproject.toml, uv.lock, and init.py (all 0.110.1). No internal - Fit with existing patterns: Functionally distinct from the three established composition shapes: fapoProposer is sequential escalation (prompt→parameters→structure, cheapest-first, confirmed at src/campaign/proposers/fapo.ts:5), compareProposers runs N proposers each in their own loop on separate holdouts (src/campaign/presets/compare-proposers.ts:9-14), and compositeProposer splits ONE population budget across N proposers i
- Real-world viability: Robust beyond the happy path. Failure isolation: one member throwing is logged and the pool proceeds; only all-members-failing throws (composite.ts:107-115). Largest-remainder budget split guarantees the shares sum exactly to populationSize (composite.ts:73-83). Dedup collapses identical surfaces across members, first-wins (composite.ts:86,103-104). Abort signal respected (composite.ts:91). decide
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
💰 Value Audit
🟡 Docs now contradict the public API [against-grain] ``
docs/improvement-glossary.md:64,75,199 still state 'There is deliberately no chainProposer/compositeProposer primitive' and 'fapoProposer is the composite ... Do not look for a compositeProposer.' That was already stale once #312 landed the implementation, and becomes actively misleading once this PR makes compositeProposer a public export. Not introduced by this diff and outside the value-audit's bug/style scope, but a reviewer may want a one-line glossary correction in the same pass so the pub
🎯 Usefulness Audit
🟠 Public docs say this primitive deliberately does not exist — in 3 places — but this PR exports it without updating them [problem-fit] ``
docs/improvement-glossary.md:64,75,199 each state there is deliberately no compositeProposer and that fapoProposer IS the composite. The glossary's own anti-staleness law (line 9: 'If code and this file disagree, the code wins — fix this file the same turn') makes updating those lines mandatory the same turn the symbol goes public. A consumer who reads the chooser table / composition section / common-traps list is actively told 'do not look for a compositeProposer' while the export now ships. Th
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
✅ No Blockers —
|
| deepseek | kimi-code | aggregate | |
|---|---|---|---|
| Readiness | 95 | 92 | 92 |
| Confidence | 85 | 85 | 85 |
| Correctness | 95 | 92 | 92 |
| Security | 95 | 92 | 92 |
| Testing | 95 | 92 | 92 |
| Architecture | 95 | 92 | 92 |
Reviewer score is advisory once the run is complete and the verdict has no blockers.
Full multi-shot audit completed 5/5 planned shots over 7 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 5/5 planned shots over 7 changed files. Global verifier still owns final merge decision.
🟡 LOW Version is duplicated across pyproject, fallback string, and lock with no sync guard — clients/python/src/agent_eval_rpc/__init__.py
The release version is maintained in three places inside this shot — pyproject.toml L7 ('0.110.1'), this fallback literal L61 ('0.110.1'), and uv.lock root package ('0.110.1') — with no automated check that they agree. Concrete evidence the manual sync has failed before: uv.lock's root package was still '0.79.0' while pyproject.toml was already '0.110.0' (this PR is what finally pulls the lock forward). Impact is low because importlib.metadata.version('agent-eval-rpc') reads build metadata from pyproject and always wins when the package is installed, so the L61 fallback only surfaces in an uninstalled source checkout; but a future bump that updates pyproject and forgets L61 would make
__version__disagree with the installed package in dev/editable edge cases. Fix: keep a single source of
tangletools · 2026-07-09T21:10:59Z · trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Approved — 1 non-blocking finding — 8a802d32
Full multi-shot audit completed 5/5 planned shots over 7 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 5/5 planned shots over 7 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-09T21:10:59Z · immutable trace
…le-generation runs propose with trace context (#334)
…into feat/export-composite-proposer
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 5d1e553f
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-09T21:20:35Z
tangletools
left a comment
There was a problem hiding this comment.
🟢 Value Audit — sound
| Verdict | sound |
| Concerns | 0 (none) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 77.7s (2 bridge agents) |
| Total | 77.7s |
💰 Value — sound
Re-exports the already-built/tested compositeProposer (the one proposer family missing from the public surface), makes single-generation runs propose from a baseline-trace diagnosis instead of blind, and locks the four version surfaces — all coherent and in the codebase's grain.
- What it does: Three concrete deltas: (1) re-exports the pre-existing, tested
compositeProposer+CompositeProposerOptionsfrom@tangle-network/agent-eval/campaign(src/campaign/index.ts:195-198); it was the only proposer family NOT re-exported — ace/evolutionary/fapo/gepa/halo/memory/policy-edit/skill-opt/trace-analyst are all present at lines 194-242. (2) InrunOptimization, firesanalyzeGenerationo - Goals it achieves: Consumers can split one population budget across GEPA/SkillOpt/FAPO/memory/trace-analysis via the canonical portfolio primitive (src/campaign/proposers/composite.ts:53) instead of copying it; this also corrects a doc/code lie, since docs/improvement-glossary.md:65 already lists compositeProposer as "Exported." Findings-grounded proposers no longer run blind on the common
generations=1path — the - Assessment: Sound. The export closes the one genuine gap in the proposer re-export list (every sibling is exported). The baseline -1 hook reuses the SAME
analyzeGenerationseam and the SAMEgeneration: -1convention the Pareto accumulator already established (run-optimization.ts:153), so it introduces no new concept — it's the minimal extension of an existing pattern. Thecells.length > 0guard is exact - Better / existing approach: none — this is the right approach. Searched for an existing portfolio allocator: only src/campaign/proposers/composite.ts exists;
fapoProposeris ordered-level escalation, explicitly distinguished from composite in the glossary (docs/improvement-glossary.md:200), so composite is not a duplicate — it was just unexported. Searched for an existing pre-gen-0 analysis path: the between-generation pro - Model: opencode/kimi-for-coding/k2p7
- Bridge attempts: 1
🎯 Usefulness — sound
Exports an already-built, already-tested, already-documented compositeProposer that was simply missing from the package entry point, plus a coherent fix so single-generation runs stop proposing blind, plus a release-version-lock guard.
- Integration: Fully reachable and wired into the established contract.
compositeProposerreturns aSurfaceProposer(src/campaign/proposers/composite.ts:53-141) — the exact interface every other proposer factory (gepa, skill-opt, fapo, ace, memory, evolutionary) returns, so it drops intorunImprovementLoop({ proposer }),runOptimization,selfImprove, andcompareProposersunchanged. The export (src/ca - Fit with existing patterns: Fits the grain precisely and does not compete with established patterns. Every proposer is a factory
xProposer(opts): SurfaceProposer(docs/improvement-glossary.md:56);compositeProposerfollows that shape exactly. It is explicitly distinct from its two nearest cousins, both documented as such:fapoProposerESCALATES through ordered levels spending on the cheapest first (src/campaign/propose - Real-world viability: Holds up off the happy path.
compositeProposerisolates per-member failures (one member throwing logs a warning and continues; only all-fail throws — src/campaign/proposers/composite.ts:107-115, tested at composite.test.ts:73-83), dedups identical surfaces across members, and uses a deterministic largest-remainder population split that sums exactly. The baseline-analysis change has correct edge - Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
No concerns — sound change, no better or existing approach found. ✅
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
The merge-base changed after approval.
✅ No Blockers —
|
| opencode-kimi | glm | deepseek | aggregate | |
|---|---|---|---|---|
| Readiness | 80 | 68 | 73 | 68 |
| Confidence | 95 | 95 | 95 | 95 |
| Correctness | 80 | 68 | 73 | 68 |
| Security | 80 | 68 | 73 | 68 |
| Testing | 80 | 68 | 73 | 68 |
| Architecture | 80 | 68 | 73 | 68 |
Reviewer score is advisory once the run is complete and the verdict has no blockers.
Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision.
🟠 MEDIUM Stale comment contradicts new compositeProposer separation — docs/improvement-glossary.md
The code-example comment reads:
//fapoProposerIS the composite proposer.— this was true when compositeProposer didn't exist. Now that compositeProposer is a distinct proposer for fixed-weight population splitting, calling fapoProposer 'the composite' is misleading. The comment should describe fapoProposer as the escalation/level-based proposer, consistent with lines 79 and 200.
🟡 LOW Entry classified under 'Fixed' but the change is additive (exports + import check) — CHANGELOG.md
The section is '### Fixed' but exporting previously-internal symbols and adding a packed-package import check are net-new surface area. Keep a Changelog reserves 'Added' for new features / newly-exposed surface and 'Fixed' for bug fixes. The repo's own header says it 'roughly follows' Keep a Changelog so this is non-blocking, but '### Added' would be the more accurate section. No behavior or correctness impact.
🟡 LOW Version gap: CHANGELOG skips 0.109.x between 0.108.1 and 0.110.1 — CHANGELOG.md
Changelog jumps from [0.108.1] directly to [0.110.1] with no 0.109.x entry. package.json is 0.110.1 so they are internally consistent, but a missing 0.109.x release in the log is either an undocumented release or a versioning irregularity. Worth a maintainer glance; not a code defect.
🟡 LOW No inline release note for the patch bump — clients/python/pyproject.toml
The version moves 0.110.0 -> 0.110.1 but no CHANGELOG/release-note line ships in this diff. If the repo convention is to record per-release notes, this is a process gap; if releases are documented externally (git tags / GitHub Releases), it is non-blocking. Informational only — no correctness or security impact.
🟡 LOW uv.lock had drifted 31 minor versions behind pyproject (0.79.0 vs 0.110.x) — clients/python/uv.lock
Before this PR the lockfile's editable-root version was 0.79.0 while pyproject.toml was already 0.110.0, meaning
uv lock/sync had not been run (or enforced) across many releases. This PR fixes the immediate drift by setting it to 0.110.1, which is correct and also necessary foruv sync --locked/--frozento keep passing. Recommend a CI check runninguv lock --locked(oruv sync --locked) in clients/python so the lock can never silently diverge from pyproject again. No functional impact on this changeset.
🟡 LOW Section heading numbering not updated after renumbering shapes — docs/improvement-glossary.md
The heading
### 2 — race proposers head-to-head for a lift CIreferences the old shape numbering where race was #2. The shapes were renumbered (1=Portfolio, 2=Escalate, 3=Race, 4=Plug-in), and the other heading was correctly updated from### 1 + 3to### 2 + 4. This heading should be### 3to match.
🟡 LOW Stale section number after four-shape renumbering — docs/improvement-glossary.md
The composition list was expanded from three to four shapes: a new '1. Portfolio' was inserted (lines 78-81), shifting Escalate to 2, Race to 3, and Plug in to 4. The heading at line 83 was correctly updated from '### 1 + 3' to '### 2 + 4', but the heading at line 151 still reads '### 2 — race proposers head-to-head for a lift CI'. Under the new numbering '2' is Escalate, not Race — Race is now item 3. The nume
🟡 LOW Stale section number for the race example after renumbering — docs/improvement-glossary.md
The composition list was rewritten to four shapes: 1=Portfolio(compositeProposer), 2=Escalate(fapoProposer), 3=Race(compareProposers), 4=Plug-in. The escalation example heading was updated to '### 2 + 4' (correct), but this race example heading was left at '### 2', which now incorrectly refers to Escalate rather than Race. A reader following the numbers lands on the wrong section. Fix: change line 151 to '### 3 — race proposers head-to-head for a lift CI'. The code below it (compareProposers usage) is unchanged and correct.
🟡 LOW Version-extraction regexes couple CI to exact uv.lock / pyproject.toml / init.py formatting — scripts/verify-package-exports.mjs
matchVersion() parses versions via literal-shape regexes (e.g. /[[package]]\s*\nname = "agent-eval-rpc"\s*\nversion = "([^"]+)"/ for uv.lock, /^version = "([^"]+)"$/m for pyproject.toml). These match the current file shapes today (verified: all return 0.110.1), but a future uv major-version lockfile refactor or a hatchling schema change would make matchVersion throw 'could not read release version from ' — a loud CI failure rather than a silent miss, so this aligns with fail-loud, but it is unrecoverable without a script edit. No fix required for this PR; worth a one-line comment noting the expected file shapes if format drift is a concern. Not blocking.
🟡 LOW pyproject version regex is whitespace-brittle — scripts/verify-package-exports.mjs
Pattern /^version = "([^"]+)"$/m requires exactly one space on each side of
=. Valid TOML likeversion="0.110.1"(which a formatter such astaplo/ruff formatcould emit) would fail matchVersion and throwcould not read release version from clients/python/pyproject.toml, blocking the verify script. Today's file uses spaces so it passes; this is latent brittleness. Fix: relax to /^version\s*=\s*"([^"]+)"$/m.
🟡 LOW verifyVersionLock/matchVersion have no unit coverage — scripts/verify-package-exports.mjs
The four-source lockstep gate (npm, pyproject, init fallback, uv.lock) and matchVersion() are exercised only by running the whole script (pnpm pack + tar + node --eval). A regression in any regex would only surface at CI/release time. Acceptable for glue, but a tiny test that feeds known-good/bad fixtures to matchVersion and the mismatch aggregation would cheaply lock behavior. Not blocking.
🟡 LOW New baseline call site silently ignores a non-array analyst return — src/campaign/presets/run-optimization.ts
Line 173
if (Array.isArray(fresh)) currentFindings = freshis NEW code in this diff. If opts.analyzeGeneration returns a non-array (e.g. undefined from a partially-wired analyst, or a malformed shape), currentFindings stays at the static opts.findings seed and generation 0 proposes with stale/blind findings — masking a real wiring defect. The same pattern exists at line 284 (between-gen), so this is consistent with existing behavior and not a regression; it does, however, add a second call site of the loose guard
🟡 LOW Baseline-analysis test omits a composite-value assertion — tests/campaign/presets.test.ts
The single-generation test asserts surfaceHash, runDir, history, and campaign.cells artifacts on the baseline candidate, but does not assert candidates[0].composite, which the implementation sets to winnerComposite = campaignMeanComposite(baselineCampaign) (run-optimization.ts:169). For the passJudge (composite=1, two cells) the expected value is 1. Not load-bearing for the findings wire under test, so non-blocking, but adding
expect(candidates[0]!.composite).toBe(1)would pin the full baseline-candidate shape.
🟡 LOW Cannot verify test execution (infrastructure) — tests/campaign/presets.test.ts
vitest hangs during test collection in this worktree environment — no test output beyond the RUN header. TypeScript typecheck passes clean. The test assertions are structurally correct against the implementation, but execution confirmation is missing. Not a code defect; environment constraint.
🟡 LOW No test for analyzeGeneration returning non-array — tests/campaign/presets.test.ts
The implementation guards
if (Array.isArray(fresh)) currentFindings = freshat run-optimization.ts:173. If analyzeGeneration returns undefined or a non-array, findings stay at the static seed. No test covers this fallback path for the baseline analysis case. Low severity because the existing 'without analyzeGeneration' test at line 1506 covers the null-provider path, and the non-array return is an edge case of the callback contract.
🟡 LOW No test for maxGenerations=0 + analyzeGeneration present (baseline-analysis skip branch) — tests/campaign/presets.test.ts
The implementation guard at run-optimization.ts:164 is
opts.analyzeGeneration && opts.maxGenerations > 0 && baselineCampaign.cells.length > 0. Two of the three new tests exercise the cells.length>0 false branch (empty scenarios) and the maxGenerations>=1 + cells>0 true branch, but no test covers maxGenerations===0 with analyzeGeneration provided and a non-empty baseline — the branch where the guard correctly skips baseline analysis because there is no generation 0 to feed. Concrete gap: that arm of the guard is unverified. A 4-line test mirroring the empty-scenarios test with scenarios: SCENARIOS, maxGenerations: 0 would close it. Not blocking — the behavior is correct by inspection and the guard is simple.
🟡 LOW New test asserts analyzeGeneration fired on baseline but not that propose() consumed its findings — tests/contract-self-improve.test.ts
The test title + inline comment claim the point is that 'propose() is not blind', but the only assertion is
expect(analyzed).toEqual([-1])(lines 232-237) — i.e. that the producer fired with generation -1, not that its returned findings[{ claim: 'baseline finding' }]actually reached proposer.propose() as ctx.findings for gen 0. The fakeproposer(lines 69-79) ignoresfindings, so a regression where selfImprove/runOptimization fired analyzeGeneration on the baseline but failed to thread the result into curre
🟡 LOW Pre-existing test assertion is now looser than needed after baseline-analysis addition — tests/contract-self-improve.test.ts
The assertion
expect(calls).toBeGreaterThanOrEqual(1)was previously justified by a comment (now removed) explaining that analyzeGeneration ran only between generations, so ≥2 generations were needed to fire at least once. After the baseline-analysis addition (generation -1 fires pre-gen-0), with generations=2 the call count is 2 (baseline + between-gen-0). The assertion still passes but no longer catches a regression that drops only the baseline call. Tighten toexpect(calls).toBeGreaterThanOrEqual(2)orexpect(calls).toBe(2)to make the contract explicit.
🟡 LOW Test title oversells what is asserted — tests/contract-self-improve.test.ts
The title claims 'so propose() is not blind', but the test only asserts analyzeGeneration is CALLED on baseline with generation=-1 (analyzed.toEqual([-1])). It does not assert the returned findings reach proposer.propose() as ctx.findings. The wiring exists at run-optimization.ts:173 (if Array.isArray(fresh)) currentFindings = fresh), so the title is directionally true, but the test would still pass if that wiring were broken. Optional strengthening: capture the findings seen by the fake proposer and assert they contain 'baseline finding'. Not blocking — the call-site contract is the more regression-prone seam and is well-pinned.
tangletools · 2026-07-09T21:36:59Z · trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Approved — 19 non-blocking findings — 5d1e553f
Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-09T21:36:59Z · immutable trace
The merge-base changed after approval.
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 2ed3826b
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-09T21:42:46Z
The merge-base changed after approval.
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 2 (2 weak-concern) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 140.7s (2 bridge agents) |
| Total | 140.7s |
💰 Value — sound
Exports the already-built, already-tested compositeProposer on the public /campaign subpath and hardens the release gate against two drift classes that just bit this repo — a clean, in-grain, non-duplicative change; ship.
- What it does: Re-exports
compositeProposer+CompositeProposerOptionsfrom src/campaign/index.ts:195-198 so it is reachable via@tangle-network/agent-eval/campaign(package.json:87-91 maps that subpath to dist/campaign/index). Adds a packed-subpath import probe for it in scripts/verify-package-exports.mjs:82-85, plus a new verifyVersionLock() (mjs:94-131) that hard-fails when npm package.json, clients/pyt - Goals it achieves: Lets consumers split one population budget across GEPA/SkillOpt/FAPO/memory/trace-analyst/ACE/evolutionary in a single generation pool (composite.ts:51-122 — largest-remainder budget split, per-member
kind:provenance labels, dedup, per-member failure isolation, unanimous-decide) instead of copying the portfolio implementation. Mechanically prevents two recently-recurring drift classes: implem - Assessment: Good on its merits and squarely in the codebase's grain. The 4-line export mirrors every sibling proposer in index.ts (ace/evolutionary/gepa/fapo all export
type XOptions, x). The primitive is real and tested (src/campaign/proposers/composite.test.ts covers budget split, dedup, one-member-down, all-down, mixed, unanimous-decide, and arg validation). The verification additions target demonstrated - Better / existing approach: None — this is the right approach. Searched for an existing proposer-combiner (
grep -E 'portfolio|ensemble|fan-?[Oo]ut|composeProposers|mergeProposers|multiProposer'across src): everyfan-outhit is LLM/judge/subprocess concurrency (judge-panel.ts:45, judge-runner.ts:34, prm/inference.ts:70), not proposer composition;compareProposers(presets/compare-proposers.ts) is a benchmark that runs - Model: opencode/kimi-for-coding/k2p7
- Bridge attempts: 1
🎯 Usefulness — sound-with-nits
Exports an already-tested, already-documented compositeProposer (closing a docs-vs-code gap), fixes a real blind-spot so single-generation runs propose with trace context, and adds durable release-version locking — all in the codebase's grain.
- Integration: The export lands at src/campaign/index.ts:195-198 alongside every other proposer (ace, evolutionary, fapo, gepa, etc.), and scripts/verify-package-exports.mjs:83-84 proves it survives
npm pack+ subpath import. docs/improvement-glossary.md:65,72,78 already described compositeProposer as 'Exported; you wire the member proposers' — so the docs were AHEAD of the export and this PR makes them honest - Fit with existing patterns: compositeProposer implements the exact SurfaceProposer shape (composite.ts:64 returns kind/propose/decide) and slots into runOptimization({proposer}) identically to its peers. It is a DISTINCT shape from the established alternatives: the glossary's four-shape taxonomy (Portfolio vs Escalate vs Race vs Plug-in) explicitly separates it from fapoProposer (ordered escalation) and compareProposers (sep
- Real-world viability: Edge handling is careful: the baseline analysis is skipped when maxGenerations===0, when analyzeGeneration is absent, and when baselineCampaign.cells.length===0 (dry/offline modes have no traces) — and tests/campaign/presets.test.ts covers all three, plus the generations=1 case asserting the baseline runDir, empty history, and baseline artifacts are passed correctly. compositeProposer itself handl
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
🎯 Usefulness Audit
🟡 analyzeGeneration contract widens to generation:-1 for existing multi-gen consumers [robustness] ``
For runs with maxGenerations>=2, an existing analyzeGeneration callback that previously only saw generation in [0, maxGenerations-2] will now ALSO be invoked once with generation:-1 (run-optimization.ts:164-173). A consumer that did e.g.
const labels = ['gen0','gen1']; return [{claim: labels[generation]}]would now index labels[-1] → undefined. This is well-documented and uses the established Pareto-baseline convention, and for the genuinely new single-generation case there was no prior call t
💰 Value Audit
🟡 verifyVersionLock() is broader than the campaign-export title and lives inside the export-probe script [maintenance] ``
scripts/verify-package-exports.mjs:10,94-131 now does two jobs — packed-export verification AND npm/python/uv.lock version-lock enforcement — under a PR titled 'fix(campaign): export composite proposer'. Both are genuinely valuable and recently motivated (version skew forced two 'lock python client' commits in a row), and co-locating with the existing
pnpm verify:packagerelease gate is defensible, so this does not gate shipping. If a reviewer wants it cleaner, the version-lock reads (mjs:94-1
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
✅ No Blockers —
|
| deepseek | kimi-code | aggregate | |
|---|---|---|---|
| Readiness | 82 | 77 | 77 |
| Confidence | 95 | 95 | 95 |
| Correctness | 82 | 77 | 77 |
| Security | 82 | 77 | 77 |
| Testing | 82 | 77 | 77 |
| Architecture | 82 | 77 | 77 |
Reviewer score is advisory once the run is complete and the verdict has no blockers.
Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision.
🟠 MEDIUM uv.lock was 31 minor versions stale (fixed here) — clients/python/uv.lock
Before this PR, uv.lock carried version 0.79.0 while pyproject.toml was at 0.110.0 — a ~31-version gap caused by never running
uv lockafter a series of bumps. This PR fixes it to 0.110.1. Impact of the stale lock was low (editable installs bypass root version metadata), but it meantuv syncon a fresh checkout used wrong package metadata. The new verifyVersionLock() gate in scripts/verify-package-exports.mjs cross-checks all four version locations and will fail CI on any future mismatch.
🟡 LOW No test covers version fallback path — clients/python/src/agent_eval_rpc/__init__.py
The
except PackageNotFoundError: __version__ = '0.110.1'fallback is not exercised by any test (test_version_via_subprocess tests the RPC version() call against the TS server, not the Python package's version). The new verifyVersionLock() release gate catches version drift at build time, but an incorrectly hardcoded fallback string would not be caught by any unit test.
🟡 LOW compositeProposer failure-isolation wording slightly narrower than implementation — docs/improvement-glossary.md
The row says composite 'isolates a member failure unless every member fails.' The implementation (composite.ts:107-115) throws whenever the merged pool is empty (
pool.length === 0), which also happens when all members simply return zero candidates without throwing. So the throw condition is 'no candidate survived from any member,' not strictly 'every member threw.' Impact: negligible — a reader could infer that silent empty returns are tolerated, when in fact an all-empty generation still errors. Fix: reword to 'isolates a member failure, and only errors if no candidate survives from any member.'
🟡 LOW Missing-file path throws raw ENOENT instead of the curated error — scripts/verify-package-exports.mjs
readFileSync for pyproject.toml, init.py, and uv.lock (lines 96-101) is unguarded; matchVersion only produces the friendly 'could not read release version from ' message when the file exists but the regex fails. If any file is absent (partial checkout, renamed path), the script surfaces a raw
ENOENTstack rather than the path-tagged message, making CI failures harder to triage. Low because all three files exist in-repo and the script is repo-root-relative. Fix: wrap each readFileSync in a try/catch that rethrows ascould not read release version from <path>: <cause>.
🟡 LOW Version-lock now duplicates an existing publish workflow step — scripts/verify-package-exports.mjs
.github/workflows/publish.yml:46-66 already runs a 'Verify version lock between npm and PyPI packages' step comparing npm vs pyproject vs python runtime
__version__. The new in-script check overlaps that coverage (plus adds the uv.lock source) and, because it lives insideverify:package, now also runs on every ci.yml invocation (ci.yml:44), not just publish. This is plausibly intentional (earlier drift detection + uv.lock coverage), but it creates two sources of truth for the same rule; if they ever diverge in what they require, CI/publish failures become confusing. Consider having the workflow step delegate to this script (single source) rather than maintaining both.
🟡 LOW pyproject match takes the first version = line — scripts/verify-package-exports.mjs
/^version\s*=\s*"([^"]+)"$/muses String.match, which returns the firstversion = "..."line in the file. Today the project version is line 7 and is first, so it resolves correctly (0.110.1). If a later edit adds aversion = "..."entry above it (e.g. under a[tool.*]table or adynamic/dependency block), the gate would silently compare against the wrong string and either false-fail or false-pass. Fix: anchor to the[project]table (e.g. match/\[project\][\s\S]*?^version\s*=\s*"([^"]+)"$/m) or use a TOML parser.
🟡 LOW uv.lock version regex depends on name/version field adjacency — scripts/verify-package-exports.mjs
matchVersion(uvLock, /\[\[package\]\]\s*\nname = "agent-eval-rpc"\s*\nversion = "([^"]+)"/, ...)requiresname = "agent-eval-rpc"to be immediately followed byversion = "..."with only a newline between. The uv lockfile format is not a stable cross-version contract: if a future uv release insertssource = ...(or another field) betweennameandversion, or reorders fields, the regex stops matching andpnpm verify:packagefails on every run with the misleading 'could not read release version from clients/python/uv.lock'. It matches today (verified), so this is latent, not active. Fix: relax to allow any fields between name and version within the same stanza, e.g. /[[package]][\s\S]?name = "agent-eval-rpc"[\s\S]?version = "([^"]+)"/ bounded to the next[[package]], or
🟡 LOW Defensive non-array return branch of baseline analyzeGeneration is untested — tests/campaign/presets.test.ts
The new baseline block in run-optimization.ts (lines 165-173) guards
if (Array.isArray(fresh)) currentFindings = fresh, so if a producer returns a non-array the loop should keep the prior findings. The three new/updated tests only cover the array-return path (gen -1 produces the finding used by gen 0) and the skipped path (no cells). No test asserts that a non-arrayanalyzeGenerationresult leaves currentFindings as the seed. Impact is low: the producer is typed(input) => Promise<unknown[]>, so a non-array return is a type violation and the guard is belt-and-suspenders. Fix: add one small test whose baselineanalyzeGenerationreturns a non-array (e
tangletools · 2026-07-09T22:00:07Z · trace
tangletools
left a comment
There was a problem hiding this comment.
✅ Approved — 8 non-blocking findings — 2ed3826b
Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 8/8 planned shots over 12 changed files. Global verifier still owns final merge decision.
Full immutable report for this review: trace
Summary comment for this run: full summary
tangletools · 2026-07-09T22:00:07Z · immutable trace
The merge-base changed after approval.
What changed
compositeProposerand its options from@tangle-network/agent-eval/campaignWhy
Consumers can now split one population budget across GEPA, SkillOpt, FAPO, memory, trace-analysis, and other proposer families without copying the existing portfolio implementation.
Verification
pnpm test(269 files, 2733 passed, 2 skipped)pnpm typecheckpnpm lint(passes with 4 pre-existing warnings)pnpm buildpnpm verify:packageuv run --extra dev pytest -v(18 passed)git merge-tree --write-tree origin/main HEAD