Skip to content

Commit 35daa48

Browse files
committed
ci(test-shards): revert Test Core to the plain-sum layout and the 2026-10-23 dataset (undo f36f619)
The wall-model placement and the cli x4 dataset landed in f36f619 at 08:35Z. From the first queue build on that layout (08:16Z) to 11:05Z, full Test Core merge_group builds went red 5 times in 27. In 05:00-08:10Z the old layout went red 0 times in 19. Each red ejects a PR and rebuilds every entry queued behind it. The red builds all fail on load-sensitive tests: the objectql hook-run-as cold import and the plugin-security shipped-sets case (5 s default timeout), and the cli hook-timeout CONTROL run. The heads that failed passed when they went back in unchanged. The new layout packs about 2141 s of four-wide whole-package work onto each of four shards, where the old one put about 1866 s on each of five. It also puts objectql and plugin-security on the same shard. Suite-duration ceiling readings for plugin-security, plugin-auth, driver-sql and objectql now run 1.3-1.5x over their ceilings. This is a clean revert of both files. Nothing else reads the exports it removes (WHOLE_LEG_PARALLELISM, loadOf, wallOf). The partitioner self-test passes on the restored dataset: 8 shards, max/mean 1.00x, cli x3. Claude-Session: https://claude.ai/code/session_01SrpCjPfYv8N9P9EJvoDyN9 Co-authored-by: Claude <noreply@anthropic.com>
1 parent 712770b commit 35daa48

2 files changed

Lines changed: 276 additions & 550 deletions

File tree

‎scripts/partition-test-shards.mjs‎

Lines changed: 20 additions & 156 deletions
Original file line numberDiff line numberDiff line change
@@ -157,14 +157,6 @@ const TIMINGS_PATH = path.join(REPO_ROOT, 'scripts', 'test-shard-timings.json');
157157
// beside them. That last figure is the floor this count cannot move: a CLI
158158
// slice (~960-1010s measured) is serial, so past eight the slowest shard is
159159
// the slice and more shards only add runners.
160-
//
161-
// ⚠ The bins above were placed by their SUM. Since #25870 partition() places
162-
// them by that same wall model (WHOLE_LEG_PARALLELISM), so a slice shard no
163-
// longer carries a whole-package leg ahead of its slice; on the committed
164-
// dataset's full list the predicted steps move from 1277s on the three slice
165-
// shards (643s elsewhere) to 966s on the slice shards and 830s elsewhere. On
166-
// the 2026-10-26 refresh, with the CLI cut into 4 slices (pin 3c, below), all
167-
// eight read ~738s.
168160
export const SHARD_COUNT = 8;
169161

170162
// The acceptance bound #10472 set: the slowest shard within ~1.3x the mean.
@@ -362,23 +354,10 @@ export function timingModeNote(event) {
362354
// n = 2 869.44s 1.52x still the floor
363355
// n = 3 579.63s 1.01x meets -- spec's task is the floor
364356
//
365-
// So n = 3, and pin 3c held it there. Slices were placed at their PLAIN weight
366-
// (partition() unchanged), so every bin kept the density the whole-package
357+
// So n = 3, and pin 3c holds it there. Slices are placed at their PLAIN weight
358+
// (partition() is unchanged), so every bin keeps the density the whole-package
367359
// split had -- see densityCap() for the hard limit on that and its pin.
368360
//
369-
// RE-DERIVED AT 4 (#25870), on the 14-run refresh of 2026-10-26 (newest run
370-
// 40272142430): the CLI at 2697.45s. `@objectstack/spec` (1168.98s) and
371-
// `@objectstack/runtime` (793.70s) each run as two tasks, so the heaviest
372-
// other serial task is `@objectstack/metadata-protocol`'s 670.12s, and a slice
373-
// must sit within 1.3 x 670.12 = 871.16s.
374-
//
375-
// n = 3 899.15s 1.34x still the floor -- pin 3c refused 3
376-
// n = 4 674.36s 1.01x meets -- metadata-protocol is the floor
377-
//
378-
// Since #25870 partition() places a slice at its wall share rather than its
379-
// plain weight (WHOLE_LEG_PARALLELISM); `total`, and so the density cap, still
380-
// reads plain sums.
381-
//
382361
// ⛔ THE COUNT IS CONFIGURED; WHETHER A RUN SLICES IS DECIDED BY THAT RUN. A
383362
// pull_request or merge_group run splits its own affected set, so planShards()
384363
// asks, of the run's own list, whether a configured package is that run's
@@ -399,7 +378,7 @@ export function timingModeNote(event) {
399378
// a change to what that generator can DECODE, which is why the map it replaced
400379
// is kept below.
401380
export const FILE_SHARDED_PACKAGES = Object.freeze({
402-
'@objectstack/cli': 4,
381+
'@objectstack/cli': 3,
403382
});
404383

405384
// THE MAP AS IT STOOD BEFORE ITS LAST CHANGE, read only by the generator's
@@ -427,10 +406,8 @@ export const FILE_SHARDED_PACKAGES = Object.freeze({
427406
// `test-core-run-summary-*`, #25771), so four days after that PR lands no
428407
// retained summary predates the change and this map decodes nothing.
429408
// The outgoing map of #22075's third round was empty -- the runs before it
430-
// carried no slice. #25870 raised the CLI from 3 slices to 4, so this holds 3.
431-
export const PREVIOUS_FILE_SHARDED_PACKAGES = Object.freeze({
432-
'@objectstack/cli': 3,
433-
});
409+
// carried no slice -- so this one is empty too.
410+
export const PREVIOUS_FILE_SHARDED_PACKAGES = Object.freeze({});
434411

435412
// The item grammar. A shard item is a package (`@objectstack/cli`) or a SLICE
436413
// of one (`@objectstack/cli 1/2`), and this pair of functions is the only place
@@ -905,58 +882,21 @@ function readManifest(dir) {
905882
}
906883
}
907884

908-
// ── A SLICE IS A SERIAL LEG: BINS ARE PLACED BY WALL, NOT BY SUM (#25870) ────
909-
//
910-
// A shard's "Run this shard's tests" step runs its whole packages four at a
911-
// time (`--concurrency=4`) and THEN each file-level slice as a turbo run of its
912-
// own, alone on the runner (ci.yml). So a slice's seconds reach the wall one for
913-
// one while a whole package's are shared about 2.9 ways -- the wall model the
914-
// SHARD_COUNT header calibrated (#25376): step ~ wholeSum / 2.9 + slice. This
915-
// constant is that 2.9, not a new measurement.
916-
//
917-
// Placing bins by their SUM of seconds treated a slice second like a whole
918-
// second, so every slice shard was filled up to the mean with whole packages and
919-
// then ran its slice after them. On the committed dataset's full list that put
920-
// ~900s of whole packages in front of each 966s CLI slice: predicted step 1277s
921-
// on the three slice shards against 643s on the other five, and measured the
922-
// same way on the pull_request runs #25870 tabulates -- 3/8-5/8 at 25-29 min,
923-
// the rest at 7-17, the gap being each slice shard's whole-package leg
924-
// (`Run this shard's tests` minus the slice leg's own turbo `Time:`).
925-
//
926-
// So partition() places by LOAD -- a bin's predicted wall, kept in whole-package
927-
// seconds (a slice counts WHOLE_LEG_PARALLELISM times its weight) so that a list
928-
// with no slice is placed by exactly the same additions and comparisons as the
929-
// plain-sum LPT it replaced, byte for byte. `total` stays the plain sum of
930-
// predicted test windows: the density cap and the drift report are statements
931-
// about that quantity, and neither changes meaning here.
932-
export const WHOLE_LEG_PARALLELISM = 2.9;
933-
934-
// One item's share of a bin's load, in whole-package seconds.
935-
export function loadOf(item) {
936-
return item.slice ? item.weight * WHOLE_LEG_PARALLELISM : item.weight;
937-
}
938-
939-
// A bin's predicted `Run this shard's tests` step under the wall model.
940-
export function wallOf(bin) {
941-
return bin.load / WHOLE_LEG_PARALLELISM;
942-
}
943-
944-
// LPT greedy on load (above): heaviest item into the currently lightest bin.
945-
// Deterministic: input order never matters because both the item sort and the
946-
// bin choice break ties explicitly (by name / by lowest bin index).
885+
// LPT greedy: heaviest package into the currently lightest bin. Deterministic:
886+
// input order never matters because both the package sort and the bin choice
887+
// break ties explicitly (by name / by lowest bin index).
947888
export function partition(items, shardCount) {
948889
const sorted = [...items].sort(
949-
(a, b) => loadOf(b) - loadOf(a) || a.name.localeCompare(b.name, 'en')
890+
(a, b) => b.weight - a.weight || a.name.localeCompare(b.name, 'en')
950891
);
951-
const bins = Array.from({ length: shardCount }, () => ({ total: 0, load: 0, names: [] }));
892+
const bins = Array.from({ length: shardCount }, () => ({ total: 0, names: [] }));
952893
for (const it of sorted) {
953894
let best = 0;
954895
for (let i = 1; i < bins.length; i++) {
955-
if (bins[i].load < bins[best].load) best = i;
896+
if (bins[i].total < bins[best].total) best = i;
956897
}
957898
bins[best].names.push(it.name);
958899
bins[best].total += it.weight;
959-
bins[best].load += loadOf(it);
960900
}
961901
return bins;
962902
}
@@ -965,34 +905,13 @@ export function partition(items, shardCount) {
965905
// `floor` is the heaviest single package: sharding is BY PACKAGE, so no split
966906
// at any shard count can put max below it, and comparing it to the mean says
967907
// whether the bound is even reachable before anyone tries to reach it.
968-
//
969-
// `walls` / `wallRatio` are the same statistic on what the bins were placed by
970-
// (wallOf, #25870). With no slice in the split they are the totals scaled by
971-
// one constant, so the two ratios agree; with a slice, only the wall ratio says
972-
// whether the shards finish together. A bin built by hand (no `load`) is read
973-
// as all whole packages.
974908
export function balanceOf(bins, items = null) {
975909
const totals = bins.map((b) => b.total);
976910
const sum = totals.reduce((a, b) => a + b, 0);
977911
const mean = sum / bins.length;
978912
const max = Math.max(...totals);
979913
const floor = items ? Math.max(0, ...items.map((i) => i.weight)) : null;
980-
const walls = bins.map((b) => wallOf({ load: b.load ?? b.total }));
981-
const wallMean = walls.reduce((a, b) => a + b, 0) / bins.length;
982-
const wallMax = Math.max(...walls);
983-
return {
984-
totals,
985-
sum,
986-
mean,
987-
max,
988-
min: Math.min(...totals),
989-
ratio: mean === 0 ? 1 : max / mean,
990-
floor,
991-
walls,
992-
wallMean,
993-
wallMax,
994-
wallRatio: wallMean === 0 ? 1 : wallMax / wallMean,
995-
};
914+
return { totals, sum, mean, max, min: Math.min(...totals), ratio: mean === 0 ? 1 : max / mean, floor };
996915
}
997916

998917
// ── THE DENSITY CAP: NO SHARD CARRIES MORE THAN THE SPLIT PROVEN GREEN (#22075) ─
@@ -1052,7 +971,7 @@ export function refineToCap(bins, items, cap) {
1052971
const weightOf = new Map(items.map((i) => [i.name, i.weight]));
1053972
const slices = new Set(items.filter((i) => i.slice).map((i) => i.name));
1054973
const movable = (name) => !slices.has(name);
1055-
const out = bins.map((b) => ({ total: b.total, load: b.load, names: [...b.names] }));
974+
const out = bins.map((b) => ({ total: b.total, names: [...b.names] }));
1056975
const EPSILON = 1e-9;
1057976
for (let step = 0; step < 10_000; step++) {
1058977
let h = 0;
@@ -1079,21 +998,15 @@ export function refineToCap(bins, items, cap) {
1079998
}
1080999
if (best === null) break;
10811000
const { to, give, take } = best;
1082-
// Only whole packages move, and a whole package's load is its weight, so
1083-
// `load` follows `total` move for move.
10841001
out[h].names.splice(out[h].names.indexOf(give), 1);
10851002
out[h].total -= weightOf.get(give);
1086-
out[h].load -= weightOf.get(give);
10871003
out[to].names.push(give);
10881004
out[to].total += weightOf.get(give);
1089-
out[to].load += weightOf.get(give);
10901005
if (take !== null) {
10911006
out[to].names.splice(out[to].names.indexOf(take), 1);
10921007
out[to].total -= weightOf.get(take);
1093-
out[to].load -= weightOf.get(take);
10941008
out[h].names.push(take);
10951009
out[h].total += weightOf.get(take);
1096-
out[h].load += weightOf.get(take);
10971010
}
10981011
}
10991012
return out;
@@ -1571,9 +1484,7 @@ const SELF_TEST_BATTERIES = Object.freeze({
15711484
// 8 of these 25 are pins 3b and 3c, the file-level slicing's (#16173), and
15721485
// every one of them runs unconditionally: 3c's fixtures and 3b's cut of the
15731486
// heaviest package do not depend on what the live map slices.
1574-
// 2 of these 27 are pin 2b, the wall placement's (#25870): a fixture, so it
1575-
// runs whatever the dataset holds.
1576-
'the balancing pins (#10472)': 27,
1487+
'the balancing pins (#10472)': 25,
15771488
'predicted-vs-measured drift (#16173)': 9,
15781489
'drift warning tier under the red (#16465)': 12,
15791490
'file-level slice items (#16173)': 20,
@@ -1820,55 +1731,16 @@ function selfTest() {
18201731
});
18211732
const real = datasetPlan.bins;
18221733
const balance = balanceOf(real, datasetItems);
1823-
// Graded on predicted WALL, the quantity the bins are placed by (#25870): with
1824-
// a slice in the split, the bin SUMS are deliberately unequal -- a slice shard
1825-
// carries its serial slice and little else -- and only the walls say whether
1826-
// the shards finish together. With no slice the two ratios are the same number.
18271734
check(() => {
1828-
if (balance.wallRatio > MAX_SHARD_OVER_MEAN) {
1735+
if (balance.ratio > MAX_SHARD_OVER_MEAN) {
18291736
throw new Error(
1830-
`balance: at ${SHARD_COUNT} shards the slowest predicted test step is ${balance.wallRatio.toFixed(2)}x the mean ` +
1831-
`(${balance.wallMax.toFixed(0)}s vs ${balance.wallMean.toFixed(0)}s), past the ${MAX_SHARD_OVER_MEAN}x bound. ` +
1832-
'Walls: ' + balance.walls.map((t) => t.toFixed(0)).join('/') + 's; bins: ' +
1833-
balance.totals.map((t) => t.toFixed(0)).join('/') + 's.'
1737+
`balance: at ${SHARD_COUNT} shards the heaviest bin is ${balance.ratio.toFixed(2)}x the mean ` +
1738+
`(${balance.max.toFixed(0)}s vs ${balance.mean.toFixed(0)}s), past the ${MAX_SHARD_OVER_MEAN}x bound. ` +
1739+
'Bins: ' + balance.totals.map((t) => t.toFixed(0)).join('/') + 's.'
18341740
);
18351741
}
18361742
});
18371743

1838-
// 2b. A SLICE IS PLACED AS THE SERIAL LEG IT IS (#25870). Three shards, a
1839-
// package cut in two (100s a slice) and three whole packages (100s, 50s,
1840-
// 50s). Placed by SUM -- the LPT this replaced -- each slice shard also
1841-
// takes a 50s whole package, which runs four-wide BEFORE the slice and
1842-
// lengthens that shard's step to ~117s. Placed by wall, the slices run
1843-
// alone and the three whole packages share the third shard (~69s). The
1844-
// two placements differ, so a revert to the sum reds here.
1845-
const serial = partition(
1846-
[
1847-
{ name: 's 1/2', weight: 100, slice: { index: 1, count: 2 } },
1848-
{ name: 's 2/2', weight: 100, slice: { index: 2, count: 2 } },
1849-
mk('x', 100),
1850-
mk('y', 50),
1851-
mk('z', 50),
1852-
],
1853-
3
1854-
);
1855-
check(() => {
1856-
const crowded = serial.filter((bin) => bin.names.some((n) => n.startsWith('s ')) && bin.names.length > 1);
1857-
if (crowded.length > 0) {
1858-
throw new Error(
1859-
`slice placement: a slice shard also carries whole packages (${crowded.map((b) => b.names.join('+')).join('; ')}) ` +
1860-
'that run before its serial slice -- are the bins placed by their plain SUM again?'
1861-
);
1862-
}
1863-
});
1864-
check(() => {
1865-
const walls = serial.map(wallOf);
1866-
const expected = [100, 100, 200 / WHOLE_LEG_PARALLELISM];
1867-
if (walls.some((w, i) => Math.abs(w - expected[i]) > 1e-9)) {
1868-
throw new Error(`slice placement: predicted walls ${walls.map((w) => w.toFixed(2)).join('/')}s, expected ${expected.map((w) => w.toFixed(2)).join('/')}s`);
1869-
}
1870-
});
1871-
18721744
// 3. The bound is REACHABLE at this shard count -- the arithmetic #10472
18731745
// asked about when it floated 6 -> 8. No split can put the heaviest bin
18741746
// below the heaviest single package, so once that package exceeds
@@ -3142,8 +3014,7 @@ function selfTest() {
31423014
console.log(
31433015
`partition-test-shards: self-test OK (${datasetPackages.length} measured packages ` +
31443016
`-> ${datasetItems.length} shard items, ${SHARD_COUNT} shards, ` +
3145-
`predicted test-step walls ${balance.walls.map((t) => t.toFixed(0)).join('/')}s, ` +
3146-
`max/mean ${balance.wallRatio.toFixed(2)}x <= ${MAX_SHARD_OVER_MEAN}x, floor ${balance.floor.toFixed(0)}s, ` +
3017+
`max/mean ${balance.ratio.toFixed(2)}x <= ${MAX_SHARD_OVER_MEAN}x, floor ${balance.floor.toFixed(0)}s, ` +
31473018
`bins ${balance.totals.map((t) => t.toFixed(2)).join('/')}s within the ${datasetCap.toFixed(2)}s density cap, ` +
31483019
`file-level slices: ` +
31493020
`${datasetPlan.decisions.filter((d) => d.count > 1).map((d) => `${d.name} x${d.count}`).join(', ') || 'none'})`
@@ -3266,7 +3137,7 @@ function main() {
32663137
{ excluded }
32673138
);
32683139
const mine = bins[shardIndex - 1];
3269-
const { max, mean, ratio, walls, wallRatio } = balanceOf(bins);
3140+
const { max, mean, ratio } = balanceOf(bins);
32703141
// Printed on every shard, not just the imbalanced one, and printed as the
32713142
// RATIO the acceptance bound is written in: the per-shard numbers alone never
32723143
// said whether the split was balanced, which is why #10472's imbalance had to
@@ -3280,13 +3151,6 @@ function main() {
32803151
`density cap ${cap.toFixed(2)}s; ` +
32813152
`${packages - estimated} measured, ${estimated} estimated from test-file count)`
32823153
);
3283-
// What the bins were placed by (wallOf, #25870): the test step each shard is
3284-
// predicted to take, a slice counted as the serial leg it runs as.
3285-
console.error(
3286-
` predicted test-step walls (whole / ${WHOLE_LEG_PARALLELISM} + slice): ` +
3287-
`${walls.map((w) => w.toFixed(0)).join('/')}s, this shard ${wallOf(mine).toFixed(0)}s; ` +
3288-
`max/mean ${wallRatio.toFixed(2)}x`
3289-
);
32903154
// Only a document that states its own slicing prints this, so a Test Core
32913155
// shard's log is unchanged; the nightly's says where its slice count came from.
32923156
if (acrossShards !== null) {

0 commit comments

Comments
 (0)