Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

## 0.1.0-alpha.6 — source candidate

- Start the next manual-window browser read immediately after validating the previous
snapshot, removing a host pause that contributed to the ARM 109.8 ms gap.
Retain the 100 ms limit, actual timestamps, fresh locators and zero retries.
Add a timer-order regression and match the native benchmark scheduler;
prior measurements remain historical until refreshed.

- Add a read-only release-evidence check that lists all pending records without
running tests or changing artifacts. Share its validation with the final package
gate and reject contradictory CI-guard exit codes, missing hashes and empty cases.
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**Development verification:** The default repeatability check runs four full rounds and a separate four-worker comparison. Final 100-round acceptance remains deferred until the candidate is frozen; [protocol and retained results](docs/reliability.md). Ordinary CI checks the real package and clean consumers without claiming final acceptance.

**Release preparation:** the [review snapshot and exact tested archive](docs/release-handoff.md) are recorded for merged `da7e663`. Main CI passed Linux 169/169 and macOS ARM 507/507. Publication and final acceptance remain pending.
**Release preparation:** main `02ee0953` passed Linux but failed macOS ARM on a 109.8 ms observation gap. The candidate removes the host timer between manual-window reads; CI at `7a60938` passed **510/510 on ARM** and **170/170 on Linux**, plus package consumers, with zero retries. Maximum ARM gaps were **35.8 / 45.2 / 31.0 ms** against the unchanged **100 ms** limit. Earlier archives, repeatability proofs and benchmarks retain their original source identity. [Diagnosis and validation](docs/ci-concurrency.md#consecutive-window-reads). Publication and final acceptance remain pending.

Inspect the remaining saved evidence with `node scripts/check-release-readiness.mjs`
from the repository root. This read-only command does not start tests or publish a package.
Expand Down Expand Up @@ -209,9 +209,9 @@ and [CI evidence](docs/evidence/ci-atomic-observations.json) bind the results to
this source revision. This short development sample is not a guarantee against
future runner stalls. [The investigation](docs/ci-concurrency.md) retains earlier
failures and measured limits. Final 100-round acceptance remains deferred; the
[current-source benchmark](docs/performance.md) has now passed all 108 tests. Real-world adoption has not been established.
[historical benchmark](docs/performance.md) passed all 108 tests at `da7e663`. Real-world adoption has not been established.

### Measured overhead (`da7e663`, 31 October 2026)
### Historical measured overhead (`da7e663`, 31 October 2026)

![KeyTrail overhead against equivalent native Playwright checks](docs/assets/performance.svg)

Expand Down
7 changes: 3 additions & 4 deletions benchmarks/native-reference.ts
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,6 @@ export async function nativeJourney(page: Page, info: TestInfo, manifest: Manife
let firstObserved: number | undefined, lastObserved: number | undefined, maxGap = 0;
const intervals: number[] = [];
while (true) {
const readStarted = performance.now();
const state = await capture(!negative);
const readings = await Promise.all(group.map(c => read(c, state))); guard();
if (negative) {
Expand All @@ -291,9 +290,9 @@ export async function nativeJourney(page: Page, info: TestInfo, manifest: Manife
for (let n = 0; n < group.length; n++) mark(group[n]!, readings[n]!);
break;
}
// Match the same 25 ms target and 10 ms maximum pause as the journey.
const pause = negative ? Math.max(1, Math.min(10, 25 - (performance.now() - readStarted))) : 20;
await delay(Math.max(1, Math.min(pause, negative ? config.observationWindowMs - (state.observedAt - firstObserved!) : deadline - performance.now(), remaining())));
// Match the journey's consecutive asynchronous reads in negative windows.
// Positive waits retain their bounded pause.
if (!negative) await delay(Math.max(1, Math.min(20, deadline - performance.now(), remaining())));
}
index += group.length - 1;
}
Expand Down
50 changes: 50 additions & 0 deletions docs/ci-concurrency.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,56 @@ The four-worker isolation comparison remains part of the local four-round
verifier. A real gap above 100 ms still blocks a negative observation; retries
remain disabled. Results below identify their source revision and runner.

## Consecutive window reads

The [post-merge main run](https://github.com/PerchAnton/keytrail/actions/runs/41194618651)
at `02ee0953` passed Linux 169/169 and its package consumers. ARM passed 168/169
in the first of three planned rounds. `tabs pass manual Home End` stopped at
`T05.backward.003.unchanged` with a **109.8 ms** gap; its snapshot took **98.514 ms**.
The remaining ARM rounds and package checks did not run. The full raw failure and
snapshot profile are retained; no retry replaces this result.

The previous reader took 12.342 ms, so the old scheduler then slept up to 10 ms.
A short previous read cannot predict the latency of the next one. The follow-up
removes that deliberate pause within negative windows. After checking one complete
snapshot, the driver immediately starts the next asynchronous browser read. There
is no overlapping snapshot or background action chain. Fresh locator resolution,
atomic state reads, cleanup and scenario budgets are unchanged. Positive waits
keep their bounded pauses.

This trades additional observations during each bounded window for more timing
headroom. It cannot make an arbitrary browser or shared host meet a hard deadline:
every actual gap above **100 ms** still blocks, and retries remain zero. The native
benchmark reference uses the same consecutive-read policy; its old numbers are
historical until remeasured. Timing records identify the slow read, not the exact
cause of a runner delay.

A new regression queues a host timer after the first real window snapshot and
requires the next read to start before it runs. Restoring the old pause must fail
that regression. Existing before/after transport-delay tests, short-window coverage,
scenario deadlines, transient component failures and handle cleanup remain required.
Validation is limited to three ARM rounds and one Linux round in ordinary CI;
final 100-round acceptance is not requested.

The [CI run at `7a60938`](https://github.com/PerchAnton/keytrail/actions/runs/41196060997)
passed all stages on both platforms. Independent validation checked four raw
reports, **680 executions**, **644 distinct journey IDs**, semantic signatures,
source hashes, guard exits and actual archive checksums. Both vanilla and React
consumers passed on each platform. [Complete record](evidence/ci-consecutive-observations.json).

| Platform / round | Passed | Maximum passing-window gap |
| --- | ---: | ---: |
| Linux / 1 | 170/170 | 32.2 ms |
| macOS ARM / 1 | 170/170 | 35.8 ms |
| macOS ARM / 2 | 170/170 | 45.2 ms |
| macOS ARM / 3 | 170/170 | 31.0 ms |

All rounds use one worker and zero retries or skips. Deliberate 150 ms gaps still
block. These short results support the change in the recorded environments; they
do not guarantee future shared-host latency. No final 100-round series or new local
four-worker comparison was run. A six-case native-reference pilot passed semantic
equivalence and negative controls, but supplies no performance claim.

## Private observation store

Each driver owns one private browser `Map`, accessed through a public Playwright
Expand Down
Loading