ci: smoke the runtime image, pin CI deps, structure test results

[ezutfen]

Four independent CI changes, from a review of the existing workflows against measured run data.

Correction (latest commit). This PR was opened as "speed up the real-DB suite …". It does not speed it up. Three runs below measured the durability change at no gain, and the title and item 1 have been rewritten to match. Details in Measured results. The other three items did what they were meant to.

Baseline

From run 30300374394 (PR #123's merge-ref run, so its test content is comparable to this PR's base):

Job	Duration
compose-real-db-merge-ref	9m 13s
compose-real-db-exact-head (separate workflow)	9m 30s
conformance-vectors	20s
compose-validate	13s
lock-drift	8s

Inside the 9m13s job: checkout 2s, buildx 7s, Docker build 21s (layer caching is already well tuned), compose stack 8m 40s, cleanup 2s. Inside that, the root pytest run is 481s — 92% of the critical path. Its --durations=25 profile is flat: slowest test 2.45s, top 25 summing to ~35s (7%), remaining 2,931 tests averaging 0.152s. There is no hot spot; the suite is dominated by per-test fixed overhead.

1. Postgres durability off in the CI stack — no measured gain

docker-compose.ci.yml ran stock durability settings while ~3000 tests each committed at least once. The cluster is rebuilt from the bundled migrations every run and destroyed with down -v, so fsync / synchronous_commit / full_page_writes protect nothing. PGDATA moves to a size-capped (2g) tmpfs.

This was expected to be the speedup, and it is not. Measured below: per-test cost is unchanged to within run-to-run variance. That is what the baseline profile already implied and I did not follow through on — a flat --durations means the suite is bound by per-test fixed overhead, not by commit durability, so removing write barriers had nothing to bite on. The premise was reasoning about a bottleneck the profile had already ruled out.

The settings stay, but as correctness-and-hygiene for a disposable cluster rather than as an optimization. max_connections is raised to 200 because each test process opens several pooled engines (runtime, owner, read, provisioner) and the default 100 leaves no headroom for running the suite in parallel — that is a prerequisite for sharding, which is where the time actually is.

2. Build and smoke the production runtime image

CI only ever built the ci target. compose-validate runs docker compose config, which validates syntax but never builds. So the runtime target — a different install path (non-editable pip install ., no dev extras) behind a different entrypoint (uvicorn) — could break and reach main undetected.

A new runtime-image-smoke job builds it, asserts the package imports and the engram console script resolves, then boots it against a real database and probes /health and /ready.

/ready is the load-bearing assertion: it proves the production image reached Postgres as the non-owner engram_app role under FORCE RLS, resolved a tenant context, and found pgvector >= 0.8. That makes this the one place CI exercises the production role configuration end to end — the main suite connects as the owner, which bypasses RLS.

The service lives in docker-compose.ci.yml behind a smoke profile so it is excluded from the default up that runs the test suite.

3. Per-test timeout and JUnit XML

The suite is concurrency-heavy (worker dedup/auto-supersede/flagging, promotion review/feedback, manual invalidation) and a deadlock previously consumed the full 30-minute job timeout without naming the offending test. pytest-timeout now caps each test at 60s, ~24x the slowest test.

--timeout-method=signal is deliberate: thread hard-exits via os._exit() and would abort pytest before it writes the results below.

All four suites emit JUnit XML, which both workflows copy out of the stopped container and upload as an artifact. Failures become structured results instead of a scroll through a Compose log interleaved with Postgres output.

4. Pin the CI image's dependencies to uv.lock

The image was built with a bare pip install -e ".[dev]", re-resolving the floating floors in pyproject.toml (fastapi>=0.115, openai>=1.0, ...) on every cache miss. CI could therefore test against versions differing from what uv sync --extra dev gives developers, and an upstream release could break the build with no change to this repository — all while the lock-drift job passed, since it only compares uv.lock to pyproject.toml.

This was not hypothetical: it surfaced a live defect. engram_mcp.server imports mcp.server.fastmcp, which exists only in mcp>=1.2,<2 — 1.1.3 and earlier lack it, and 2.0.0 removes it in favour of mcp.server.mcpserver. The declared mcp>=1.0 admitted both broken ends, and CI stayed green only because the newest release happened to sit inside the working range. It stopped being green the moment 2.0.0 shipped, with no change here. The bound is now mcp>=1.2,<2.

Pinning is done by exporting uv.lock at build time as a pip constraints file, with the SDK and adapters added as uv workspace members so one consistent set is resolved across everything installed together. Constraints alone would have been unsound: a root-only lock has no entry for httpx-sse, jsonschema, pyjwt and the rest of mcp's tree, and a constraints file bounds versions without forcing any, so pip would silently resolve whatever fits instead of failing.

uv export --frozen never rewrites the lockfile and fails if it has drifted, so this also repeats the lock-drift gate inside the build. uv is pinned to 0.11.29, the same version the lock-drift job uses.

Also

Adds the missing timeout-minutes to compose-validate, the only job that was inheriting the 6-hour default.

Measured results

From runs 30376403187 / 30376403318 and 30377598800, all six checks green.

Root suite, the 92% critical path:

	Tests	Wall clock	Per test
Before	2956	481.58s	0.1629s
After (dadf4fc, merge-ref)	2992	508.32s	0.1699s
After (dadf4fc, exact-head)	2992	500.01s	0.1671s
After (70447d4, merge-ref)	2992	494.41s	0.1652s

The three "after" rows are the same suite on unchanged test code, and they span 13.9s. That spread is the measurement floor, and the gap to baseline sits barely outside it while the suite carries 36 more tests (+1.2%). The durability change is therefore not measurable — treat it as zero, not as a small win.

Whole-job wall clock, merge-ref: 9m13s baseline → 10m33s on dadf4fc → 9m29s on 70447d4. The middle figure was a one-time Dockerfile layer-cache miss, now confirmed rather than assumed: 70447d4 leaves the Dockerfile untouched and its build step came in at 24.1s against the 21s baseline, versus 64s when the dependency layer rebuilt. So the steady-state cost of item 4 is ~3s of uv export, and the residual +16s on the job is the extra tests.

New job: runtime-image-smoke, 1m 15s, runs in parallel and does not extend the critical path.

Net: this PR buys coverage (production image, RLS-as-app-role), diagnosability (JUnit artifacts, per-test timeout) and reproducibility (pinned deps, plus one real latent break fixed), at a steady-state cost of ~3s of build time. It does not buy time.

Verification

Validated locally before pushing:

• uv lock --check under uv 0.11.29 (the version lock-drift pins)
• ruff check . under ruff 0.15.21 (the version uv.lock pins)
• docker compose config -q on both compose files; the smoke service is absent from the default service list and present under --profile smoke
• Both workflow YAMLs parse with the expected job/step structure
• tests/test_ci_contract.py — 13 passed, updated to pin the new gates
• Reproduced the Dockerfile install (pip with the exported constraints, all four workspace members editable): mcp resolves to 1.29.0; MCP adapter 36, SDK 55, engram-hooks 186 all pass

Confirmed in the hosted run: the constraints install, the tmpfs mount, /ready under the app role, and the JUnit artifacts (4 XML files uploaded per real-DB job).

Not in scope

The review behind this PR also covered sharding the root suite, the duplicated full-suite run across ci.yml and exact-head-ci.yml, a dead engram-ci-exact-head cache scope, untested Python 3.11 support, mypy covering only engram/, a missing ruff format --check, and no coverage measurement despite pytest-cov being a dev dependency.

Sharding is now the load-bearing one. With durability ruled out and the profile flat, per-test fixed overhead across parallel runners is the only remaining lever on the 8-minute suite.

---

Generated by Claude Code