Repository navigation
build: lay out tests and scripts by lane, stash the demo around e2e - #108
Merged
Merged
Conversation
The e2e rig refuses to run over a dev instance at the repo root, and the only way round it was to destroy that instance. The two share only the root config and the container names, not data, so a swap is enough. `stacktests stash` stops the instance's running stacklets, moves `stack.toml`, `users.toml` and `.stack/` to `~/famstack-stash/instance` and brings the rig's config back from `~/famstack-stash/rig`, whose secrets match the test data it left. `stacktests unstash` reverses it and starts what was running. The preserved ai stacklet keeps running throughout. Neither command copies or deletes data.
`stacktests e2e` and `smoke` now swap the demo instance out when it is at the repo root and swap it back when the run ends, whether it passed, failed or was interrupted. On the rig itself they run as before. `stash` refuses any instance whose `stack.toml` does not name the Matrix server `simpson`, so a family's instance is never stopped for a test run. The name is fixed at install and is part of every user id.
The lane table told people to uninstall the instance before `make test-e2e`. It now stashes the Simpsons demo for the run and restores it afterwards, and refuses any other instance. The e2e time is unknown again: the old 20-30 minutes included a Synapse invite limit that no longer applies, and no full run has been timed since.
One entry point per job under `script/`, as GitHub's convention has it: - `script/setup` sets a clone up; it is `tools/init-repo`, moved - `script/update` re-runs setup and restarts stale stacklets - `script/test [lane]` runs a lane; `script/test help` prints the table - `script/cibuild` is everything CI checks, runnable on a laptop - `script/server` starts every installed stacklet The Makefile targets stay as one-line aliases. CI calls `script/cibuild` and nothing else, so local and CI runs check the same things: before, commit-lint, the lock check and the version check only ran in CI.
Each backup engine test creates and mounts a 100 MB APFS disk image with `hdiutil`, about 1.2 seconds apiece: 20 of the unit lane's 67 seconds. They drive the host rather than Python, like the Docker lifecycle tests, so they now run in that lane. The unit lane takes about 47 seconds.
The lane now runs the backup engine against real APFS disk images as well as the Docker lifecycle tests, so "lifecycle" no longer describes it. `make test-integration` and `script/test integration` are the lane for the heavier profiles that run real things on the host; `make test-lifecycle` is gone.
Each lane is now a directory, so where a test lives says what it
costs:
- `tests/unit/{framework,stacklets}`: the unit lane, offline
- `tests/integration`: the Docker lifecycle and the APFS backup engine,
which drive the host rather than Python
- `tests/e2e`: demo, smoke, e2e and eval, run through
`tests/e2e/stacktests` (was `tests/integration/stacktests`)
The unit lane no longer needs an ignore list, and `pytest
tests/unit` can no longer start Docker by accident, which running
`pytest tests/framework` used to do. Unit tests sit one level deeper,
so the ones that locate the repo root count one more parent.
arthware-dev
changed the title
Commits and PR descriptions here carry no credit for the tool that helped write them. `tools/commit-lint` now refuses a Co-Authored-By trailer, the tool's generated-with footer, its product link and an Anthropic noreply address. It reads whole messages rather than only subjects, so the `commit-msg` hook stops the commit, CI stops the PR, and a new `--body` input checks the PR description. CI also runs again when a description is edited. The product name on its own is not a pattern: commits that describe the tool, as two on main do, still pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
stacktests e2eandsmokestash the Simpsons demo for the run and restore it afterwards, pass, fail or interrupted; any instance whose Matrix server is notsimpsonis refused.script/setup,update,test,cibuild,server. The Makefile targets are one-line aliases, and CI runs onlyscript/cibuild.tests/unit,tests/integration(Docker lifecycle, APFS backup, wastest-lifecycle),tests/e2e(wastests/integration, includingstacktests). The unit lane drops from ~70s to ~50s.Several commits, meant to be merged with rebase.