Repository navigation
feat(agentctl): add exec --json #810
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Real-substrate conformance in kind. Heavy (kind + KVM + ate-system + micro-VM assets); it gates | |
| # every PR as well as running nightly. It copies agent-substrate's own pr-workflow.yaml recipe | |
| # (/dev/kvm udev rule, kind bring-up). | |
| # | |
| # The suite is an ordinary Go test package (integrations/substrate/e2e), compiled with `go test -c` | |
| # and run as an in-cluster Job. It reaches each harness the way a production host does: harnesswire | |
| # over mTLS to the atenet-router Service, with the ate-target-actor metadata naming the actor. A new | |
| # capability is a new Test function, not another driver binary, Job manifest, or polling shell. | |
| # | |
| # ate-system is installed with substrate's default envoy dataplane, which serves only mTLS clients | |
| # whose SPIFFE ID is on --ingress-allowed-spiffe-ids. The only router client in this cluster is the | |
| # suite's Job, which presents the SVID of its ServiceAccount (deploy/substrate/e2e-job.yaml), so the | |
| # allowlist names that one ID. | |
| # | |
| # ActorTemplates are substrate API resources, not Kubernetes objects, so the suite creates its own | |
| # through Control with the harness image built here; the workflow only applies the WorkerPools. | |
| # | |
| # Substrate is installed from the images its release publishes (ghcr.io/agent-substrate/substrate, | |
| # tag SUBSTRATE_VERSION), not built: ate-setup's --image-repo/--image-tag pins each control-plane | |
| # image to its digest, and hack/substrate-image.sh does the same for the worker image the pools run. | |
| # The manifests, ate-setup and kubectl-ate still come from a checkout of SUBSTRATE_REF, which must | |
| # be the commit the tag names. Only this repository's images (harnessnode, the suite) are built. | |
| # | |
| # Both tiers run on the micro-VM sandbox class, so the suite exercises the same isolation class in | |
| # both and covers the micro-VM path end to end. ate-setup stages the class (assets and the | |
| # `microvm` SandboxConfig) while installing ate-system, before any tier runs. The suite then runs in | |
| # passes against one cluster so the cheap tier fails first: | |
| # 1. stateless tier: the echo harness + byte-identical replay, and a suspend under an idle stream. | |
| # 2. idle stream: one harness stream held idle through the router for 6m30s. | |
| # 3. stateful tier + fork: the in-RAM counter driven to N, suspended to a memory snapshot and | |
| # continued at N+1, which no journal replay could reconstruct, then forked k ways, with every | |
| # child required to continue at N+1 independently. | |
| # | |
| # Scheduled and manual runs add a canary entry that runs the same suite against agent-substrate's | |
| # newest release tag, installed from that release's images, to surface drift before a bump. It runs | |
| # only while that tag differs from SUBSTRATE_VERSION, so it skips until a newer release exists, and | |
| # a tag whose images are not published yet fails it. It never runs on a pull request, and its | |
| # failure does not fail the run. | |
| # | |
| # On a pull request the changes job first diffs the merge commit against the base branch with | |
| # hack/ci-path-filter.sh. When only paths the suite does not read changed (that script's substrate | |
| # list: docs, markdown, the site and the other paths no image, test or workflow reads), both live | |
| # jobs skip their cluster work and the required gate reports success in seconds. Push to main, the | |
| # nightly schedule and manual runs always run the full suite. | |
| # | |
| # ate-setup, kubectl-ate and the suite are still compiled here, so the Go build and module caches | |
| # are kept between runs, and a warm run reuses them instead of compiling from scratch. | |
| # The durable example has a separate runner and cluster, exercising up.sh, reuse, smoke.sh and | |
| # down.sh without conformance setup hiding bootstrap bugs. The existing required check combines | |
| # the pinned conformance suite and the example lifecycle. | |
| name: substrate-e2e | |
| on: | |
| push: | |
| branches: [main] | |
| # No paths filter: substrate-conformance is a required check, and a workflow a paths filter skips | |
| # never reports it, so the pull request could not merge. The changes job below decides instead, | |
| # and the live jobs skip their cluster work when nothing they check changed. | |
| pull_request: | |
| schedule: | |
| - cron: "0 7 * * *" # nightly, so a green main is re-proven against a fresh cluster | |
| workflow_dispatch: {} | |
| permissions: | |
| contents: read | |
| # One cluster run per ref: a new push supersedes the in-flight run rather than burning a second | |
| # cluster alongside it. Scheduled and main runs key on their own ref and are a day apart. | |
| concurrency: | |
| group: substrate-e2e-${{ github.ref }} | |
| cancel-in-progress: true | |
| # The agent-substrate release our adapter's go.mod pins (the v1 API, api/ateapi/v1), and the commit | |
| # its tag names; keep all three in lockstep so the cluster's ate-api and our generated client agree. | |
| # The client decodes Actors by field number, and upstream has renumbered fields before without a | |
| # wire error, so a mismatch corrupts silently rather than failing. The examples read both lines. | |
| env: | |
| SUBSTRATE_VERSION: v1.0.0 | |
| SUBSTRATE_REF: b34fbee742906494b5eb4f1f36987176bf8770a4 | |
| KO_DOCKER_REPO: localhost:5001 | |
| E2E_IMAGE: localhost:5001/agentsessions-e2e:${{ github.sha }} | |
| # ate-setup requires an image credential provider; the kind recipe uses the k8s.io one. | |
| ATE_CREDENTIAL_PROVIDER: '{"name":"k8s.io"}' | |
| jobs: | |
| changes: | |
| # Decides whether this pull request changes anything substrate-conformance checks, from the | |
| # merge commit's diff against the base branch (hack/ci-path-filter.sh holds the ignore list). | |
| # Every other event runs the full suite. | |
| name: substrate-e2e path filter | |
| runs-on: ubuntu-latest | |
| outputs: | |
| decision: ${{ steps.filter.outputs.decision }} | |
| canary: ${{ steps.canary.outputs.version }} | |
| steps: | |
| - name: Checkout agentsessions | |
| if: github.event_name == 'pull_request' | |
| # A failed checkout must not fail this job: the next step then decides `run`. | |
| continue-on-error: true | |
| uses: actions/checkout@v7 | |
| with: | |
| fetch-depth: 2 # the merge commit and both its parents | |
| persist-credentials: false | |
| - name: Decide whether the suite runs | |
| id: filter | |
| env: | |
| EVENT: ${{ github.event_name }} | |
| run: | | |
| # This step never fails: any doubt decides `run`, so the jobs below run in full. | |
| decision=run | |
| if [[ "${EVENT}" != pull_request ]]; then | |
| echo "${EVENT}: the full suite runs." | |
| elif ! git rev-parse --verify --quiet 'HEAD^2' >/dev/null; then | |
| # A pull_request checkout is the merge commit; its first parent is the base branch head. | |
| echo "::error::HEAD is not the pull request's merge commit; the full suite runs." | |
| elif ! hack/ci-path-filter-test.sh; then | |
| echo "::error::hack/ci-path-filter-test.sh failed; the full suite runs." | |
| else | |
| # Decide with the base branch's copy of the filter, so a pull request that edits the | |
| # filter never decides its own run: the filter's own path is relevant in every profile. | |
| # The pull request's copy decides only while the base branch has none. | |
| filter="${RUNNER_TEMP}/ci-path-filter.sh" | |
| git show 'HEAD^1:hack/ci-path-filter.sh' >"${filter}" 2>/dev/null || | |
| filter=hack/ci-path-filter.sh | |
| echo "Deciding with ${filter}." | |
| answer="$(bash "${filter}" substrate 'HEAD^1' HEAD)" || answer=error | |
| if [[ "${answer}" == run || "${answer}" == skip ]]; then | |
| decision="${answer}" | |
| else | |
| echo "::error::the path filter answered '${answer}'; the full suite runs." | |
| fi | |
| fi | |
| echo "decision=${decision}" | tee -a "${GITHUB_OUTPUT}" | |
| - name: Pick the canary's Substrate release | |
| # Scheduled and manual runs only: agent-substrate's newest release tag (an rc sorts before | |
| # its release), or nothing when it is the pinned SUBSTRATE_VERSION, so the canary entry runs | |
| # only against a newer release. A failed lookup warns and runs no canary. | |
| id: canary | |
| if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' | |
| run: | | |
| newest="$(git -c versionsort.suffix=- ls-remote --tags --refs --sort=-v:refname \ | |
| https://github.com/agent-substrate/substrate 'v*' | | |
| sed -n 's#^[0-9a-f]*\trefs/tags/##p' | | |
| grep -m1 -E '^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$')" || newest= | |
| if [[ -z "${newest}" ]]; then | |
| echo "::warning::cannot list agent-substrate's release tags; no canary runs." | |
| elif [[ "${newest}" == "${SUBSTRATE_VERSION}" ]]; then | |
| echo "The newest Substrate release is the pinned ${SUBSTRATE_VERSION}; no canary runs." | |
| else | |
| echo "version=${newest}" | tee -a "${GITHUB_OUTPUT}" | |
| fi | |
| substrate-conformance-suite: | |
| # The final substrate-conformance gate combines this suite and the independent example. | |
| # The canary (release) exists only on scheduled and manual runs, and only when the changes job | |
| # found a release newer than the pin; continue-on-error keeps its failure out of the run's | |
| # result. | |
| # | |
| # The pinned entry and the canary share the v1 layout (go.ate.dev/substrate/api/ateapi/v1, | |
| # cmd/ate-setup --kind --image-repo), so the canary fails only on drift past the pin. It has not | |
| # run green yet; until it does, read its result in the job list. continue-on-error hides it from | |
| # the run's status and notifications. | |
| # TODO(canary): drop continue-on-error once a nightly canary is green, so new drift turns the | |
| # nightly run red (it still never runs on a pull request). | |
| name: ${{ matrix.substrate == 'release' && 'substrate-conformance (newest substrate release)' || 'substrate-conformance suite' }} | |
| needs: changes | |
| # Runs even when the changes job fails, so a filter failure cannot silently skip coverage. | |
| if: ${{ !cancelled() }} | |
| runs-on: ubuntu-latest | |
| continue-on-error: ${{ matrix.substrate == 'release' }} | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| substrate: >- | |
| ${{ fromJSON( | |
| (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') | |
| && needs.changes.outputs.canary != '' | |
| && '["pinned", "release"]' || '["pinned"]' | |
| ) }} | |
| steps: | |
| - name: Fail when the path filter failed | |
| # The changes job never fails on its own (any doubt decides `run`). If it still does (a | |
| # runner fault), its decision is empty: fail substrate-conformance rather than trust it. | |
| if: ${{ !cancelled() && needs.changes.result != 'success' }} | |
| env: | |
| RESULT: ${{ needs.changes.result }} | |
| run: | | |
| echo "::error::the path filter job ended '${RESULT}'; rerun the workflow." | |
| exit 1 | |
| - name: Skip the suite (no substrate-relevant changes) | |
| # The changes job found only paths the suite does not read, so every step below is skipped | |
| # and this job reports success; the changes job's log lists each path and its verdict. | |
| if: needs.changes.outputs.decision == 'skip' | |
| run: | | |
| msg="No substrate-relevant path changed: substrate-conformance passes without a cluster." | |
| echo "${msg}" | tee -a "${GITHUB_STEP_SUMMARY}" | |
| - name: Free disk space (in the background) | |
| # Nothing here uses these toolchains. Deleting them took ~100s in the foreground, so delete | |
| # them at the lowest priority while the cluster comes up; "Wait for the disk cleanup" below | |
| # collects the result. Nothing waits on it before then: substrate's own e2e, a superset of | |
| # this cluster, runs with no cleanup at all, and the df here records the space without it. | |
| if: needs.changes.outputs.decision != 'skip' | |
| run: | | |
| df -h / | |
| { | |
| status=0 | |
| sudo nice -n 19 ionice -c 3 \ | |
| rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL || | |
| status=$? | |
| echo "${status}" >"${RUNNER_TEMP}/free-disk.done" | |
| } >"${RUNNER_TEMP}/free-disk.log" 2>&1 </dev/null & | |
| - name: Checkout agentsessions | |
| if: needs.changes.outputs.decision != 'skip' | |
| uses: actions/checkout@v7 | |
| - name: Setup Go | |
| if: needs.changes.outputs.decision != 'skip' | |
| id: setup-go | |
| uses: actions/setup-go@v7 | |
| with: | |
| # The adapter module tracks substrate's Go floor, which can be ahead of the root module's. | |
| go-version-file: integrations/substrate/go.mod | |
| # The Go caches below cover this module and substrate's; setup-go's own cache keys on | |
| # the root go.sum only and would restore over the same paths. | |
| cache: false | |
| - name: Setup ko | |
| if: needs.changes.outputs.decision != 'skip' | |
| uses: ko-build/setup-ko@v0.10 | |
| - name: Clone agent-substrate | |
| # The release whose images are installed (`version`: SUBSTRATE_VERSION, or the canary's | |
| # newer tag) and the commit its tag names (`commit`), for the manifests and tools. The | |
| # pinned entry refuses a tag that no longer names SUBSTRATE_REF: the images would not match | |
| # the manifests checked out here. | |
| if: needs.changes.outputs.decision != 'skip' | |
| id: substrate | |
| env: | |
| SUBSTRATE: ${{ matrix.substrate }} | |
| CANARY_VERSION: ${{ needs.changes.outputs.canary }} | |
| run: | | |
| version="${SUBSTRATE_VERSION}" | |
| [[ "${SUBSTRATE}" != release ]] || version="${CANARY_VERSION}" | |
| git clone https://github.com/agent-substrate/substrate _substrate | |
| commit="$(git -C _substrate rev-parse --verify "refs/tags/${version}^{commit}")" | |
| if [[ "${SUBSTRATE}" != release && "${commit}" != "${SUBSTRATE_REF}" ]]; then | |
| echo "::error::tag ${version} names ${commit}, not SUBSTRATE_REF ${SUBSTRATE_REF}." | |
| exit 1 | |
| fi | |
| git -C _substrate -c advice.detachedHead=false checkout "${commit}" | |
| git -C _substrate log -1 --format="agent-substrate ${version} %H (%cd)" | |
| { | |
| echo "version=${version}" | |
| echo "commit=${commit}" | |
| } >> "$GITHUB_OUTPUT" | |
| - name: Restore the Go build and module caches | |
| # ate-setup and kubectl-ate are compiled from the substrate checkout, the suite from this | |
| # one. Keyed on the substrate commit and the Go version (what decides substrate's builds) | |
| # plus our go.sum files; the prefixes fall back to the newest cache for the same commit, | |
| # then Go version. | |
| if: needs.changes.outputs.decision != 'skip' | |
| id: go-cache | |
| uses: actions/cache/restore@v6 | |
| with: | |
| path: | | |
| ~/.cache/go-build | |
| ~/go/pkg/mod | |
| key: substrate-e2e-go-${{ runner.os }}-${{ steps.setup-go.outputs.go-version }}-${{ steps.substrate.outputs.commit }}-${{ hashFiles('go.sum', 'integrations/substrate/go.sum') }} | |
| restore-keys: | | |
| substrate-e2e-go-${{ runner.os }}-${{ steps.setup-go.outputs.go-version }}-${{ steps.substrate.outputs.commit }}- | |
| substrate-e2e-go-${{ runner.os }}-${{ steps.setup-go.outputs.go-version }}- | |
| - name: Cache micro-VM assets (kata-static + cloud-hypervisor + virtiofsd) | |
| # The SHA-256 digests in sandboxconfig-microvm.yaml.tmpl pin the assets, so key the cache on | |
| # its hash, as substrate's own CI does. ate-setup skips assembling when these are present. | |
| if: needs.changes.outputs.decision != 'skip' | |
| uses: actions/cache@v6 | |
| with: | |
| path: _substrate/bin/microvm-assets | |
| key: microvm-assets-${{ hashFiles('_substrate/manifests/ate-install/sandboxconfig-microvm.yaml.tmpl') }} | |
| - name: Enable KVM | |
| # Grant /dev/kvm via the udev rule substrate's CI uses: both tiers run on micro-VMs (kata + | |
| # cloud-hypervisor), which need it, and create-kind-cluster.sh mounts it into the node. | |
| if: needs.changes.outputs.decision != 'skip' | |
| run: | | |
| echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \ | |
| | sudo tee /etc/udev/rules.d/99-kvm4all.rules | |
| sudo udevadm control --reload-rules | |
| sudo udevadm trigger --name-match=kvm | |
| - name: Create kind cluster + install ate-system | |
| # Global ate-setup flags come before the `deploy ate-system` command. The default | |
| # --sandbox-classes=gvisor,microvm also assembles the micro-VM assets (cached above), stages | |
| # them into the in-cluster object store and applies the cluster-scoped `microvm` | |
| # SandboxConfig, which our pools (sandboxClasses: [{name: microvm}], no configRef) resolve | |
| # to. ate-setup waits for the api-server before its database has converged on a cold | |
| # runner, so give the rollouts more than the default budget. `rollout status` returns as | |
| # soon as a rollout is ready, so this costs nothing on a healthy cluster. | |
| # The default envoy router admits only the allowlisted SPIFFE IDs: here the suite's Job | |
| # (deploy/substrate/e2e-job.yaml), the only router client this cluster runs. | |
| # --image-repo/--image-tag install the release's published images; ate-setup runs no ko. | |
| if: needs.changes.outputs.decision != 'skip' | |
| working-directory: _substrate | |
| env: | |
| SUBSTRATE_VERSION: ${{ steps.substrate.outputs.version }} | |
| run: | | |
| hack/create-kind-cluster.sh | |
| go run ./cmd/ate-setup --kind --rollout-timeout=10m \ | |
| --ingress-allowed-spiffe-ids=spiffe://cluster.local/ns/ate-agentsessions/sa/substrate-e2e \ | |
| --image-repo=ghcr.io/agent-substrate/substrate --image-tag="${SUBSTRATE_VERSION}" \ | |
| deploy ate-system | |
| - name: Wait for the disk cleanup | |
| if: needs.changes.outputs.decision != 'skip' | |
| run: | | |
| done_file="${RUNNER_TEMP}/free-disk.done" | |
| for _ in $(seq 300); do | |
| [[ -s "${done_file}" ]] && break | |
| sleep 2 | |
| done | |
| cat "${RUNNER_TEMP}/free-disk.log" | |
| echo "cleanup exit status: $(cat "${done_file}" 2>/dev/null || echo 'still running after 10m')" | |
| df -h / | |
| - name: Build the harness image (shared by every tier) | |
| # Every tier runs the same harnessnode image (HARNESS_KIND selects echo vs counter). ko prints | |
| # the digest-pinned reference, which substrate requires of an ActorTemplate image. | |
| if: needs.changes.outputs.decision != 'skip' | |
| run: echo "HARNESS_IMAGE=$(ko build ./cmd/harnessnode)" >> "$GITHUB_ENV" | |
| - name: Apply the echo micro-VM WorkerPool | |
| # The release's ateworker-microvm, pinned to its digest once; the counter pool below reuses | |
| # it. | |
| if: needs.changes.outputs.decision != 'skip' | |
| env: | |
| SUBSTRATE_VERSION: ${{ steps.substrate.outputs.version }} | |
| run: | | |
| ATEWORKER_MV_IMG="$(hack/substrate-image.sh ateworker-microvm)" | |
| echo "ATEWORKER_MV_IMG=${ATEWORKER_MV_IMG}" >> "$GITHUB_ENV" | |
| kubectl --context kind-kind apply -f deploy/substrate/namespace.yaml | |
| sed -e "s#ko://go.ate.dev/substrate/cmd/ateworker-microvm#${ATEWORKER_MV_IMG}#" \ | |
| deploy/substrate/echo-microvm-workerpool.yaml | kubectl --context kind-kind apply -f - | |
| hack/wait-worker-pool.sh echo-microvm | |
| - name: Provision the suite's atespaces and access | |
| # Substrate always enforces authorization. The suite's Job runs as the substrate-e2e | |
| # ServiceAccount with only the atespace editor role, the role operators grant agentsessionsd, | |
| # so this creates its atespaces and grants that role as kubectl-ate's bootstrap owner | |
| # (ate-client). kubectl-ate is built from the same substrate commit as the cluster. | |
| if: needs.changes.outputs.decision != 'skip' | |
| run: | | |
| (cd _substrate && go build -o "${RUNNER_TEMP}/kubectl-ate" ./cmd/kubectl-ate) | |
| KUBECTL_ATE="${RUNNER_TEMP}/kubectl-ate" hack/provision-e2e-access.sh | |
| - name: Build the adapter against the newest substrate release (canary only) | |
| # The lockstep rule above applies to the canary too: the cluster runs the newer release, so | |
| # the suite's client must come from the same tag. This edits the runner's checkout only. | |
| if: needs.changes.outputs.decision != 'skip' && matrix.substrate == 'release' | |
| working-directory: integrations/substrate | |
| env: | |
| SUBSTRATE_VERSION: ${{ steps.substrate.outputs.version }} | |
| run: | | |
| go get "go.ate.dev/substrate@${SUBSTRATE_VERSION}" | |
| go mod tidy | |
| - name: Build the conformance test binary image | |
| # `go test -c` rather than ko: this is a test binary, and ko builds main packages. Static | |
| # (CGO_ENABLED=0, pure-Go sqlite), pushed to the same local registry kind pulls from. | |
| if: needs.changes.outputs.decision != 'skip' | |
| run: | | |
| cd integrations/substrate | |
| CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go test -c -o "${RUNNER_TEMP}/e2e.test" ./e2e | |
| cp ../../deploy/substrate/Dockerfile.e2e "${RUNNER_TEMP}/Dockerfile" | |
| docker build -t "${E2E_IMAGE}" "${RUNNER_TEMP}" | |
| docker push "${E2E_IMAGE}" | |
| - name: Save the Go build and module caches | |
| # After the last build, so a test failure below still keeps them. Pinned entry only, and | |
| # only when the exact key missed: the canary's release changes with each new tag, and | |
| # saving each one would evict the cache pull requests restore. | |
| if: >- | |
| needs.changes.outputs.decision != 'skip' | |
| && matrix.substrate == 'pinned' | |
| && steps.go-cache.outputs.cache-hit != 'true' | |
| uses: actions/cache/save@v6 | |
| with: | |
| path: | | |
| ~/.cache/go-build | |
| ~/go/pkg/mod | |
| key: ${{ steps.go-cache.outputs.cache-primary-key }} | |
| - name: Stateless-tier conformance (in-cluster Job) | |
| # Runs before the stateful tier, so a break in the cheap tier fails first. The idle-stream | |
| # suspend test holds a turn's harness stream open through the router: Suspend must refuse the | |
| # session while the turn runs and succeed promptly once it returns. A suspended stateless | |
| # session whose actor was deleted must be refused, not cold-created. The router must refuse | |
| # a client that presents no certificate, and must wake a suspended actor for a routed call. | |
| if: needs.changes.outputs.decision != 'skip' | |
| env: | |
| ECHO_SANDBOX_CLASS: microvm | |
| run: hack/run-e2e-job.sh '^(TestRouterRefusesClientWithoutCertificate|TestRoutedCallWakesSuspendedActor|TestStatelessReplay|TestSessionSuspendResume|TestStatelessSessionWhoseActorIsLostIsRefused|TestSuspendUnderAnIdleHarnessStream|TestSessionDeletePending|TestExecutionState.*)$' "${E2E_IMAGE}" 1200 | |
| - name: Idle harness stream through the router (in-cluster Job) | |
| # A turn parked on a slow model call holds its Connect stream idle. The envoy router bounds | |
| # streams by --route-timeout (5m default) plus a 30s idle timeout, and reset the idle stream | |
| # at 5m30s on kind. The gap stays well past that bound so the outcome is not a race. The | |
| # step records the router's behavior rather than gating a change, and costs about 6 | |
| # minutes, so it runs on push, nightly and manual runs only, not on every pull request. | |
| if: needs.changes.outputs.decision != 'skip' && github.event_name != 'pull_request' | |
| env: | |
| ECHO_SANDBOX_CLASS: microvm | |
| E2E_IDLE_GAP: 6m30s | |
| run: hack/run-e2e-job.sh 'TestHarnessStreamIdlePastRouteTimeout' "${E2E_IMAGE}" 1200 | |
| - name: Apply the counter micro-VM WorkerPool | |
| if: needs.changes.outputs.decision != 'skip' | |
| run: | | |
| sed -e "s#ko://go.ate.dev/substrate/cmd/ateworker-microvm#${ATEWORKER_MV_IMG}#" \ | |
| deploy/substrate/counter-microvm-workerpool.yaml | kubectl --context kind-kind apply -f - | |
| hack/wait-worker-pool.sh counter-microvm | |
| - name: Stateful-tier + fork conformance (in-cluster Job) | |
| # Memory continuity across suspend/restore (the differentiator a plain pod cannot do), and | |
| # fork: fan out k children from one parent checkpoint and require each to continue at N+1 | |
| # independently, the claim that a stateful fork clones RAM instead of silently cold-booting. | |
| # A stateful session whose actor was deleted must be refused, not cold-created. | |
| # Then two sessions' calls through the router at once, each of which must reach its own actor. | |
| # The egress tests read back the policy installed at Create, resume and fork, and check that | |
| # a policy substrate refuses leaves no running compute. | |
| if: needs.changes.outputs.decision != 'skip' | |
| env: | |
| COUNTER_SANDBOX_CLASS: microvm | |
| run: hack/run-e2e-job.sh 'TestMemorySnapshotSuspendResume|TestMemorySessionWhoseActorIsLostIsRefused|TestFork|TestConcurrentSessionsReachTheirOwnActors|TestSessionDeletePendingOnMicroVM|TestEgressPolicy' "${E2E_IMAGE}" 2400 | |
| - name: Dump diagnostics on failure | |
| if: needs.changes.outputs.decision != 'skip' && failure() | |
| run: | | |
| df -h / || true | |
| kubectl --context kind-kind get workerpool,jobs,pods -A -o wide || true | |
| for ns in ate-agentsessions ate-system; do | |
| for p in $(kubectl --context kind-kind get pods -n "$ns" -o name 2>/dev/null); do | |
| echo "=== logs: $ns/$p ===" | |
| kubectl --context kind-kind logs -n "$ns" "$p" --all-containers --tail=200 2>/dev/null || true | |
| done | |
| done | |
| substrate-kind-example: | |
| name: substrate-kind example lifecycle | |
| needs: changes | |
| if: ${{ !cancelled() && needs.changes.outputs.decision != 'skip' }} | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 60 | |
| env: | |
| CLUSTER: sessions-example-ci | |
| SUBSTRATE_KIND_WORK: ${{ github.workspace }}/_local/substrate-kind-ci | |
| PROJECT: smoke | |
| MODEL_SOURCE: stub | |
| SANDBOX_CLASS: microvm | |
| steps: | |
| - name: Fail when the path filter failed | |
| if: needs.changes.result != 'success' | |
| run: | | |
| echo "::error::The path filter failed; refusing to skip lifecycle coverage." | |
| exit 1 | |
| - uses: actions/checkout@v7 | |
| - name: Setup Go | |
| uses: actions/setup-go@v7 | |
| with: | |
| go-version-file: integrations/substrate/go.mod | |
| cache-dependency-path: | | |
| go.sum | |
| integrations/substrate/go.sum | |
| - name: Setup ko | |
| uses: ko-build/setup-ko@v0.10 | |
| - name: Offline example regressions | |
| working-directory: examples/substrate-kind | |
| run: go test -race -shuffle=on ./... | |
| - name: Free runner disk space | |
| # This dedicated runner needs room for the Substrate release images and micro-VM assets. | |
| run: | | |
| df -h / | |
| sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL | |
| df -h / | |
| - name: Enable KVM | |
| run: | | |
| echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' \ | |
| | sudo tee /etc/udev/rules.d/99-kvm4all.rules | |
| sudo udevadm control --reload-rules | |
| sudo udevadm trigger --name-match=kvm | |
| - name: Require the example to bootstrap its own kind tool | |
| # Do not let a runner's preinstalled kind hide a missing-checkout bootstrap failure. | |
| run: | | |
| mkdir -p "${RUNNER_TEMP}/without-system-kind" | |
| cat >"${RUNNER_TEMP}/without-system-kind/kind" <<'EOF' | |
| #!/usr/bin/env bash | |
| echo "Unexpected system kind invocation; use the pinned checkout wrapper." >&2 | |
| exit 1 | |
| EOF | |
| chmod +x "${RUNNER_TEMP}/without-system-kind/kind" | |
| echo "${RUNNER_TEMP}/without-system-kind" >>"${GITHUB_PATH}" | |
| - name: Set up the example from scratch | |
| id: up | |
| run: bash examples/substrate-kind/up.sh | |
| - name: Reuse the owned cluster without changing its identity | |
| run: | | |
| cp "${SUBSTRATE_KIND_WORK}/${CLUSTER}/owner" "${RUNNER_TEMP}/owner-before" | |
| bash examples/substrate-kind/up.sh | |
| cmp "${RUNNER_TEMP}/owner-before" "${SUBSTRATE_KIND_WORK}/${CLUSTER}/owner" | |
| - name: Verify persistence and continuation across daemon replacement | |
| run: bash examples/substrate-kind/smoke.sh | |
| - name: Dump diagnostics before teardown on failure | |
| if: ${{ failure() && steps.up.outcome != '' && steps.up.outcome != 'skipped' }} | |
| run: | | |
| df -h / | |
| docker ps --all | |
| config="${SUBSTRATE_KIND_WORK}/${CLUSTER}/kubeconfig" | |
| kubectl --context "kind-${CLUSTER}" --kubeconfig "${config}" get workerpool,pods -A -o wide || true | |
| for ns in ate-system sessions-example; do | |
| for pod in $(kubectl --context "kind-${CLUSTER}" --kubeconfig "${config}" get pods -n "${ns}" -o name 2>/dev/null); do | |
| echo "=== logs: ${ns}/${pod} ===" | |
| kubectl --context "kind-${CLUSTER}" --kubeconfig "${config}" logs -n "${ns}" "${pod}" --all-containers --tail=200 || true | |
| done | |
| done | |
| - name: Tear down the example-owned cluster | |
| if: ${{ always() && steps.up.outcome != '' && steps.up.outcome != 'skipped' }} | |
| run: bash examples/substrate-kind/down.sh | |
| substrate-conformance: | |
| # Preserve the existing required status name while gating both independent live clusters. | |
| name: substrate-conformance | |
| needs: [changes, substrate-conformance-suite, substrate-kind-example] | |
| if: ${{ always() }} | |
| runs-on: ubuntu-latest | |
| env: | |
| FILTER_RESULT: ${{ needs.changes.result }} | |
| DECISION: ${{ needs.changes.outputs.decision }} | |
| SUITE_RESULT: ${{ needs.substrate-conformance-suite.result }} | |
| EXAMPLE_RESULT: ${{ needs.substrate-kind-example.result }} | |
| steps: | |
| - name: Require conformance and example lifecycle coverage | |
| run: | | |
| [[ "${FILTER_RESULT}" == success ]] || { echo "::error::Path filter failed."; exit 1; } | |
| [[ "${SUITE_RESULT}" == success ]] || { echo "::error::Conformance suite did not pass."; exit 1; } | |
| if [[ "${DECISION}" == skip && "${EXAMPLE_RESULT}" == skipped ]]; then | |
| exit 0 | |
| fi | |
| [[ "${EXAMPLE_RESULT}" == success ]] || { echo "::error::Example lifecycle did not pass."; exit 1; } |