Repository navigation
ci: restore sub-20-minute validation and make deep functional smoke opt-in #167
Description
Completed through normally merged PR #168: actual PR 13m04s, natural merged-main 9m03s, both successful. The requested beta.1 release followthrough continues through #152.
The owner policy amendment of 7 October 2026 requires routine PR/main validation to succeed in strictly less than 1,200 seconds (target 10–15 minutes), with functional acceptance performed explicitly by the owner.
Measured successful comparison:
| Run | Result | Workflow | .NET | Compile | Generic/visual |
|---|---|---|---|---|---|
| #539 | success | 16m27s | 14m47s | 1m16s | 4m10s |
| #549 | success | 32m50s | 32m39s | 1m41s | 27m52s |
| #565 | success | 37m58s | 37m49s | 2m22s | 32m14s |
| #572 | failure | 36m46s | 36m37s | 1m57s | 31m27s |
| #573 | cancelled | 40m20s | 40m11s | 2m36s | 34m17s |
API integration grew from 20 passing cases/45.638s to 67 passing cases/30m32.759s. Full NetRatel/published-RatelDesk pair fixtures and real rotation/recovery windows dominate this regression; compilation does not. Four rotation rows in #572 summed to 18m32.755s; overlapping test sums are not workflow wall time. Core rose from 2m29.844s to 11m19.730s while concurrent with API. Component tests remain cheap.
Implement a focused PR from current main a284782:
- Keep disclosure, version/source/PostgreSQL/release static checks, one visibly separate Release restore/build, three prebuilt fast Core/API/component assemblies, nonzero all-passing per-assembly TRX verification, and the existing meaningful aggregate.
- Explicitly classify the eight full-pair API classes as category=manual-integration and exclude compose, hosted, manual-integration from default selection. Preserve bodies/assertions and cheap protocol, canonical, payload, network, auth/PostgreSQL and provider regressions.
- Move browser/Chromium, full physical/native client, real-time recovery/rotation, deployment/upgrade and release-image functional smoke to one workflow_dispatch-only selected-suite workflow; remove redundant image/rehearsal work and the redundant provider execution.
- Apply the same selection to release-build while preserving native/CLI/stdio archives, licenses/SBOM/checksums, authenticated artifact receipts and hosted production.
- Remove promoter's automatic OIDC/MCP functional smoke calls. Preserve legacy verification.state consumers, source/checksum/digest/immutability/public-availability/recovery controls; label build/distribution integrity complete separately from functional acceptance pending owner testing.
- Update AGENTS/current CI instructions and existing selector/promotion checks. Normally integrate this policy into feat(flows): durable RatelDesk incident delivery through shared M2M #152 without reintroducing the old scheduler or mandatory evidence workload.
Acceptance: one final successful PR run and the natural successful merged-main run both strictly below 1,200 seconds from run_started_at through last required job completion. Record queue separately; include restore/build/test times, source/test-merge SHA, actual per-assembly selected/executed/pass/fail counts and separate sum of runner-job durations. Use clean standard hosted checkout; report cache honestly. Close only after merged-main success. No timeout, cancellation, skipped required execution, zero-test result, bypass or fake compatibility context counts as success.
Historical manual observations remain unresolved: #572 monitoring-clear timed out at 30s; the later #573 1.202s pass does not establish its cause. #573 physical incident hit its 300s timeout, and API had no final TRX when cancelled. Moving cadence does not claim those failures fixed.
Context: #152, #166, #141. This issue changes CI cadence and release integrity semantics; it does not mark unrelated feature issues complete or start beta.2. The already requested v0.1.1-beta.1 release continues after this fix.
Merged PR #168 implements the 7 October owner CI amendment at e3897c3d9c8362de23c006aa6cb259c07db7f50e. Routine validation now restores/builds once and runs the three prebuilt fast regression assemblies. Deep functional work has a separate, selected manual entry point.
| Required workflow | Result | Elapsed workflow time |
|---|---|---|
| Successful baseline #539 | Success | 16m27s |
| Successful newer main #565 | Success | 37m58s |
| Final CI PR #168 | Success | 13m04s (784s) |
| Natural post-merge main | Success | 9m03s (543s) |
The PR tested merge SHA was c958000b23217d775f30c5d6c50c3a311ea605fc. The natural main run is 37587846936, started at 2026-10-07T07:31:26Z, and its tested SHA is e3897c3d9c8362de23c006aa6cb259c07db7f50e. The owner’s acceptance criterion is actual success for both runs, each strictly below 1,200 seconds from workflow start through completion of the last required job. Pre-start queue time is reported separately and does not reduce workflow elapsed time.
| Timing | Final PR | Natural main |
|---|---|---|
| Pre-start queue | 0s | 0s |
| Sum of runner-job durations | 792s | 547s |
| Restore | 23s | 23s |
| Release build | 127s | 85s |
| Fast regression commands | 587s | 387s |
| Assembly | PR executed / passed / failed | PR command time | Main executed / passed / failed | Main command time |
|---|---|---|---|---|
Core (NetRatel.Tests) |
2827 / 2827 / 0 | 456s | 2827 / 2827 / 0 | 304s |
API (NetRatel.API.IntegrationTests) |
35 / 35 / 0 | 123s | 35 / 35 / 0 | 78s |
| Components | 368 / 368 / 0 | 8s | 368 / 368 / 0 | 5s |
The PR executed 3,230 passing tests with zero failures. The workflow retains source/disclosure/version/PostgreSQL/release-script checks and honest nonzero per-assembly TRX validation. Restore, compilation and fast-test outcomes are separately visible. It uses standard GitHub-hosted Ubuntu runners with no Actions build/NuGet cache configured. Optional functional smoke was not run by these automatic validations. Main execution/cache notes: fresh standard-runner checkout; restore and Release build actually executed, with no explicit NuGet/build-output cache or undeclared local prebuilt output.
Default test commands explicitly exclude category=compose, category=hosted and category=manual-integration. The new manual category covers eight full-pair API classes: ServiceLinkLifecycleTests, ServiceLinkOrchestrationRoundTripTests, ServiceLinkRotationHttpPostgresTests, ServiceLinkTerminalConvergencePostgresTests, ServiceLinkDatabaseConflictPostgresTests, ServiceLinkPayloadHttpPostgresTests, ServiceLinkWorkerConsentPostgresTests and ServiceLinkOutboundJournalPostgresTests. It also covers the four measured Core fixtures: LinuxInstallerTransactionTests, LinuxUpdaterRollbackTests, AgentGatewayRenewalTerminalTests and RemoteSupportReplicaSafeClusterPostgresTests. The default lane separately excludes full physical-client proof, sustained gateway/native install/update/PTY execution, real-time recovery/soak, the entire Playwright assembly and deployment/upgrade rehearsals. Cheap component, local API and focused PostgreSQL/security regressions remain selected; the redundant provider invocation and smoke-only image builds are removed.
The actual PR TRXs exclude all 12 newly manual classes and retain, among others, token-cache 70P, canonical JSON 56P, payload normalization 32P, dynamic network policy 8P, profile-token PostgreSQL expiry 13P, PostgreSQL provider 3P and ordinary gateway 9P.
Optional integration validation is workflow_dispatch only, with one selected group and no automatic dependencies or matrix fanout:
service-link,gateway,browser,deploymentrelease-images,native-client,physical-incidentupgrade-local,upgrade-oidc
For example: gh workflow run integration-validation.yml --repo BostonTechnologies/netratel --ref main -f suite=browser -f source_ref=main. Published-image and upgrade choices require an explicit existing completed release tag. Physical incident testing requires a selected source containing the original connector helper. Each requested group records the actual tested source and relevant peer/artifact identities, preserves existing assertions/timeouts/cleanup, and retains generated TRX, screenshots and receipts alongside its job logs. A requested failure remains a failure.
Release packaging stays on GitHub-hosted Actions. Exact tag/version/source, successful build receipts, five fixed image repositories, scans, immutable digests, anonymous pulls, checksums and partial-publication recovery remain required. Publication metadata now distinguishes completed build/distribution integrity from functional acceptance pending owner testing while retaining legacy consumer compatibility. Actual subsequent beta.1 packaging took 17m40s through its build aggregate; natural publication took 5m28s through its final job (5m29s to terminal metadata).
Historical functional observations remain unresolved: #572 timed out for 30 seconds waiting for monitoring-clear; #573 later passed all 87 browser cases, which does not establish the earlier cause. The dedicated physical incident hit its 300-second timeout, and the cancelled 40-minute .NET run had no completed generic API TRX. Moving these checks to manual selection does not turn those results into passes or establish a product fix.
Both actual natural runs are verified successful below 1,200 seconds, so the focused CI issue is complete. The policy has also been normally merged into connector PR #152 at 979e832fb0b07fafe6ce922f48442bbb64a05e5a, with fresh validation. Merge/release followthrough must use its actual results and preserve its feature fixes before epic #141's requested v0.1.1-beta.1 hosted build/publication. Owner functional acceptance remains pending and non-gating; beta.2 remains separate work.
Release follow-through completed — 7 October 2026
Connector #152 was normally merged at 8cb37bcaf5f5a2e290327fb38efb185eabf44391. Its final natural Main validation passed in 13m32s with 3,156 Core + 35 API + 400 Components passes, zero failed/skipped cases. The exact annotated tag v0.1.1-beta.1 was built and normally published once: six-job build, seven-job publication, public release and owner guide. All 13 public download byte/digest/checksum records, ten strict build-provenance file verifications, authenticated derived Compose binding and five anonymous OCI identities passed. The five prior stable image digests and stable/latest release remain unchanged. Owner functional acceptance is still pending; no optional suite was run as an automatic release gate.
Activity