Skip to content

Commit a23dccf

Browse files
authored
Merge branch 'main' into tuntu/fix-phabricator-http-errors
2 parents 7be48ac + cc185e0 commit a23dccf

53 files changed

Lines changed: 2626 additions & 128 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -269,6 +269,34 @@ jobs:
269269
if: ${{ failure() }}
270270
uses: ./.github/actions/upload-testlogs
271271

272+
validator-integration-test:
273+
name: Validator Extension Test
274+
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.draft == false }}
275+
runs-on: ubuntu-latest
276+
steps:
277+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
278+
with:
279+
# This job executes untrusted PR code (make build/test/lint). Don't
280+
# leave the GITHUB_TOKEN in the workspace git config while it runs.
281+
persist-credentials: false
282+
- uses: ./.github/actions/setup
283+
284+
- name: Run validator extension tests
285+
env:
286+
# Opens pull requests in the test repository; see the secrets guard
287+
# in workflow-security.
288+
SQ_GITHUB_TOKEN: ${{ secrets.SQ_TEST_REPO_TOKEN }}
289+
SQ_GITHUB_TEST_REPO: ${{ vars.SQ_GITHUB_TEST_REPO }}
290+
# Fail rather than skip when the secret is missing, but only in a
291+
# repository that set up a test repo (forks without one skip), and
292+
# not on fork PRs, which GitHub never gives secrets to.
293+
SQ_GITHUB_TEST_REQUIRED: ${{ vars.SQ_GITHUB_TEST_REPO != '' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) }}
294+
run: make integration-test-submitqueue-validator
295+
296+
- name: Upload Bazel failure logs
297+
if: ${{ failure() }}
298+
uses: ./.github/actions/upload-testlogs
299+
272300
storage-integration-test:
273301
name: Storage Extension Test
274302
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.draft == false }}
@@ -346,10 +374,10 @@ jobs:
346374
# .github/. GITHUB_TOKEN (least-privilege, read-only here) is allowlisted.
347375
#
348376
# SQ_TEST_REPO_TOKEN is the one deliberate exception. The live GitHub
349-
# tests (e2e and merger extension) merge real pull requests in a separate
350-
# test repository, which GITHUB_TOKEN cannot reach. It is a fine-grained
351-
# token limited to that repository, so a PR that exfiltrated it could only
352-
# reach that repository.
377+
# tests (e2e, merger and validator extension) open and merge real pull
378+
# requests in a separate test repository, which GITHUB_TOKEN cannot
379+
# reach. It is a fine-grained token limited to that repository, so a PR
380+
# that exfiltrated it could only reach that repository.
353381
- name: Guard — no repository secrets on the untrusted-code path
354382
run: |
355383
hits="$(grep -rnE '\$\{\{[^}]*secrets\.' \
@@ -387,6 +415,7 @@ jobs:
387415
- counter-integration-test
388416
- queue-integration-test
389417
- merger-integration-test
418+
- validator-integration-test
390419
- storage-integration-test
391420
- consumer-integration-test
392421
- web

‎Makefile‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@ define assert_clean
155155
fi
156156
endef
157157

158-
.PHONY: build build-all-linux build-runway-linux build-submitqueue-gateway-client build-submitqueue-gateway-linux build-submitqueue-gateway-server build-submitqueue-orchestrator-linux build-stovepipe-linux build-stovepipe-linux-debug check-gazelle check-mocks check-tidy clean clean-proto demo-requests deps docs-build docs-serve e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-binary lint-fmt lint-license local-init-runway-queue-schema local-init-stovepipe-schemas local-runway-start local-runway-stop local-submitqueue-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-debug-start local-stovepipe-logs local-stovepipe-start local-stovepipe-stop mocks proto query-deps query-targets run-client-runway run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe run-queue-admin test test-no-cache test-race tidy tidy-bazel tidy-go web-build web-check web-e2e-test web-image-load web-install web-proto help
158+
.PHONY: build build-all-linux build-runway-linux build-submitqueue-gateway-client build-submitqueue-gateway-linux build-submitqueue-gateway-server build-submitqueue-orchestrator-linux build-stovepipe-linux build-stovepipe-linux-debug check-gazelle check-mocks check-tidy clean clean-proto demo-requests deps docs-build docs-serve e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator integration-test-submitqueue-validator license-fix lint lint-binary lint-fmt lint-license local-init-runway-queue-schema local-init-stovepipe-schemas local-runway-start local-runway-stop local-submitqueue-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-debug-start local-stovepipe-logs local-stovepipe-start local-stovepipe-stop mocks proto query-deps query-targets run-client-runway run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe run-queue-admin test test-no-cache test-race tidy tidy-bazel tidy-go web-build web-check web-e2e-test web-image-load web-install web-proto help
159159

160160

161161
build: ## Build all services and examples
@@ -320,6 +320,10 @@ integration-test-submitqueue-orchestrator: ## Run Orchestrator integration tests
320320
@echo "Running Orchestrator integration tests..."
321321
@$(BAZEL) test //test/integration/submitqueue/orchestrator:go_default_test --test_output=streamed
322322

323+
integration-test-submitqueue-validator: ## Run the GitHub validator extension test (needs SQ_GITHUB_TOKEN and SQ_GITHUB_TEST_REPO=owner/repo, else skips)
324+
@echo "Running GitHub validator extension tests..."
325+
@$(BAZEL) test //test/integration/submitqueue/extension/validator/... --test_output=errors
326+
323327
land: ## Land a change or a stack (PR=<url>, PRS="<url> <url>", or URI=<change-uri>)
324328
@if [ -z "$(PR)$(PRS)$(URI)$(URIS)" ]; then \
325329
echo "Usage: make land PR=https://github.com/owner/repo/pull/7"; \

‎api/stovepipe/proto/stovepipe.proto‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,7 @@ message GetProjectStatusByIDRequest {
116116
string queue = 1;
117117
// Canonical decimal request identifier returned from Ingest.
118118
string request_id = 2;
119-
// Required consumer-defined project IDs whose recorded results are requested.
119+
// Optional consumer-defined project IDs. Empty returns all recorded project results.
120120
repeated string projects = 3;
121121
// Maximum requested projects to return in one page. Zero selects the server default.
122122
int32 page_size = 4;
@@ -130,7 +130,7 @@ message GetProjectStatusByURIRequest {
130130
string queue = 1;
131131
// Exact VCS-agnostic URI of the commit under validation.
132132
string change_uri = 2;
133-
// Required consumer-defined project IDs whose recorded results are requested.
133+
// Optional consumer-defined project IDs. Empty returns all recorded project results.
134134
repeated string projects = 3;
135135
// Maximum requested projects to return in one page. Zero selects the server default.
136136
int32 page_size = 4;

‎api/stovepipe/protopb/stovepipe.pb.go‎

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎doc/howto/TESTING.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ make build-all-linux # Build Linux binaries for the local docker-
9494

9595
### Live GitHub Tests
9696

97-
Two suites land real pull requests on github.com: `TestGitHubLandE2E` (in `make e2e-test`) and the Runway GitHub merger extension test (`make integration-test-runway-merger`). They run only when `SQ_GITHUB_TOKEN` and `SQ_GITHUB_TEST_REPO=owner/repo` are set, and skip otherwise, so every other run is unaffected. The token needs write access to the test repository; everything the tests open is named under `sq-it/` and removed afterwards, while what they merge stays on its default branch.
97+
Three suites open real pull requests on github.com: `TestGitHubLandE2E` (in `make e2e-test`), which lands them through the whole stack with the GitHub change provider, validator and merger; the Runway GitHub merger extension test (`make integration-test-runway-merger`); and the GitHub validator extension test (`make integration-test-submitqueue-validator`). They run only when `SQ_GITHUB_TOKEN` and `SQ_GITHUB_TEST_REPO=owner/repo` are set, and skip otherwise, so every other run is unaffected. The token needs write access to the test repository; everything the tests open is named under `sq-it/` and removed afterwards, while what they merge stays on its default branch.
9898

9999
```bash
100100
SQ_GITHUB_TOKEN=$(gh auth token) SQ_GITHUB_TEST_REPO=<owner>/<repo> make e2e-test
@@ -107,7 +107,7 @@ CI takes the repository from the `SQ_GITHUB_TEST_REPO` repository variable and t
107107
Any contributor can run these suites against a repository of their own:
108108

109109
1. Create a repository on github.com with a default branch (an initial commit is enough). It must allow squash and rebase merges, and its default branch must not require reviews or status checks, since the tests merge directly.
110-
2. Create a token that can write to it: a fine-grained token limited to that repository with read and write access to contents, pull requests and issues, or simply `gh auth token`.
110+
2. Create a token that can write to it: a fine-grained token limited to that repository with read and write access to contents, pull requests and issues and read access to checks and commit statuses, or simply `gh auth token`.
111111
3. Run locally with `SQ_GITHUB_TOKEN` and `SQ_GITHUB_TEST_REPO` set, as above.
112112
4. For CI in your fork, set the `SQ_GITHUB_TEST_REPO` repository variable and the `SQ_TEST_REPO_TOKEN` repository secret in the fork's Actions settings. Runs in the fork then exercise your test repository.
113113

‎doc/rfc/stovepipe/project-status-api.md‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ The request and response fields have the following API-level meaning. The publis
1717
| `queue` | Yes | Identifies the queue in which to find the validation request. |
1818
| `request_id` | ID lookup | Canonical positive decimal request ID returned by `Ingest`. |
1919
| `change_uri` | URI lookup | Identifies the exact commit URI under validation. |
20-
| `projects` | Yes | Consumer-defined project IDs whose results are requested. At least one ID is required. |
20+
| `projects` | No | Consumer-defined project IDs whose results are requested. Omitted or empty returns all recorded project results. |
2121
| `page_size` | No | Limits one page of results for the requested projects. |
2222
| `page_token` | No | Continues a previous page of results for the requested projects. |
2323

@@ -29,7 +29,7 @@ The request and response fields have the following API-level meaning. The publis
2929
| `updated_at_ms` | Records, in Unix milliseconds, the newest durable lifecycle or result record represented by the response. |
3030
| `repository_breakage_degree` | Returns the whole-repository result when it is durably recorded. |
3131
| `project_results_complete` | Indicates whether the implementation has durably finished producing results for the supplied project IDs. |
32-
| `projects` | Returns recorded results for requested project IDs. |
32+
| `projects` | Returns recorded results for requested project IDs, or all recorded project results when none are supplied. |
3333
| `next_page_token` | Continues requested-project result pagination when another page exists. |
3434

3535
`request_state` is a stable public lifecycle vocabulary: `accepted`, `processing`, `succeeded`, `failed`, `cancelled`, or `superseded`. Clients must tolerate a future value. A terminal request state does not by itself mean that project results are complete.
@@ -56,25 +56,27 @@ Both lookups are queue-scoped and use the same durable request summary and valid
5656

5757
The request-URI mapping admits one request per `(queue, change_uri)`, so the endpoint returns one authoritative request. Revalidation support must introduce an explicit authoritative-request rule; it must not silently change this lookup's meaning.
5858

59-
Queue, URI, request ID, and project ID comparisons are byte-exact and are limited to 255 bytes. An empty queue, lookup selector, project list, or project ID is invalid. A request cannot contain duplicate project IDs. `page_size=0` selects the default of 50; the maximum is 200. Request IDs use the same canonical positive decimal format as `Ingest` and request-history lookups. Page tokens are opaque and bound to the selected request and requested project list; either endpoint can continue a page for the same queue, request, and projects.
59+
Queue, URI, request ID, and project ID comparisons are byte-exact and are limited to 255 bytes. An empty queue, lookup selector, or individual project ID is invalid. A request cannot contain duplicate project IDs. `page_size=0` selects the default of 50; the maximum is 200. Request IDs use the same canonical positive decimal format as `Ingest` and request-history lookups. Page tokens are opaque and bound to the selected request and requested project list; either endpoint can continue a page for the same queue, request, and projects.
6060

6161
## Project Results and Pagination
6262

63-
The caller supplies the project IDs for which it wants recorded results. The API does not define project discovery, validation selection, or attribution. A project absent from the response has no universal meaning and consumers must not infer that it is green.
63+
The caller may supply the project IDs for which it wants recorded results. The API does not define project discovery, validation selection, or attribution. A project absent from the response has no universal meaning and consumers must not infer that it is green.
6464

65-
The response includes only results for the supplied project IDs, in the supplied order, and paginates that list. An implementation may record results for every requested project, only requested projects with a non-green result, or another documented subset.
65+
When project IDs are supplied, the response includes only results for those IDs, in the supplied order, and paginates that list. An implementation may record results for every requested project, only requested projects with a non-green result, or another documented subset.
66+
67+
With no project IDs supplied, the response lists all recorded project results in project-ID order, excluding the repository fact. Results are a live view; restart pagination to discover facts recorded behind the cursor.
6668

6769
Each project result is an immutable validation fact keyed by queue, commit URI, and project ID. A result must identify the selected request. `project_results_complete` is false until the implementation has durably finished producing results for the supplied project IDs; a completed empty set is valid.
6870

6971
Callers use `project_results_complete`, rather than the presence or absence of an individual project result, to determine whether the implementation has finished producing results for the requested projects. `next_page_token` is empty only on the final page. A terminal lifecycle notification is emitted only after the repository result and any applicable project completion record are durable. If notification delivery fails, consumers can recover the same result through either endpoint.
7072

7173
## Implementation
7274

73-
Both endpoints return durable lifecycle summaries, the whole-repository fact when present, and any recorded facts for requested projects in paginated request order. `project_results_complete` remains false until a durable completion projection is available. Missing facts are omitted; an explicitly recorded zero degree remains present. `updated_at_ms` is the maximum timestamp of the summary and the facts represented on the page.
75+
Both endpoints return durable lifecycle summaries, the whole-repository fact when present, and recorded facts for requested projects in paginated request order, or all recorded project results in project-ID order when none are requested. `project_results_complete` remains false until a durable completion projection is available. Missing facts are omitted; an explicitly recorded zero degree remains present. `updated_at_ms` is the maximum timestamp of the summary and the facts represented on the page.
7476

7577
## Errors and Authorization
7678

77-
- Invalid selectors, an empty project list, an invalid project ID, duplicate project IDs, or malformed page tokens are user errors.
79+
- Invalid selectors, an invalid project ID, duplicate project IDs, or malformed page tokens are user errors.
7880
- An unknown queue-scoped request ID or commit URI is not found.
7981
- A request-URI mapping whose summary is not visible is retryable.
8082
- A selector and summary that disagree, a result belonging to another request, or a completion record with a missing project result is an internal consistency error.

‎service/submitqueue/demo/provider/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Each directory here is one **provider** — a code-hosting system SubmitQueue la
44

55
| File | Selects |
66
|---|---|
7-
| `profiles.yaml` | the change provider, build runner, conflict analyzer, scorer and build budget each queue resolves to (read by the orchestrator) |
7+
| `profiles.yaml` | the change provider, validator, build runner, conflict analyzer, scorer and build budget each queue resolves to (read by the orchestrator) |
88
| `merge.yaml` | the merge target each queue lands on (read by Runway) |
99

1010
Neither holds a secret. Each integration names the *environment variable* carrying its credential, so these files stay committable and rotating a token needs no edit.

‎service/submitqueue/orchestrator/server/BUILD.bazel‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ go_library(
6868
"//submitqueue/extension/speculation/speculator/standard:go_default_library",
6969
"//submitqueue/extension/validator:go_default_library",
7070
"//submitqueue/extension/validator/fake:go_default_library",
71+
"//submitqueue/extension/validator/github:go_default_library",
7172
"//submitqueue/orchestrator:go_default_library",
7273
"//submitqueue/orchestrator/extension/storage:go_default_library",
7374
"//submitqueue/orchestrator/extension/storage/mysql:go_default_library",
@@ -144,6 +145,7 @@ go_test(
144145
"//submitqueue/extension/queuepause/fake:go_default_library",
145146
"//submitqueue/extension/speculation/scorer:go_default_library",
146147
"//submitqueue/extension/speculation/speculator:go_default_library",
148+
"//submitqueue/extension/validator:go_default_library",
147149
"//submitqueue/orchestrator/extension/storage:go_default_library",
148150
"@com_github_stretchr_testify//assert:go_default_library",
149151
"@com_github_stretchr_testify//require:go_default_library",

0 commit comments

Comments
 (0)