feat(stovepipe): define repair coordination contracts and RFC

Author: mnoah1

Makefile

@@ -629,6 +629,7 @@ GO_GENERATE_PACKAGES := \
 	./stovepipe/core/requestlog/... \
 	./stovepipe/extension/buildissue/... \
 	./stovepipe/extension/buildrunner/... \
+	./stovepipe/extension/fixer/... \
 	./stovepipe/extension/projectstatus/... \
 	./stovepipe/extension/queueconfig/... \
 	./stovepipe/extension/sourcecontrol/... \

doc/rfc/stovepipe/fixer.md

@@ -0,0 +1,45 @@
+# First-failure repair submission
+
+Build analysis records confirmed failures. The `fix` queue step selects the first eligible failed unit for each build and submits one repair through a Go extension. Submission can happen while the build is running. See the [workflow](workflow.md#workflow).
+
+## 1. Publish a confirmed failure
+
+[`buildanalysis.Controller.Process`](../../../stovepipe/controller/buildanalysis/buildanalysis.go) obtains confirmed [`BuildIssue`](../../../stovepipe/entity/build_issue.go) observations from [`buildissue.Detector`](../../../stovepipe/extension/buildissue/buildissue.go). The detector owns provider confirmation and retry rules.
+
+The build-analysis integration persists the per-build observation through [`BuildIssueStore`](../../../stovepipe/extension/storage/build_issue_store.go) and reconciles its [`OpenIssue`](../../../stovepipe/entity/open_issue.go). Each continuous opening retains its immutable `FirstSeenBuildID`. A recurrence after recovery has a new opening build. Detecting builds and opening builds are separate identities.
+
+Publish [`FixRequest`](../../../stovepipe/core/messagequeue/proto/fix.proto) with the queue, detecting BuildID, TaskID, UnitID, and FirstSeenBuildID. Partition by detecting BuildID. Replay publication after duplicate observation writes so interrupted handoffs can finish.
+
+## 2. Select one issue per build
+
+[`fix.Controller.Process`](../../../stovepipe/controller/fix/fix.go) reloads the Build and Request and checks queue identity. For a new attempt, it verifies the persisted failure and current issue opening.
+
+1. Create a queued [`Fix`](../../../stovepipe/entity/fix.go) using the detecting BuildID as its queue-scoped ID. Create-if-absent selects the first eligible request processed, including when deliveries race.
+2. If that key already exists, reload its saved selection. Preparation retries use the saved issue, even when another target caused the delivery.
+3. Before dispatch, recheck the selected opening. If it recovered or reopened, mark the queued attempt skipped and retain the build's selection.
+
+Build analysis continues collecting all failures. Later deliveries reconcile the saved submission and its history. Each build has its own initial attempt; cross-build deduplication and decisions about additional root causes are follow-up work.
+
+## 3. Submit once
+
+[`RepairExecutor.Submit`](../../../stovepipe/extension/fixer/fixer.go) accepts `SubmitRequest{BuildID, Issue}` and returns a nonempty opaque provider reference or an error. The executor is bound to its queue by its factory. Resolve it before claiming dispatch.
+
+Conditionally persist `FixSubmitting` before calling Submit. Only the delivery whose conditional write succeeds calls the executor. Redelivery after that marker reconciles history and finishes without another trigger.
+
+On acceptance, persist `FixSubmitted` and `ExecutionRef`. On an error or empty reference, persist `FixSubmissionUnknown` and the diagnostic in metadata. A trigger error can follow acceptance, so it leaves execution unknown. A failed result write leaves the submitting marker.
+
+This is at-most-once dispatch. A crash between the durable marker and the outbound call can leave the repair unlaunched. A lost response leaves acceptance unknown. These cases require explicit follow-up. Execution monitoring can be added separately.
+
+## 4. Record request history and provenance
+
+Record `fix_submission_started` for the durable dispatch claim and `fix_submission_recorded` for a retained accepted or uncertain trigger result. Stable occurrence IDs derive from the fix ID. Replay reconstructs missing history from the saved Fix, including when a later target supplies the delivery. History failures never repeat submission. An attempt left submitting has only a started event.
+
+[`FixStore`](../../../stovepipe/extension/storage/fix_store.go) adds one table with primary-key create, get, and conditional update. It retains the detecting BuildID, issue-opening references, submission state, optional provider reference and metadata, timestamps, and version. Build and OpenIssue keep their existing schemas.
+
+Navigate `Fix.BuildID → Build.RequestID → Request.URI` for the initiating request and change. `Fix.Issues` retains opening identities after recovery. Build analysis owns failure and recovery evidence.
+
+## Integration
+
+[`core/fixrequest.Publish`](../../../stovepipe/core/fixrequest/publish.go) is the handoff after observation and positive open-issue persistence. The build-analysis owner supplies reconciliation and calls the helper, including on replay. The reference host retains its no-op detector until that integration lands.
+
+The initial flow ends after submission and request-history recording. Execution monitoring, recovery correlation, and decisions about related failures can build on these records later.

doc/rfc/stovepipe/workflow.md

@@ -169,6 +169,8 @@ Per-stage design detail lives under `steps/` so this doc stays a pipeline overvi
 - [buildsignal.md](steps/buildsignal.md) — the poll loop: hold-based re-poll cadence, per-build partitioning, and the fail-closed handoff to record
 - [record.md](steps/record.md) — turning a terminal build outcome into an immutable validation fact, monotonic last-green advancement and ref promotion, and the hook event announcing the outcome. Project analysis already runs inline through `projectstatus.Resolver` and may later use results calculated by build analysis
 
+For repair decisions, execution, and recovery, see the [repair RFC](fixer.md).
+
 ## Dedup, idempotency, and history rewrites
 
 Ingestion is idempotent on `(Queue, head URI)`, so duplicate poller reports — and any future webhook producer racing the poller — converge on one Request. The pipeline persists the Queue's last-green URI and per-URI greenness, so it must tolerate a **history rewrite**: when `SourceControl` reports that the last-green URI is no longer an ancestor of the current head, `process` falls back to a full-monorepo build rather than trusting a baseline that no longer exists on the branch. The system converges to a correct greenness rather than wedging on a stale pointer.

stovepipe/core/messagequeue/README.md

@@ -11,3 +11,9 @@ Payloads are defined in proto3 (`proto/`, generated into `protopb/`) and seriali
 - **buildsignal** (`TopicKeyBuildSignal`, `BuildSignal`) — build publishes the build id here after triggering; buildsignal holds the delivery between polls until the build reaches a terminal status. Partitioned by build id, so each build's poll loop is an independent partition.
 - **buildanalysis** (`TopicKeyBuildAnalysis`, `BuildAnalysis`) — identifies a persisted build for issue detection, partitioned by build id. Build publishes a start notice and buildsignal publishes a completion notice with a distinct message id. See [build analysis](../../controller/buildanalysis/README.md) for polling and persistence behavior.
 - **record** (`TopicKeyRecord`, `Record`) — buildsignal publishes the request id after persisting the terminal outcome; record reloads the request and records its validation fact. Partitioned by request id.
+
+## Repair stage contract
+
+**fix** (`TopicKeyFix`, [`FixRequest`](proto/fix.proto)) identifies a persisted observation and issue opening using queue, detecting build, task, unit, and first-seen build. Partition by detecting build ID. The `consumer.Controller` selects the first eligible failed unit by creating a Fix keyed by the detecting BuildID, persists a submitting marker, and calls `RepairExecutor.Submit` once. Later deliveries reconcile the selected submission and its request history.
+
+Persist observations and their current opening before publishing a request. Replay publication after duplicate observation writes so a crash between persistence and handoff can recover. Redelivery can complete preparation, but a durable submitting marker suppresses any further outbound call. Execution monitoring and verified recovery are follow-up work.

stovepipe/core/messagequeue/messagequeue.go

@@ -56,6 +56,9 @@ type (
 	// Record is the payload buildsignal publishes to the record stage once a
 	// build reaches a terminal status: the build id to record.
 	Record = protopb.Record
+
+	// FixRequest identifies a confirmed issue opening and the detecting build.
+	FixRequest = protopb.FixRequest
 )
 
 // marshalOpts keeps the JSON field names identical to the proto field names

stovepipe/core/messagequeue/messagequeue_test.go

@@ -66,6 +66,40 @@ func TestRecordRoundTrip(t *testing.T) {
 	assert.True(t, proto.Equal(rec, got), "round-tripped Record should equal the original")
 }
 
+func TestFixMessagesRoundTrip(t *testing.T) {
+	for _, tt := range []struct {
+		name    string
+		message proto.Message
+	}{
+		{
+			name: "issue request",
+			message: &FixRequest{
+				QueueName: "repo/main",
+				BuildId:   "build-42", TaskId: "compile", UnitId: "package", FirstSeenBuildId: "build-41",
+			},
+		},
+	} {
+		t.Run(tt.name, func(t *testing.T) {
+			data, err := Marshal(tt.message)
+			require.NoError(t, err)
+			got := tt.message.ProtoReflect().New().Interface()
+			require.NoError(t, Unmarshal(data, got))
+			assert.True(t, proto.Equal(tt.message, got))
+		})
+	}
+}
+
+func TestFixRequestWireFormat(t *testing.T) {
+	message := &FixRequest{QueueName: "repo/main", BuildId: "build-42", TaskId: "check-a", FirstSeenBuildId: "build-41"}
+	data, err := Marshal(message)
+	require.NoError(t, err)
+	assert.JSONEq(t, `{"queue_name":"repo/main","build_id":"build-42","task_id":"check-a","first_seen_build_id":"build-41"}`, string(data))
+
+	var got FixRequest
+	require.NoError(t, Unmarshal([]byte(`{"queue_name":"repo/main","build_id":"build-42","task_id":"check-a","first_seen_build_id":"build-41","future_field":true}`), &got))
+	assert.True(t, proto.Equal(message, &got))
+}
+
 // TestWireFormat locks the protojson encoding decision the contract relies on:
 // snake_case field names (UseProtoNames).
 func TestWireFormat(t *testing.T) {
@@ -80,7 +114,7 @@ func TestWireFormat(t *testing.T) {
 // no topic_keys option names an unknown key.
 func TestTopicKeysBindEveryTopicKey(t *testing.T) {
 	bound := map[string]int{}
-	for _, m := range []proto.Message{&ProcessRequest{}, &BuildRequest{}, &BuildSignal{}, &BuildAnalysis{}, &Record{}} {
+	for _, m := range []proto.Message{&ProcessRequest{}, &BuildRequest{}, &BuildSignal{}, &BuildAnalysis{}, &Record{}, &FixRequest{}} {
 		keys := TopicKeys(m)
 		require.NotEmpty(t, keys, "message must declare a non-empty topic_keys option")
 		for _, key := range keys {
@@ -94,6 +128,7 @@ func TestTopicKeysBindEveryTopicKey(t *testing.T) {
 		TopicKeyBuildSignal,
 		TopicKeyBuildAnalysis,
 		TopicKeyRecord,
+		TopicKeyFix,
 	}
 
 	valid := map[string]bool{}

stovepipe/core/messagequeue/proto/BUILD.bazel

@@ -3,6 +3,7 @@ exports_files(
         "build.proto",
         "buildanalysis.proto",
         "buildsignal.proto",
+        "fix.proto",
         "process.proto",
         "record.proto",
     ],

stovepipe/core/messagequeue/proto/fix.proto

@@ -0,0 +1,36 @@
+// Copyright (c) 2026 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+package uber.stovepipe.messagequeue;
+
+import "api/base/messagequeue/proto/messagequeue.proto";
+
+option go_package = "github.com/uber/submitqueue/stovepipe/core/messagequeue/protopb";
+option java_multiple_files = true;
+option java_outer_classname = "FixRequestProto";
+option java_package = "com.uber.submitqueue.stovepipe.messagequeue";
+
+// FixRequest identifies a confirmed issue opening and the build requesting repair.
+// The observation and open issue must be persisted before publication.
+message FixRequest {
+    option (uber.base.messagequeue.topic_keys) = "fix";
+
+    string queue_name = 1;
+    string build_id = 2;
+    string task_id = 3;
+    string unit_id = 4;
+    string first_seen_build_id = 5;
+}

stovepipe/core/messagequeue/protopb/BUILD.bazel

@@ -6,6 +6,7 @@ go_library(
         "build.pb.go",
         "buildanalysis.pb.go",
         "buildsignal.pb.go",
+        "fix.pb.go",
         "process.pb.go",
         "record.pb.go",
     ],