Skip to content

Commit d37bfce

Browse files
committed
feat(stovepipe): Orchestrate issue checkpoints
1 parent c76be01 commit d37bfce

8 files changed

Lines changed: 868 additions & 2 deletions

File tree

‎stovepipe/controller/issuetrack/BUILD.bazel‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,11 @@ go_library(
77
visibility = ["//visibility:public"],
88
deps = [
99
"//platform/base/id:go_default_library",
10+
"//platform/base/messagequeue:go_default_library",
1011
"//platform/consumer:go_default_library",
1112
"//platform/metrics:go_default_library",
1213
"//stovepipe/core/loader:go_default_library",
14+
"//stovepipe/core/messagequeue:go_default_library",
1315
"//stovepipe/entity:go_default_library",
1416
"//stovepipe/extension/storage:go_default_library",
1517
"@com_github_uber_go_tally//:go_default_library",
@@ -19,9 +21,15 @@ go_library(
1921

2022
go_test(
2123
name = "go_default_test",
22-
srcs = ["issuetrack_test.go"],
24+
srcs = [
25+
"delivery_test.go",
26+
"issuetrack_test.go",
27+
],
2328
embed = [":go_default_library"],
2429
deps = [
30+
"//platform/base/messagequeue:go_default_library",
31+
"//platform/consumer/mock:go_default_library",
32+
"//platform/publish:go_default_library",
2533
"//stovepipe/core/messagequeue:go_default_library",
2634
"//stovepipe/entity:go_default_library",
2735
"//stovepipe/extension/storage:go_default_library",

‎stovepipe/controller/issuetrack/README.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,13 @@ After all issue and history writes succeed, the controller saves the last fully
1212

1313
Build issues added behind a refresh's cursor are picked up by a later refresh, which starts from the beginning. Before requesting finalization, the producer must successfully detect and persist the build's complete issue set. Finalization uses all issues accumulated for that build, not just the latest detector response.
1414

15-
This layer does not receive messages, publish repair requests, register consumers, or implement recovery. Delivery orchestration is added separately. A future repair handoff must persist what the repair needs and publish before saving a cursor that would skip that work on retry.
15+
## Delivery orchestration
16+
17+
Notifications carry a queue, build ID, and finalization flag. They must share a queue partition and consumer group. A delivery resumes pending work before admitting another source. When no pass is pending, request ingest order rejects older sources; an already-finalized request cannot accept more work. A completed owning notification is also ignored on replay.
18+
19+
Every accepted pass saves its owning message ID and starts with an empty cursor. Each delivery applies one batch and saves the checkpoint. Pending passes hold for immediate redelivery. Completed passes acknowledge their owning message; a different notification that helped finish them holds once more so its own source can be considered. Source ordering is rechecked after conditional-write conflicts.
20+
21+
This controller is not registered in the reference host. Repair publication and terminal recovery remain follow-ups. A future repair handoff must persist its prerequisites and publish before advancing the checkpoint past that obligation. Permanently failing pending work requires recovery before issue tracking can advance.
1622

1723
## Ownership prerequisite
1824

0 commit comments

Comments
 (0)