diff --git a/docs/04-Modules/04-Payments.md b/docs/04-Modules/04-Payments.md index 0743261a..b28186e5 100644 --- a/docs/04-Modules/04-Payments.md +++ b/docs/04-Modules/04-Payments.md @@ -6,6 +6,10 @@ Formance Payments requires: - **PostgreSQL**: See configuration guide [here](../05-Infrastructure%20services/01-PostgreSQL.md). - (Optional) **Broker**: See configuration guide [here](../05-Infrastructure%20services/02-Message%20broker.md). +Payments cannot be installed alongside Ledger v3 (including prereleases). +Remove the Payments module before switching the stack's Ledger from v2 to v3. +Stacks without a Ledger module remain supported. + ## Payments Object :::info diff --git a/docs/10-Development/02-Module dependencies.md b/docs/10-Development/02-Module dependencies.md index 9575e40a..1f7a1fb1 100644 --- a/docs/10-Development/02-Module dependencies.md +++ b/docs/10-Development/02-Module dependencies.md @@ -12,12 +12,13 @@ Some module reconcilers currently coordinate with another object in the same Sta - Connectivity requires a ready Ledger whose effective version supports the Ledger v3 integration; - future versions of a module may support only a bounded range of another module's versions. -The first implementation records seven confirmed Ledger relationships: +The first implementation records eight confirmed Ledger relationships: - Connectivity declares Ledger presence in the shared requirements layer, then its reconciler requires the effective Ledger version to be a semantic major v3 (`>= v3.0.0-0` and `< v4.0.0-0`); +- Payments permits Ledger absence but requires Ledger `< v3.0.0-0` when present. - MCP, Orchestration (Flows), Reconciliation, TransactionPlane, Wallets, and Webhooks require Ledger `< v3.0.0-0`. -All seven relationships require presence. The six legacy consumers also declare their version constraint in the shared layer. Connectivity keeps its exact version and readiness gates in its reconciler because a non-SemVer reference must trigger its module-specific hard teardown, whereas the shared version evaluator correctly classifies such a reference as unresolved. The other six modules do not add a readiness gate in this change. +Seven relationships require presence. Payments uses `IfPresent()` to preserve stacks without Ledger, while requiring Ledger `< v3.0.0-0` when installed. All seven legacy consumers declare their version constraint in the shared layer. Connectivity keeps its exact version and readiness gates in its reconciler because a non-SemVer reference must trigger its module-specific hard teardown, whereas the shared version evaluator correctly classifies such a reference as unresolved. The seven legacy modules do not add a readiness gate in this change. These rules are currently implicit in individual reconcilers. Each caller is responsible for finding the dependency, resolving versions, checking readiness, installing watches, reporting errors, and deciding whether reconciliation can continue. The resulting behaviour is difficult to discover and is not consistent across modules. @@ -59,13 +60,13 @@ Making `Requirements(...)` or `NoRequirements()` a required argument ensures tha ### Requirement -A **requirement** is a condition declared by a module about one other Stack-bound object. The first implementation supports: +A **requirement** is a condition declared by a module about one other Stack-bound object. The requirements model supports: -- **presence**: exactly one matching, non-deleting object exists in the same Stack; +- **presence**: exactly one matching, non-deleting object exists in the same Stack, unless `IfPresent()` permits absence; - **readiness**: the dependency reports `status.ready=true`; - **effective version**: a dependency that implements `v1beta1.Module` resolves to a version inside a declared SemVer range. -All requirements are mandatory and combined with logical AND. Optional relationships remain ordinary reconciler behaviour until a concrete optional-requirement use case justifies extending the interface. +All requirements are combined with logical AND. Presence is mandatory by default. `IfPresent()` permits an absent dependency but still enforces its version and readiness constraints when present; ambiguous or failed lookups remain unsatisfied or unknown. ### Dependency @@ -118,6 +119,7 @@ func Require( options ...RequirementOption, ) Requirement +func IfPresent() RequirementOption func Ready() RequirementOption func VersionAtLeast(minInclusive string) RequirementOption func VersionBefore(maxExclusive string) RequirementOption @@ -229,8 +231,8 @@ Modules such as Connectivity may need explicit cleanup after a certain incompati Legacy Ledger consumers require an additional transition guard. Changing the desired Ledger version to v3 does not immediately delete their working v2 runtimes. Instead, the Ledger reconciler refuses to materialize the primary v3 -`Cluster` while MCP, Orchestration, Reconciliation, TransactionPlane, Wallets, -or Webhooks objects still exist. The module objects are watched, so deleting the +`Cluster` while MCP, Orchestration, Payments, Reconciliation, TransactionPlane, +Wallets, or Webhooks objects still exist. The module objects are watched, so deleting the last incompatible module automatically resumes Ledger reconciliation. If a primary Ledger v3 `Cluster` already exists alongside one of these legacy @@ -264,7 +266,7 @@ For example, a future ready `Broker` requirement could remove the repeated prese 1. Implement and unit-test the requirements model. 2. Integrate it into `WithModuleReconciler` and require every module registration to choose `Requirements(...)` or `NoRequirements()`. -3. Record the confirmed Ledger relationships for Connectivity, MCP, TransactionPlane, Orchestration, Reconciliation, Wallets, and Webhooks; keep Connectivity's exact major-v3 gate local while declaring the six legacy ranges in the shared layer. +3. Record the confirmed Ledger relationships for Connectivity, MCP, TransactionPlane, Orchestration, Payments, Reconciliation, Wallets, and Webhooks; keep Connectivity's exact major-v3 gate local while declaring the seven legacy ranges in the shared layer. 4. Demonstrate Broker presence/readiness and Ledger version constraints with focused core tests. 5. Add focused tests for effective-version overrides, `Versions` lookup, invalid SemVer, range boundaries, generated dependency watches, and module-specific cleanup hooks. 6. Separate preparation from gating before migrating Broker readiness or another dependency provisioned by its consumer. @@ -299,5 +301,5 @@ The design is demonstrated when: - Connectivity cleanup runs only when Ledger is missing or its effective version definitively resolves outside semantic major v3; - a Ledger v2 to v3 transition keeps legacy module runtimes active while blocking primary v3 Cluster materialization; - an already materialized primary Ledger v3 Cluster removes incompatible module runtimes and exposure without deleting durable data; -- Connectivity declares Ledger presence plus its local semantic-major-v3 gate, while MCP, TransactionPlane, Orchestration, Reconciliation, Wallets, and Webhooks declare their confirmed Ledger ranges; +- Connectivity declares Ledger presence plus its local semantic-major-v3 gate, while MCP, TransactionPlane, Orchestration, Payments, Reconciliation, Wallets, and Webhooks declare their confirmed Ledger ranges (Payments permits Ledger absence); - existing module tests continue to pass. diff --git a/internal/core/requirements.go b/internal/core/requirements.go index c48ab743..39b08a47 100644 --- a/internal/core/requirements.go +++ b/internal/core/requirements.go @@ -38,13 +38,14 @@ type ModuleRequirements struct { requirements []Requirement } -// Requirement describes one mandatory Stack-bound dependency. +// Requirement describes constraints on one Stack-bound dependency. // // Fields are private so declarations can only be built with Require and its // options, and validated once when the controller is set up. type Requirement struct { dependency v1beta1.Dependent requireReady bool + ifPresent bool minVersionInclusive string maxVersionExclusive string minVersionSet bool @@ -63,7 +64,7 @@ func (option requirementOptionFunc) apply(requirement *Requirement) { option(requirement) } -// Requirements declares one or more mandatory dependencies. +// Requirements declares one or more dependency constraints. func Requirements(requirements ...Requirement) ModuleRequirements { return ModuleRequirements{ declared: true, @@ -96,6 +97,14 @@ func Ready() RequirementOption { }) } +// IfPresent applies the other constraints only when the dependency exists. +// Ambiguous or failed lookups still block reconciliation. +func IfPresent() RequirementOption { + return requirementOptionFunc(func(requirement *Requirement) { + requirement.ifPresent = true + }) +} + // VersionAtLeast sets an inclusive lower bound on the dependency's effective // module version. func VersionAtLeast(minInclusive string) RequirementOption { @@ -309,6 +318,13 @@ func evaluateRequirement(ctx Context, stack *v1beta1.Stack, requirement Requirem err := GetSingleDependency(ctx, stack.Name, dependency) switch { case errors.Is(err, ErrNotFound): + if requirement.ifPresent { + return requirementsEvaluation{ + status: metav1.ConditionTrue, + reason: requirementsSatisfiedReason, + message: fmt.Sprintf("optional dependency %s is absent in Stack %s", kind, stack.Name), + } + } return requirementsEvaluation{ status: metav1.ConditionFalse, reason: dependencyNotFoundReason, diff --git a/internal/core/requirements_test.go b/internal/core/requirements_test.go index ca8f84bd..4de4f68c 100644 --- a/internal/core/requirements_test.go +++ b/internal/core/requirements_test.go @@ -2,6 +2,7 @@ package core import ( "context" + "fmt" "testing" "github.com/stretchr/testify/require" @@ -256,6 +257,45 @@ func TestEvaluateLedgerEffectiveVersionRequirement(t *testing.T) { } } +func TestEvaluateOptionalLedgerRequirement(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + versions []string + wantStatus metav1.ConditionStatus + wantReason string + }{ + {name: "no Ledger", wantStatus: metav1.ConditionTrue, wantReason: requirementsSatisfiedReason}, + {name: "Ledger v2", versions: []string{"v2.4.0"}, wantStatus: metav1.ConditionTrue, wantReason: requirementsSatisfiedReason}, + {name: "Ledger v3 prerelease", versions: []string{"v3.0.0-beta.9"}, wantStatus: metav1.ConditionFalse, wantReason: dependencyVersionMismatchReason}, + {name: "Ledger v3", versions: []string{"v3.0.0"}, wantStatus: metav1.ConditionFalse, wantReason: dependencyVersionMismatchReason}, + {name: "opaque Ledger version", versions: []string{"main"}, wantStatus: metav1.ConditionUnknown, wantReason: dependencyVersionNotSemverReason}, + {name: "unresolved Ledger version", versions: []string{""}, wantStatus: metav1.ConditionUnknown, wantReason: dependencyVersionUnresolvedReason}, + {name: "multiple Ledgers", versions: []string{"v2.4.0", "v2.4.0"}, wantStatus: metav1.ConditionFalse, wantReason: multipleDependenciesFoundReason}, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + var objects []client.Object + for i, version := range test.versions { + objects = append(objects, &v1beta1.Ledger{ + ObjectMeta: metav1.ObjectMeta{Name: fmt.Sprintf("ledger-%d", i)}, + Spec: v1beta1.LedgerSpec{ + StackDependency: v1beta1.StackDependency{Stack: "stack"}, + ModuleProperties: v1beta1.ModuleProperties{Version: version}, + }, + }) + } + ctx, stack := newRequirementsTestContext(t, objects...) + evaluation := evaluateModuleRequirements(ctx, stack, Requirements( + Require(&v1beta1.Ledger{}, IfPresent(), VersionBefore(v1beta1.LedgerV3Version)), + )) + require.Equal(t, test.wantStatus, evaluation.status) + require.Equal(t, test.wantReason, evaluation.reason) + }) + } +} + func TestForModuleGatesReconcilerOnRequirements(t *testing.T) { t.Parallel() diff --git a/internal/resources/ledgers/compatibility.go b/internal/resources/ledgers/compatibility.go index bd3f993c..8d4b60cf 100644 --- a/internal/resources/ledgers/compatibility.go +++ b/internal/resources/ledgers/compatibility.go @@ -18,6 +18,7 @@ var ledgerV3IncompatibleModules = []struct { }{ {kind: "MCP", newModule: func() client.Object { return &v1beta1.MCP{} }}, {kind: "Orchestration", newModule: func() client.Object { return &v1beta1.Orchestration{} }}, + {kind: "Payments", newModule: func() client.Object { return &v1beta1.Payments{} }}, {kind: "Reconciliation", newModule: func() client.Object { return &v1beta1.Reconciliation{} }}, {kind: "TransactionPlane", newModule: func() client.Object { return &v1beta1.TransactionPlane{} }}, {kind: "Wallets", newModule: func() client.Object { return &v1beta1.Wallets{} }}, diff --git a/internal/resources/ledgers/init.go b/internal/resources/ledgers/init.go index 534df997..8be039c9 100644 --- a/internal/resources/ledgers/init.go +++ b/internal/resources/ledgers/init.go @@ -168,6 +168,7 @@ func init() { WithWatchDependency[*v1beta1.Ledger](&v1beta1.Auth{}), WithWatchDependency[*v1beta1.Ledger](&v1beta1.MCP{}), WithWatchDependency[*v1beta1.Ledger](&v1beta1.Orchestration{}), + WithWatchDependency[*v1beta1.Ledger](&v1beta1.Payments{}), WithWatchDependency[*v1beta1.Ledger](&v1beta1.Reconciliation{}), WithWatchDependency[*v1beta1.Ledger](&v1beta1.Search{}), WithWatchDependency[*v1beta1.Ledger](&v1beta1.TransactionPlane{}), diff --git a/internal/resources/payments/init.go b/internal/resources/payments/init.go index ba0ba12b..fc5b8384 100644 --- a/internal/resources/payments/init.go +++ b/internal/resources/payments/init.go @@ -32,6 +32,7 @@ import ( "github.com/formancehq/operator/v3/internal/resources/databases" "github.com/formancehq/operator/v3/internal/resources/gatewayhttpapis" "github.com/formancehq/operator/v3/internal/resources/jobs" + "github.com/formancehq/operator/v3/internal/resources/ledgers" "github.com/formancehq/operator/v3/internal/resources/registries" ) @@ -139,7 +140,13 @@ func Reconcile(ctx Context, stack *v1beta1.Stack, p *v1beta1.Payments, version s func init() { Init( WithModuleReconciler(Reconcile, - NoRequirements(), + Requirements( + Require(&v1beta1.Ledger{}, IfPresent(), VersionBefore(v1beta1.LedgerV3Version)), + ), + WithUnsatisfiedRequirementsHandler(ledgers.CleanupLegacyModuleOnV3[*v1beta1.Payments]( + &v1beta1.GatewayHTTPAPI{}, &appsv1.Deployment{}, &batchv1.Job{}, + &corev1.Service{}, &corev1.ConfigMap{}, &v1beta1.BenthosStream{}, + )), WithFinalizer[*v1beta1.Payments]("clean-payments", Clean), WithOwn[*v1beta1.Payments](&appsv1.Deployment{}), WithOwn[*v1beta1.Payments](&corev1.Service{}), diff --git a/internal/tests/module_ledger_compatibility_test.go b/internal/tests/module_ledger_compatibility_test.go index 1dffdfa3..bd5822ba 100644 --- a/internal/tests/module_ledger_compatibility_test.go +++ b/internal/tests/module_ledger_compatibility_test.go @@ -45,6 +45,18 @@ var _ = Describe("Ledger v3 module compatibility", func() { } }, }, + { + name: "Payments", + newModule: func(stack string) v1beta1.Module { + return &v1beta1.Payments{ + ObjectMeta: RandObjectMeta(), + Spec: v1beta1.PaymentsSpec{ + StackDependency: v1beta1.StackDependency{Stack: stack}, + ModuleProperties: v1beta1.ModuleProperties{Version: "v2.0.0"}, + }, + } + }, + }, { name: "Reconciliation", newModule: func(stack string) v1beta1.Module { @@ -91,6 +103,98 @@ var _ = Describe("Ledger v3 module compatibility", func() { }, } + It("reevaluates Payments when only the catalog Ledger version changes and permits Ledger absence", func() { + versions := &v1beta1.Versions{ + ObjectMeta: RandObjectMeta(), + Spec: map[string]string{"ledger": "v2.4.0", "payments": "v2.0.0"}, + } + stack := &v1beta1.Stack{ + ObjectMeta: RandObjectMeta(), + Spec: v1beta1.StackSpec{VersionsFromFile: versions.Name}, + } + ledger := &v1beta1.Ledger{ + ObjectMeta: RandObjectMeta(), + Spec: v1beta1.LedgerSpec{StackDependency: v1beta1.StackDependency{Stack: stack.Name}}, + } + payments := &v1beta1.Payments{ + ObjectMeta: RandObjectMeta(), + Spec: v1beta1.PaymentsSpec{StackDependency: v1beta1.StackDependency{Stack: stack.Name}}, + } + Expect(Create(versions, stack, ledger, payments)).To(Succeed()) + DeferCleanup(func() { + for _, object := range []client.Object{payments, ledger, stack, versions} { + Expect(client.IgnoreNotFound(Delete(object))).To(Succeed()) + } + }) + assertDependencies := func(status metav1.ConditionStatus, reason string) { + Eventually(func(g Gomega) string { + g.Expect(LoadResource("", payments.Name, payments)).To(Succeed()) + condition := payments.GetConditions().Get(core.DependenciesSatisfiedCondition) + g.Expect(condition).NotTo(BeNil()) + g.Expect(condition.Status).To(Equal(status)) + return condition.Reason + }).Should(Equal(reason)) + } + assertDependencies(metav1.ConditionTrue, "RequirementsSatisfied") + Expect(LoadResource("", versions.Name, versions)).To(Succeed()) + patch := client.MergeFrom(versions.DeepCopy()) + versions.Spec["ledger"] = "v3.0.0-beta.9" + Expect(Patch(versions, patch)).To(Succeed()) + assertDependencies(metav1.ConditionFalse, "DependencyVersionMismatch") + + Expect(LoadResource("", ledger.Name, ledger)).To(Succeed()) + patch = client.MergeFrom(ledger.DeepCopy()) + ledger.Spec.Version = "v2.4.0" + Expect(Patch(ledger, patch)).To(Succeed()) + assertDependencies(metav1.ConditionTrue, "RequirementsSatisfied") + + Expect(Delete(ledger)).To(Succeed()) + assertDependencies(metav1.ConditionTrue, "RequirementsSatisfied") + Eventually(func() bool { + err := Get(types.NamespacedName{Name: ledger.Name}, &v1beta1.Ledger{}) + return apierrors.IsNotFound(err) + }).Should(BeTrue()) + }) + + It("blocks Ledger v3 materialization when Payments is the only incompatible module", func() { + stack := &v1beta1.Stack{ObjectMeta: RandObjectMeta(), Spec: v1beta1.StackSpec{Version: "v3.0.0"}} + ledger := &v1beta1.Ledger{ + ObjectMeta: RandObjectMeta(), + Spec: v1beta1.LedgerSpec{StackDependency: v1beta1.StackDependency{Stack: stack.Name}}, + } + payments := &v1beta1.Payments{ + ObjectMeta: RandObjectMeta(), + Spec: v1beta1.PaymentsSpec{ + StackDependency: v1beta1.StackDependency{Stack: stack.Name}, + ModuleProperties: v1beta1.ModuleProperties{Version: "v2.0.0"}, + }, + } + Expect(Create(stack, payments, ledger)).To(Succeed()) + DeferCleanup(func() { + for _, object := range []client.Object{payments, ledger, stack} { + Expect(client.IgnoreNotFound(Delete(object))).To(Succeed()) + } + }) + Eventually(func(g Gomega) string { + g.Expect(LoadResource("", ledger.Name, ledger)).To(Succeed()) + condition := ledger.GetConditions().Get("LedgerV3ClusterReady") + g.Expect(condition).NotTo(BeNil()) + g.Expect(condition.Reason).To(Equal("IncompatibleModules")) + return condition.Message + }).Should(ContainSubstring("Payments")) + Consistently(func() bool { + err := Get(types.NamespacedName{Namespace: stack.Name, Name: stack.Name}, primaryLedgerV3Cluster(stack.Name)) + return apierrors.IsNotFound(err) + }).Should(BeTrue()) + Expect(Delete(payments)).To(Succeed()) + Eventually(func(g Gomega) string { + g.Expect(LoadResource("", ledger.Name, ledger)).To(Succeed()) + condition := ledger.GetConditions().Get("LedgerV3ClusterReady") + g.Expect(condition).NotTo(BeNil()) + return condition.Reason + }).ShouldNot(Equal("IncompatibleModules")) + }) + for _, testCase := range testCases { testCase := testCase It("refuses to install "+testCase.name+" with Ledger v3", func() {