fix(core, service-messaging): a failed settings read is retried, not dead-lettered as TEMPLATE_INVALID

.changeset/27257-localization-read-failed.md

@@ -0,0 +1,13 @@
+---
+'@objectstack/core': patch
+'@objectstack/service-messaging': patch
+---
+
+fix(core, service-messaging): a failed settings read no longer dead-letters a templated mail as `TEMPLATE_INVALID`; only a read that finds no default currency is permanent (#27257)
+
+Clause-②: yes (widening) — `resolveLocalizationContext`'s result (`@objectstack/core`) gains one optional member, `readFailed?: true`. Nothing is narrowed.
+Metadata-protocol: none
+
+- **Before.** `resolveLocalizationContext` answered `currency: undefined` both when the organization has no default currency and when its `sys_setting` read failed (the failure is memoized for 30 s). So a messaging delivery whose template has a `{{ amount | currency }}` hole with no argument was refused `TEMPLATE_INVALID` during a transient settings fault, graded permanent and never sent, although a retry would have worked.
+- **Now.** The answer carries `readFailed: true` when the direct `sys_setting` read threw, on the fresh read and on a hit of the failure memo alike. A settings-service refusal does not count, and the memo is not lengthened. Messaging's tenant-currency read rejects on it; the email and inbox channels then render without a currency. When that render answers `TEMPLATE_INVALID`, they render the template once more without sending, with the no-currency code `XXX` as the default: if that passes, the missing default was the only refusal, and the delivery is recorded without a `TEMPLATE_*` code and retried; if it still refuses, that refusal stands and is graded permanent, as before. Once the read succeeds, an organization with no default currency gets `TEMPLATE_INVALID` with its remedy, graded permanent, as before.
+- **Unchanged.** A template with no no-argument `currency` hole is sent during a settings fault, as before. Every other template failure keeps its grade. The other callers of `resolveLocalizationContext` see the same values; the new member is optional and absent on every successful read.

packages/core/src/security/resolve-authz-context.test.ts

@@ -455,7 +455,8 @@ describe('resolveLocalizationContext — failure-only cross-request cache (#1022
     const second = await resolveLocalizationContext({ ql, organizationId: 'o1' });
     // Functional behavior is unchanged — still the built-in defaults — the
     // second call just doesn't repeat the failing query (and its log line).
-    expect(first).toEqual({ timezone: 'UTC', locale: 'en-US', currency: undefined });
+    // [#27257] Both say the read failed, the memoized one included.
+    expect(first).toEqual({ timezone: 'UTC', locale: 'en-US', currency: undefined, readFailed: true });
     expect(second).toEqual(first);
     expect(ql.counts.sys_setting).toBe(1);
   });

packages/core/src/security/resolve-authz-context.ts

@@ -2211,7 +2211,22 @@ export interface ResolveLocalizationInput {
   userId?: string;
 }
 
-type LocalizationResult = { timezone: string; locale: string; currency?: string };
+type LocalizationResult = {
+  timezone: string;
+  locale: string;
+  currency?: string;
+  /**
+   * Present only when the direct `sys_setting` read THREW (a backend fault),
+   * on a fresh read and on a memoized one alike: every value above is then the
+   * built-in default, not a stored answer. Absent, a missing `currency` means
+   * the read succeeded and the organization has no default, which is permanent
+   * until someone sets one; present, it means nothing was learned, and a caller
+   * that would refuse work for want of a default retries instead. A
+   * settings-service refusal never sets it (#11877), and it does not lengthen
+   * the failure memo. (#27257)
+   */
+  readFailed?: true;
+};
 
 /**
  * Process-local TTL cache for the FAILED-READ outcome of
@@ -2630,7 +2645,9 @@ function putLocalizationEntry(ql: object, key: string, entry: LocalizationCacheE
  * {@link LOCALIZATION_FAILURE_CACHE_TTL_MS} per `(ql, organizationId, userId)` so
  * the failing query — and the driver's log line for it — does not repeat
  * every request (#10221). A settings-service refusal is not a backend fault
- * and never populates that memo (#11877).
+ * and never populates that memo (#11877). Such an answer carries
+ * `readFailed: true`, fresh and memoized alike, so a caller can tell "the read
+ * failed" from "the read found no default" (#27257).
  *
  * A SUCCESSFUL read is cached too, since #11966 (leg C of #11633) — but only
  * when the engine carries the write-epoch and middleware seams, and only until
@@ -2667,7 +2684,10 @@ export async function resolveLocalizationContext(input: ResolveLocalizationInput
     }
   }
 
-  const { value, backendFailed } = await resolveLocalizationContextUncached(input);
+  const { value: read, backendFailed } = await resolveLocalizationContextUncached(input);
+  // [#27257] The kind travels ON the answer, and the memo stores that same
+  // answer, so a hit inside the failure TTL still says the read failed.
+  const value: LocalizationResult = backendFailed ? { ...read, readFailed: true } : read;
   if (!cacheable) return value;
 
   if (backendFailed) {

packages/core/src/security/resolve-localization-cache.test.ts

@@ -463,6 +463,7 @@ describe('resolveLocalizationContext — the failure memo survives what retires
       timezone: 'UTC',
       locale: 'en-US',
       currency: undefined,
+      readFailed: true,
     });
     expect(ql.counts.sys_setting).toBe(1);
 

packages/core/src/security/resolve-localization-read-failed.test.ts

@@ -0,0 +1,106 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+//
+// [#27257] `resolveLocalizationContext` tells "the read failed" from "the read
+// found no default". A messaging channel refuses a mail whose
+// `{{ amount | currency }}` hole has no default, and grades that refusal
+// permanent; it may do so only when the read succeeded. So the answer carries
+// `readFailed: true` exactly when the direct `sys_setting` read threw, on the
+// fresh read AND on the hit the failure memo serves inside its TTL, and never
+// otherwise.
+
+import { describe, it, expect, vi, afterEach } from 'vitest';
+import { resolveLocalizationContext } from './resolve-authz-context.js';
+
+/** A `sys_setting` read answering `rows`, recording every read. */
+function makeQl(rows: Array<Record<string, unknown>>) {
+  const reads: string[] = [];
+  return {
+    reads,
+    async find(object: string, o: any) {
+      reads.push(object);
+      const where = o?.where ?? {};
+      const matched = rows.filter((r) =>
+        Object.entries(where).every(([k, v]) => {
+          if (k.startsWith('$')) throw new Error(`fake driver: unsupported operator ${k}`);
+          if (v && typeof v === 'object' && '$in' in (v as any)) return (v as any).$in.includes(r[k]);
+          return r[k] === v;
+        }),
+      );
+      // Hold the caller's bound, after the filter and by presence.
+      return typeof o?.limit === 'number' ? matched.slice(0, o.limit) : matched;
+    },
+  };
+}
+
+/** A `sys_setting` read that throws: the backend fault the failure memo is for. */
+function makeBrokenQl() {
+  const reads: string[] = [];
+  return {
+    reads,
+    async find(object: string): Promise<never> {
+      reads.push(object);
+      throw new Error('connection reset reading sys_setting');
+    },
+  };
+}
+
+afterEach(() => {
+  vi.useRealTimers();
+});
+
+describe('resolveLocalizationContext — a failed read says so (#27257)', () => {
+  it('a direct read that throws answers the built-ins WITH `readFailed: true`', async () => {
+    const ql = makeBrokenQl();
+    const answer = await resolveLocalizationContext({ ql, organizationId: 'o1' });
+    expect(answer).toEqual({ timezone: 'UTC', locale: 'en-US', currency: undefined, readFailed: true });
+    expect(ql.reads).toHaveLength(1);
+  });
+
+  it('the failure memo\'s hit keeps the kind: no second read, still `readFailed: true`', async () => {
+    const ql = makeBrokenQl();
+    await resolveLocalizationContext({ ql, organizationId: 'o1' });
+    const hit = await resolveLocalizationContext({ ql, organizationId: 'o1' });
+    expect(ql.reads).toHaveLength(1); // served by the memo, not a fresh read
+    expect(hit.readFailed).toBe(true);
+    expect(hit.currency).toBeUndefined();
+  });
+
+  it('a read that SUCCEEDS and finds no default carries no `readFailed` at all', async () => {
+    const ql = makeQl([
+      { namespace: 'localization', key: 'timezone', scope: 'tenant', organization_id: 'o1', value: 'Asia/Tokyo' },
+    ]);
+    const answer = await resolveLocalizationContext({ ql, organizationId: 'o1' });
+    expect(answer.timezone).toBe('Asia/Tokyo');
+    expect(answer.currency).toBeUndefined();
+    expect(answer).not.toHaveProperty('readFailed');
+  });
+
+  it('a settings-service refusal in front of a SUCCESSFUL direct read is not a failed read (#11877)', async () => {
+    const settings = {
+      get: async () => { throw new Error('unused'); },
+      getMany: async () => { throw new Error("unknown settings namespace 'localization'"); },
+    };
+    const ql = makeQl([]);
+    const answer = await resolveLocalizationContext({ ql, settings, organizationId: 'o1' });
+    expect(answer.currency).toBeUndefined();
+    expect(answer).not.toHaveProperty('readFailed');
+  });
+
+  it('the memo is not lengthened: past its TTL a recovered read answers the stored default, no kind', async () => {
+    vi.useFakeTimers();
+    const broken = makeBrokenQl();
+    const healthy = makeQl([
+      { namespace: 'localization', key: 'currency', scope: 'tenant', organization_id: 'o1', value: 'CNY' },
+    ]);
+    // ONE engine (the memo is keyed on it), whose backend recovers mid-test.
+    let backend: { find(object: string, o?: any): Promise<unknown> } = broken;
+    const ql = { find: (object: string, o: any) => backend.find(object, o) };
+    expect((await resolveLocalizationContext({ ql, organizationId: 'o1' })).readFailed).toBe(true);
+    backend = healthy;
+    await vi.advanceTimersByTimeAsync(30_001);
+    const recovered = await resolveLocalizationContext({ ql, organizationId: 'o1' });
+    expect(recovered.currency).toBe('CNY');
+    expect(recovered).not.toHaveProperty('readFailed');
+    expect(broken.reads.length + healthy.reads.length).toBe(2);
+  });
+});

packages/services/service-messaging/src/email-channel.ts

@@ -117,10 +117,58 @@ export interface EmailChannelOptions {
      * hole with no argument formats in. `undefined` (no default set, or not
      * wired) leaves the render without one, so such a hole refuses the delivery
      * with `TEMPLATE_INVALID` instead of formatting in a guessed currency.
+     * REJECTS when the read itself failed: the render then runs without one,
+     * and a `TEMPLATE_INVALID` that a default would cure is graded retryable
+     * ({@link templateRefusalAfterCurrencyReadFault}).
      */
     getTenantCurrency?(organizationId: string | undefined): Promise<string | undefined>;
 }
 
+/**
+ * The ISO 4217 code for "no currency" (`XXX`, formatted `¤1,234.50`): the
+ * stand-in default {@link templateRefusalAfterCurrencyReadFault} renders with
+ * to ask whether a default would cure a refusal. Never sent: the probe only
+ * renders.
+ */
+const NO_CURRENCY_PROBE = 'XXX';
+
+/**
+ * [#27257] The row error of a templated delivery whose render failed. A read
+ * that FAILED is not a read that found no default: when `getTenantCurrency`
+ * rejected and the render then answered `TEMPLATE_INVALID`, the template is
+ * rendered once more WITHOUT sending (`renderWithDefault`), with
+ * {@link NO_CURRENCY_PROBE} as the default.
+ *
+ * - The probe renders: the only refusal was the missing default, which the
+ *   read would have supplied. The error is rewritten WITHOUT a `TEMPLATE_*`
+ *   code, so `classifyError` grades it retryable.
+ * - The probe still answers `TEMPLATE_INVALID`: a default cannot cure the
+ *   template (a hole the grammar refuses, an `@path` value that is not a
+ *   code). That refusal is returned as it stands, graded permanent.
+ * - No probe (an email service without `renderTemplate`) or a probe failing
+ *   any other way: the cause cannot be told apart, and the read fault is
+ *   transient, so it is rewritten and retried.
+ *
+ * Any other failure, and every refusal after a read that succeeded, is
+ * returned unchanged. Shared by the email and inbox channels.
+ */
+export async function templateRefusalAfterCurrencyReadFault(
+    message: string,
+    currencyReadFault: string | undefined,
+    renderWithDefault: ((currency: string) => Promise<unknown>) | undefined,
+): Promise<string> {
+    if (currencyReadFault === undefined || !/\bTEMPLATE_INVALID\b/.test(message)) return message;
+    if (renderWithDefault) {
+        try {
+            await renderWithDefault(NO_CURRENCY_PROBE);
+        } catch (err) {
+            const standing = (err as Error)?.message ?? String(err);
+            if (/\bTEMPLATE_INVALID\b/.test(standing)) return standing;
+        }
+    }
+    return `the template was not rendered: the organization's default currency could not be read (${currencyReadFault}); the delivery is retried until that read succeeds`;
+}
+
 /**
  * The ONE token both members of this channel use for "there is no transport"
  * (commit 879b51270) — the reason `isAvailable()` already returns, reused verbatim so the
@@ -351,8 +399,16 @@ export function createEmailChannel(opts: EmailChannelOptions): MessagingChannel
                 }
                 const templateLocale = recipientLocale;
                 const data = (payload.templateData ?? undefined) as Record<string, unknown> | undefined;
+                let currency: string | undefined;
+                let currencyReadFault: string | undefined;
+                try {
+                    currency = await opts.getTenantCurrency?.(n.organizationId);
+                } catch (err) {
+                    // The read failed and learned nothing: render without a
+                    // currency, and grade a refusal below (#27257).
+                    currencyReadFault = (err as Error)?.message ?? String(err);
+                }
                 try {
-                    const currency = await opts.getTenantCurrency?.(n.organizationId);
                     const result = (await email.sendTemplate({
                         template: templateName,
                         to: address,
@@ -377,8 +433,21 @@ export function createEmailChannel(opts: EmailChannelOptions): MessagingChannel
                     // TEMPLATE_INACTIVE / MISSING_VARIABLES / TEMPLATE_INVALID)
                     // arrives as a thrown Error — keep the code at the front of
                     // the row's error so classifyError() below can grade it
-                    // permanent.
-                    return { ok: false, error: (err as Error)?.message ?? String(err) };
+                    // permanent — unless the currency read failed first.
+                    const renderTemplate = email.renderTemplate?.bind(email);
+                    return {
+                        ok: false,
+                        error: await templateRefusalAfterCurrencyReadFault(
+                            (err as Error)?.message ?? String(err),
+                            currencyReadFault,
+                            renderTemplate && ((probe) => renderTemplate({
+                                template: templateName,
+                                ...(data !== undefined ? { data } : {}),
+                                ...(templateLocale ? { locale: templateLocale } : {}),
+                                ...{ currency: probe },
+                            })),
+                        ),
+                    };
                 }
             }
 

packages/services/service-messaging/src/inbox-channel.ts

@@ -8,7 +8,7 @@ import type {
     MessagingChannelContext,
     SendResult,
 } from './channel.js';
-import type { EmailSenderSurface } from './email-channel.js';
+import { templateRefusalAfterCurrencyReadFault, type EmailSenderSurface } from './email-channel.js';
 import { RECIPIENT_LOCALE_FIELD, USER_OBJECT, resolveRecipientLocale } from './recipient-locale.js';
 import { FAN_OUT_SYSTEM_CONTEXT } from './fan-out-system-context.js';
 import { assertActorReferenceResolves } from './actor-reference.js';
@@ -61,6 +61,9 @@ export interface InboxChannelOptions {
      * hole with no argument formats in. `undefined` (no default set, or not
      * wired) leaves the render without one, so such a hole refuses the delivery
      * with `TEMPLATE_INVALID` instead of formatting in a guessed currency.
+     * REJECTS when the read itself failed: the render then runs without one,
+     * and a `TEMPLATE_INVALID` that a default would cure is graded retryable
+     * (`templateRefusalAfterCurrencyReadFault` in `email-channel.ts`).
      */
     getTenantCurrency?(organizationId: string | undefined): Promise<string | undefined>;
     /** User identity object the recipient's locale is read from (default `sys_user`). */
@@ -194,8 +197,16 @@ export function createInboxChannel(opts: InboxChannelOptions): MessagingChannel
                     deploymentLocale,
                 );
                 const templateData = (payload.templateData ?? undefined) as Record<string, unknown> | undefined;
+                let currency: string | undefined;
+                let currencyReadFault: string | undefined;
+                try {
+                    currency = await opts.getTenantCurrency?.(n.organizationId);
+                } catch (err) {
+                    // The read failed and learned nothing: render without a
+                    // currency, and grade a refusal below (#27257).
+                    currencyReadFault = (err as Error)?.message ?? String(err);
+                }
                 try {
-                    const currency = await opts.getTenantCurrency?.(n.organizationId);
                     const rendered = await email.renderTemplate({
                         template: templateName,
                         ...(templateData !== undefined ? { data: templateData } : {}),
@@ -209,8 +220,20 @@ export function createInboxChannel(opts: InboxChannelOptions): MessagingChannel
                     // TEMPLATE_INACTIVE / MISSING_VARIABLES / TEMPLATE_INVALID)
                     // arrives as a thrown Error — keep the code at the front of
                     // the row's error so classifyError() below can grade it
-                    // permanent.
-                    return { ok: false, error: (err as Error)?.message ?? String(err) };
+                    // permanent — unless the currency read failed first.
+                    return {
+                        ok: false,
+                        error: await templateRefusalAfterCurrencyReadFault(
+                            (err as Error)?.message ?? String(err),
+                            currencyReadFault,
+                            (probe) => email.renderTemplate!({
+                                template: templateName,
+                                ...(templateData !== undefined ? { data: templateData } : {}),
+                                ...(templateLocale ? { locale: templateLocale } : {}),
+                                ...{ currency: probe },
+                            }),
+                        ),
+                    };
                 }
             }