feat(lake): enforce retention TTL per dataset + soft per-org Lake quota (CL-9)

[TerrifiedBug]

CL-9 — Lake retention enforcement + per-org Lake byte quota (GA-hardening slice)

LakeRetentionPolicy(hotDays, coldDays) was attached to datasets but only the table-default TTL was ever applied, and there was no per-org Lake byte quota gate. This ships a coherent, testable enforcement slice for both.

Retention enforcement — src/server/services/lake/lake-retention.ts

• effectiveRetention(policy) — pure; computes a dataset's effective hot/cold window, falls back to the table defaults (7/90) when no policy, and clamps coldDays >= hotDays.
• buildLakeTtlClause(retention, coldTierEnabled) — pure; owns the ClickHouse TTL clause shape. migrate.ts now builds the base table TTL through this same function (buildLakeTtlClause(effectiveRetention(null), …)), so the table default and any per-dataset window share one code path (behaviour unchanged: still INTERVAL 7 DAY TO VOLUME 'cold' / INTERVAL 90 DAY DELETE / storage_policy = 'vf_hot_cold').
• enforceDatasetRetention(...) — applies a dataset's coldDays as a drop horizon: a bounded, org+pipeline-scoped DELETE FROM lake_events WHERE … AND timestamp < {cutoff} (bound params), behind the shared lake client. Catches datasets whose policy is shorter than the global table TTL.
• sweepLakeRetention() + a leader-gated scheduler (initLakeRetentionScheduler, mirrors lake-alerts) walk the catalog daily and enforce each dataset's horizon; wired into instrumentation.node.ts. Per-dataset errors are logged and skipped (one bad dataset never stalls the sweep).

Retention deletion is the intended data lifecycle, so this path does delete expired rows — distinct from the quota path, which never drops.

Per-org Lake byte quota — src/server/services/lake/lake-quota.ts

• LakeQuotaProvider seam mirroring src/server/services/quotas.ts: OSS ships DefaultUnlimitedLakeQuotaProvider (unmetered self-hosted); a commercial deployment registers its own via setLakeQuotaProvider(...) to map an org's tier → byte ceiling.
• checkLakeQuota(orgId, currentBytes, quotaBytes) — pure (under/over/at/unlimited/zero-ceiling).
• evaluateLakeQuota(orgId) — sums the catalog byteCount and soft-signals when over quota: a structured warn log + the read-only lake.quotaStatus tRPC query (UI badge surface). It never drops, rejects, or rewrites data. Wired into the catalog/heartbeat ingest path (updateLakeCatalogFromHeartbeat), best-effort. Free in OSS — the default provider short-circuits before any DB read.

No migration

Over-quota is derived on read (sum of LakeDataset.byteCount vs the provider ceiling) and retention reuses the existing LakeRetentionPolicy/LakeDataset fields — so no schema change was needed (preferred per the task).

Tests (npx vitest run src/server/services/lake/ → 116 pass)

• lake-quota.test.ts — pure check under/over/at-ceiling/unlimited/zero; evaluateLakeQuota unmetered short-circuit, under-quota (no signal), over-quota fires the signal with zero data mutation (explicit no-data-drop invariant), empty-catalog null sum.
• lake-retention.test.ts — effectiveRetention defaults/policy/malformed/clamp; buildLakeTtlClause cold on/off + per-dataset window; enforceDatasetRetention disabled no-op + per-policy coldDays cutoff + bound-param scope; sweepLakeRetention disabled no-op, per-dataset horizons, error tolerance.
• Prisma and @clickhouse/client are mocked.

Verification notes

• npx tsc --noEmit: clean except the pre-existing @clickhouse/client module-resolution error in clickhouse.ts (the native dep isn't installed in this dev env; resolves on CI).
• lake.quotaStatus takes no tenant-id input (org from ctx.organizationId), so the cross-org-access linter skips it — verified by loading appRouter and inspecting the live procedure.
• ClickHouse/scheduler/instrumentation paths aren't locally runnable; unit tests + types are the bar.