Skip to content

docs: AWS deployment architecture proposals (subdomain and single-host tenant-picker variants) - #6443

Open
iliyan-velichkov wants to merge 6 commits into
masterfrom
docs/aws-deployment-proposal
Open

docs: AWS deployment architecture proposals (subdomain and single-host tenant-picker variants)#6443
iliyan-velichkov wants to merge 6 commits into
masterfrom
docs/aws-deployment-proposal

Conversation

@iliyan-velichkov

@iliyan-velichkov iliyan-velichkov commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Documentation only — two sibling architecture proposals for deploying a multitenant Dirigible application on AWS, both right-sized to one production unit (~50–100 tenants) and designed independently of the docs/aws-multitenancy-research branch (each records where it agrees/differs).

1. AWS_DEPLOYMENT_PROPOSAL.md — subdomain per tenant

  • Compute: ECS Fargate, desiredCount=1, stop-then-start deploys. The runtime is single-writer by construction (embedded ActiveMQ vm:// broker holding a JDBC lock on SystemDB, node-local synchronizer effects, LocalRepository) — the availability consequence (~99.5% SLO) is stated honestly instead of worked around.
  • Data: RDS PostgreSQL Multi-AZ with the shipped schema-per-tenant provisioning; documents on S3 (engine-cms-s3); content baked into the image with publishing disabled; optional internal authoring instance as the In-System-Programming security boundary.
  • Identity: ONE Cognito user pool (one identity = same credentials in every tenant) and one confidential app client per tenant — decided on merit in §4.2 against the single-shared-client alternative (100-callback hard ceiling at the design capacity, no tenant signal for the pre-token Lambda, multi-tenant bearer tokens with app-side-only denial). The shipped security-client-registration module is explicitly not a design input.
  • Cross-tenant login: membership and roles as prefixed Cognito groups (t:<subdomain>:<role>); a pre-token-generation Lambda scopes every token to its tenant (cognito:groups = roles-here-only + a dirigible:tenant audience claim). One interactive sign-in, silent SSO into other tenants, different roles in each.
  • Fork changes: seven must-fix items with file:line anchors, plus four follow-ups; onboarding/offboarding runbooks, hardening checklist, operations, cost ballpark (~$700/month), how-it-grows, and six pre-go-live hypotheses flagged rather than asserted.

2. AWS_SINGLE_HOST_TENANT_PICKER_PROPOSAL.md — no subdomains, in-app tenant picker

The variant for changed requirements: one host, one login, and the user changes the current tenant from a picker in the UI, holding different roles in each. Everything below the identity layer is inherited from proposal 1 unchanged.

  • Dropping subdomains removes the Host header as the tenant signal, and the picker means the tenant is chosen after login and changes without re-auth — so the active tenant becomes server-side session state, and the default-tenant fallback is removed for authenticated traffic (no selection → picker page, never silent default).
  • The token carries the full membership graph as prefixed groups, parsed at login into a membership map with zero authorities; POST /services/core/security/tenants/current validates membership, rebuilds the session's authorities to the picked tenant's roles, rotates the session id and reloads the UI. Switching tenants is the same call — no re-authentication.
  • Cognito simplifies radically: ONE app client (shipped static registration nearly as-is), no pre-token Lambda (Lite plan may suffice), no clientId→tenant map; onboarding shrinks to provision-the-schema + create-the-groups.
  • Honest costs stated: one browser session = one active tenant (concurrent multi-tenant tabs — free under subdomains — are impossible here), multi-tenant tokens with no audience binding, tenant-less deep links, and a larger fork surface (S1–S6: session/header resolver, neutral login mapper, switch endpoint, picker UI, header-based M2M tenancy, stale-tab guard). Path-based tenancy (/t/<tenant>/…) is analysed and rejected.
  • §8 is a side-by-side trade-off table between the two models; §9 lists the model-specific go-live hypotheses (e.g. authority rebuild on a live session).
  • §10 answers horizontal scaling head-on: raising desiredCount today just blocks the second task at boot (ActiveMQ JDBC lock — safe, never split traffic); the layer-by-layer bill for true replicas (external broker, per-node synchronizer replay, boot-race fixes, cache invalidation — shared by both models; plus spring-session + Redis as the only picker-specific item, after which the design survives unchanged); and the picker-specific unit-routing catch (the edge can't see a session-resident tenant → routing cookie + CloudFront Function). Proposal 1's §11 cross-references it.

3. AWS_TENANCY_MODELS_COMPARISON.md — the decision aid

Puts the two models side by side: what is identical in both (groups-only membership, everything below the identity layer, the horizontal-scaling blockers), a 19-row comparison table, explicit pros/cons per model, decision guidance ("subdomains treat tenants as separate products sharing a platform; the picker treats tenants as workspaces inside one product"), the hybrid option (a picker-style dropdown on top of subdomains), and the migration story between the models (the shared t:<tenant>:<role> group convention means switching changes plumbing, never data).

Test plan

  • Documentation-only change — no code, no configuration, no CI-relevant files touched
  • Markdown renders correctly on GitHub (tables, diagrams, section anchors) in both documents
  • Cross-references resolve: file:line anchors spot-checked against current sources (DataSourceInitializer.java:155-159, TenantExtractor.java:44, security-cognito classes, TenantContext.java); the two documents' links to each other work

🤖 Generated with Claude Code

iliyan-velichkov and others added 2 commits July 29, 2026 10:58
Documentation only. No code, no configuration, nothing implemented.

Adds AWS_DEPLOYMENT_PROPOSAL.md - a production architecture for deploying a
multitenant Dirigible application on AWS, designed independently of the
docs/aws-multitenancy-research branch (SS10 records where the two agree and
deliberately differ), right-sized to one production unit (~50-100 tenants).

Shape, in short:
- ECS Fargate, desiredCount=1 with stop-then-start deploys - the runtime is
  single-writer by construction (embedded ActiveMQ vm:// broker with a JDBC
  lock on SystemDB, node-local synchronizer effects, LocalRepository).
  Realistic SLO ~99.5%, stated honestly.
- RDS PostgreSQL Multi-AZ: SystemDB + DefaultDB with the provisioner's
  schema-per-tenant model; documents on S3 (engine-cms-s3); content baked
  into the image, publishing disabled; optional internal authoring instance
  as the In-System-Programming security boundary.
- Identity: ONE Cognito user pool (one identity = same credentials in every
  tenant); ONE confidential app client per tenant - decided on merit in
  SS4.2 against the single-shared-client alternative (100-callback ceiling,
  no tenant signal for the pre-token Lambda, multi-tenant bearer tokens),
  NOT inherited from the shipped security-client-registration module, which
  is explicitly not a design input.
- Membership and roles as prefixed Cognito groups (t:<subdomain>:<role>)
  projected per token by a pre-token-generation Lambda: cognito:groups
  carries only that tenant's roles plus a dirigible:tenant audience claim -
  cross-tenant login with different roles per tenant, silent via the
  pool-wide SSO cookie.
- Seven must-fix fork changes with file:line anchors (configurable Hikari
  pool sizing, tenant assertion for session+M2M principals, method security
  under the cognito profile, filter-chain ordering, per-tenant logout,
  tenant cache sizing, host-keyed ClientRegistrationRepository reading
  Secrets Manager), plus onboarding/offboarding runbooks, hardening
  checklist, operations, cost ballpark and a how-it-grows section.
- Six hypotheses flagged for verification before go-live rather than
  asserted (e.g. @RolesAllowed inert under the cognito profile).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documentation only. No code, no configuration, nothing implemented.

Adds AWS_SINGLE_HOST_TENANT_PICKER_PROPOSAL.md - the sibling of
AWS_DEPLOYMENT_PROPOSAL.md for a changed set of requirements: no dedicated
subdomain per tenant, one login, and an in-app tenant picker from which the
user changes the current tenant, holding different roles in each. Everything
below the identity layer (single-writer ECS Fargate unit, RDS Multi-AZ
schema-per-tenant, S3 CMS, hardening, operations, cost) is inherited from
the companion unchanged.

Shape, in short:
- Dropping subdomains removes the Host header as the tenant signal, and the
  picker means the tenant is chosen AFTER login and changes WITHOUT re-auth,
  so tokens cannot be tenant-scoped at issuance. The active tenant becomes
  server-side session state; the default-tenant fallback is removed for
  authenticated traffic (no selection -> picker page, never silent default).
- The token carries the full membership graph as prefixed Cognito groups
  (t:<tenant>:<role>), parsed at login into a membership map with ZERO
  authorities granted; POST /services/core/security/tenants/current
  validates membership, stores activeTenant in the session, REBUILDS the
  session authorities to that tenant's roles only, rotates the session id
  and reloads the UI. Switching tenants is the same call - the requirement.
- Cognito simplifies radically: ONE app client (the shipped static
  registration nearly as-is), no pre-token Lambda (Lite plan may suffice),
  no clientId->tenant map, shipped single-client logout correct unchanged.
  Onboarding shrinks to provision-the-schema + create-the-groups.
- Honest costs stated: one browser session = one active tenant (concurrent
  multi-tenant tabs, free under subdomains, are impossible here - mitigated
  with an active-tenant chip and a stale-tab guard); multi-tenant tokens
  with no audience binding; tenant-less deep links; and a LARGER fork
  surface than the subdomain model (S1-S6: session/header resolver, neutral
  login mapper, switch endpoint, picker UI, header-based M2M tenancy, tab
  guard). Path-based tenancy (/t/<tenant>/...) analysed and rejected.
- Model-specific go-live hypotheses (authority rebuild on a live session,
  no residual host/default fallback, group-claim size, Lite-plan fit,
  no stale per-tenant state after the reload-on-switch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@iliyan-velichkov iliyan-velichkov changed the title docs: AWS deployment architecture proposal (one unit, ECS + Cognito) docs: AWS deployment architecture proposals (subdomain and single-host tenant-picker variants) Jul 29, 2026
iliyan-velichkov and others added 4 commits July 29, 2026 12:15
Documentation only. Answers, layer by layer, whether the proposed
architectures work with horizontal scaling, and if not, what enabling it
would take.

Adds SS10 to AWS_SINGLE_HOST_TENANT_PICKER_PROPOSAL.md:
- 10.1: raising desiredCount today just blocks the second task at boot on
  the embedded ActiveMQ JDBC lock - a stuck deployment, never split traffic,
  so the in-heap session is never silently wrong.
- 10.2: the layer-by-layer bill for true replicas. Platform blockers shared
  by both models: external broker (vm:// URL is hardcoded), per-node
  synchronizer reconciliation (shared checksums vs per-JVM side effects -
  the deepest rework), boot-time DDL races (Quartz initialize-schema=always
  et al.), per-JVM cache invalidation. The repository layer is already fine
  for runtime-only replicas (baked-in content expands per task); authoring
  stays at 1 forever. The only picker-specific item is the HTTP session:
  spring-session + ElastiCache Redis, after which the tenant-in-session
  design survives unchanged. Cross-cutting: connection pools multiply per
  (instance, tenant) - the pool-sizing fix becomes non-negotiable - and
  WebSocket paths need stickiness regardless.
- 10.3: the picker-specific scale-out catch - with one host the edge cannot
  see the tenant (session state; ALB rules cannot match cookies), so routing
  tenants to units needs a routing cookie + CloudFront Function
  (recommended), a gateway tier, or a same-unit placement rule; hybrid
  fallback is hostnames for routing only.
- 10.4: recommended posture per phase, including the cheap early win of
  adding spring-session + Redis ahead of need so sessions survive deploys.

Also: a scale-out-routing row in the SS8 comparison table, and a pointer
from AWS_DEPLOYMENT_PROPOSAL.md SS11 (blockers 1-5 apply to both models).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GitHub-anchor TOCs after the header blocks: top-level sections in both
documents, plus the subsection level where navigation pays off (SS4 identity
decisions in the subdomain proposal; SS2/SS3 identity+picker mechanics and
the SS10 horizontal-scaling analysis in the single-host proposal).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Makes explicit in both proposals what was previously implied: there is NO
tenant user attribute of any kind (the custom:tenant attribute the shipped
CognitoTenantFilter reads is retired). A user's tenants are DERIVED from
the t:<tenant>:<role> group pattern - membership in a tenant is holding at
least one of its groups. One claim (cognito:groups) feeds the picker list,
the membership check and the per-tenant roles, so there is no second source
of truth to drift.

Spelled-out consequences in both documents:
- administration is a single operation surface: add to a tenant = add a
  group, change roles = change groups, remove = delete the user's
  t:<tenant>:* groups - nothing else to keep in sync;
- a "member with zero roles" state does not exist (removing the last role
  group removes the membership itself); a conventional marker role
  (t:<tenant>:member) is the escape hatch if that state is ever needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…picker)

Documentation only. Adds AWS_TENANCY_MODELS_COMPARISON.md - a decision aid
putting the two proposed tenancy models side by side, not a third design.

Content: each model in one paragraph; what is identical in both (the
groups-only membership model, everything below the identity layer, the
horizontal-scaling blockers, the shared fork fixes) - making explicit that
this is purely an identity/UX-layer decision; a 19-row side-by-side table
(login UX, switching, concurrent tabs, deep links, URL legibility, token
scoping and stolen-token blast radius, denial point, Cognito surface and
plan tier, DNS/TLS, onboarding, fork surface, pre-login branding,
federation, vanity domains, per-tenant edge controls, unit scale-out,
future replicas, M2M, logout, CSRF exposure); explicit pros/cons lists per
model; decision guidance ("subdomains treat tenants as separate products
sharing a platform; the picker treats tenants as workspaces inside one
product"); the hybrid option (picker-style dropdown on top of subdomains -
keeps every subdomain advantage, fails only the literal no-per-tenant-URL
requirement); and the migration story between the models (shared group
convention means switching changes plumbing, never data).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant