Issue #1271: feat: record which organization's Query Planner emitted each statistic

[dereck-symmetry]

Stacked on #1301 — merge #1301 first. This branch is cut from #1301's, so until that merges this diff also shows its commit (01138b3). Review only 3845e36, which is this PR. The base is main so CI runs.

Description of Change

Problem. One Query Planner runs per organization (query-planner-org1/2/3), but nothing in its environment says which, so every statistics event #341 emits collapses into one bucket across all three orgs. LIF_ORG_KEY did not exist anywhere in the repo.

Solution. A new optional LIF_ORG_KEY, read once by the base into LIFQueryPlannerConfig.org_key and recorded as org_key on both statistics events (query_planned and query_completed), next to #1301's client.

• Optional. Unset or blank records unknown, and the query proceeds as before. Blank is treated as unset because that's what a CloudFormation Value: yields when its source is missing (the base's Config hygiene: unguarded int(os.getenv(...)) at module scope crash-loops services on a malformed value (6 on main, 3 more in flight) #1179 convention).
• One naming scheme, not two. The value is CloudFormation's OrganizationName (org1/org2/org3), the same one that names each planner's stack, ECS service, compose container and SSM parameters. I considered SEED_DATA_KEY's advisor-demo-orgN, but it names a sample dataset, not the organization.

Deployment config. This is the live-environment part, so here is what it does precisely:

• No *.params file changes. cloudformation/lif-query-planner-taskdef-includes.yml already builds values from ${OrganizationName} (e.g. LIF_QUERY_CACHE_URL), so a single entry, LIF_ORG_KEY: Fn::Sub "${OrganizationName}", covers all six deployed planners: {dev,demo}-lif-query-planner-org{1,2,3}. Each sets OrganizationName in its params, and all six are in dev.aws / demo.aws. The unsuffixed *-lif-query-planner.params files aren't deployed.
• Local: deployments/advisor-demo-docker/docker-compose.yml sets LIF_ORG_KEY: org1/org2/org3 on the three planners.
• What merging does, and doesn't do. The deploy workflow rebuilds the image and runs aws ecs update-service, which redeploys with the existing task definition. It doesn't update the stack. So after merge, dev and demo run the new image without LIF_ORG_KEY and record org_key: "unknown". That's the graceful fallback, and it's safe to leave in place. The variable arrives when someone runs, sequentially:

  ./aws-deploy.sh -s dev --only-stack dev-lif-query-planner-org1   # then org2, org3; same for demo

  Nobody has run that as part of this PR.

How to test. uv run pytest test/components/lif/query_planner_service test/bases/lif/query_planner_restapi.

Related Issues

Closes #1271
Refs #341
Refs #1131

Type of Change

• New feature (non-breaking change which adds functionality)
• Infrastructure/deployment change

Project Area(s) Affected

• bases/
• components/
• deployments/
• cloudformation/ or sam/ templates
• test/ or e2e/
• Documentation (docs/, READMEs, ARCHITECTURE.md, CLAUDE.md)

---

Checklist

• commit message follows commit guidelines (see commitlint.config.mjs)
• tests are included (unit and/or integration tests)
• code passes linting checks (uv run ruff check)
• code passes formatting checks (uv run ruff format)
• code passes type checking (uv run ty check)
• pre-commit hooks have been run successfully
• configuration changes: relevant folder README updated

Testing

• Automated tests added/updated

The acceptance criteria, each with its test:

• Read by the planner and included in every event. test_both_events_carry_the_configured_org_key goes from submission through the results callback. test_query_statistics_through_the_endpoint_carry_the_org_key runs through the real TestClient.

• Absence degrades gracefully. test_a_planner_without_an_org_key_still_serves_the_query_and_emits_unknown. test_unset_or_blank_org_key_falls_back_to_unknown covers both unset and whitespace-only.

• The variable name itself. It's read at import, so test_org_key_is_read_from_lif_org_key imports the base in a fresh interpreter, the same technique the timeout tests use, since importlib.reload is off-limits.

• Both deployment files were parsed to confirm the entries land where intended:

  • the taskdef include's Environment has LIF_ORG_KEY: {Fn::Sub: ${OrganizationName}};
  • the three compose planners get org1/org2/org3.

  The Fn::Sub itself can't be exercised locally. Its form is identical to the existing LIF_QUERY_CACHE_URL entry in the same file.

Mutation-checked. Each of these mutants fails at least one test:

• the config never receives org_key;
• the wrong env var name;
• a blank value not treated as unset;
• the planned event omits it;
• the completed event omits it.

Additional Notes

• Merge order: after Issue #1272: feat: record the Query Planner's caller from an optional X-LIF-Client header #1301, since this is stacked on it.
• Other open PRs: Issue #1264: fix: surface a Query Planner failure as a GraphQL error, not an empty result #1291, Issue #1258: fix: narrow the identity mapping key so it is a real B-tree #1299 and Issue #212: feat: Implement LangChain message trimming #1148 also touch CHANGELOG.md. No other open PR touches the taskdef include or the compose planners.
• Workflow filter: lif_query_planner_api.yml's paths: already includes cloudformation/lif-query-planner-taskdef-includes.yml and both planner bricks, so merging rebuilds the image. As above, that doesn't update the task definition.

🤖 Generated with Claude Code

[bjagg]

Approving. Reviewed 3845e36 only, as asked. The branch carries #1301's 01138b3 underneath, and #1301 is now approved separately.

The "what merging does, and doesn't do" section is the part I most wanted to see. It says plainly that the image redeploy reuses the existing task definition, so dev and demo run the new code without LIF_ORG_KEY until someone deploys the stacks. That's the exact trap #1239 hit, which cost four hours of crash-looping on dev. Because this one fails safe, as unknown rather than a crash, the worst case is a statistic that stays unknown until someone deploys, not an outage.

One correction to that section: don't use aws-deploy.sh for the stack deploy. I found this while fixing the #1239 incident, and it's written up in #1287:

• aws-deploy.sh calls login() unconditionally (line 20), which sources scripts/login.sh.
• login.sh sets AWS_CONFIG_FILE=$PWD/.aws/config and runs a bare aws sso login with no --profile.
• The repo-local .aws/config defines only [profile AWS-Gates-CLIR] and no default. So which account the rest of the script acts on depends on the operator's ambient AWS environment, not on the script. On my machine it resolved to a different account entirely.

The safe equivalent pins the profile and calls the underlying script directly:

export AWS_PROFILE=lif
aws sts get-caller-identity --query Account --output text   # expect 381492161417
aws s3 sync --size-only --exclude="*" --include="*.yml" --include="*.params" \
  cloudformation s3://clir-cf-templates/dev-lif
./scripts/cfn-deploy.sh -s dev-lif-query-planner-org1 \
  -t "https://clir-cf-templates.s3.amazonaws.com/dev-lif/service.yml" \
  -r us-east-1 -p cloudformation/dev-lif-query-planner-org1.params \
  --no-wait-for-completion --changeset-name dev-lif-query-planner-org1-dev
./scripts/cfn-wait.sh dev-lif-query-planner-org1 us-east-1

That's what I used to fix the identity-mapper stacks. Worth swapping into the PR body so whoever does the deploy doesn't use the risky path.

The per-org value is proven syntax, not new syntax. A single shared include has to yield org1/org2/org3. The new entry uses Fn::Sub: "${OrganizationName}", the same long form that three entries in this include already deploy successfully (LIF_QUERY_CACHE_URL at :4, the orchestrator URL at :7, the SSM ARN at :34). So it works inside an AWS::Include snippet, and it resolves to the same OrganizationName that names query-cache-org1.

Mutation-checked, all three killed:

mutation	result
drop the blank → unknown fallback	1 failed
never pass org_key into the config	1 failed
omit org_key from query_completed	2 failed

The first one matters most. Blank is exactly what a CloudFormation Value: yields when its source is missing, so without that fallback, a half-deployed stack would record an empty string rather than a recognisable unknown. 78 pass in the affected components and 882 across the full suite.