diff --git a/.gitignore b/.gitignore index cf96f36..5de3a6e 100644 --- a/.gitignore +++ b/.gitignore @@ -263,6 +263,7 @@ terraform.rc backend/lambda-deploy.zip backend/lambda-placeholder.zip backend/placeholder.js +backend/build/ response.json logs.txt diff --git a/.tasks/task-deploy-infra/2026-08-30-144740-review.md b/.tasks/task-deploy-infra/2026-08-30-144740-review.md new file mode 100644 index 0000000..0c4a3b9 --- /dev/null +++ b/.tasks/task-deploy-infra/2026-08-30-144740-review.md @@ -0,0 +1,95 @@ +# Dev infrastructure deployment: state backend reconciliation, variable wiring, and conditional CloudFront/WAF/DNS (v1 review) + +This change gets the full Taskly Terraform stack to apply cleanly on a fresh AWS account that is missing several account-level capabilities. It reconciles the S3 state backend to the real bucket, wires every required root-module variable through the thin `dev` wrapper (including sensitive `documentdb_master_password`/`jwt_signing_key` that had no home before), and makes three resources conditional so an unverified/unprovisioned account still converges: CloudFront distributions (account not CloudFront-verified), the WAF→API-Gateway association (WAFv2 cannot attach to an HTTP API v2 stage), and Route 53 failover records (no hosted zone). A new `scripts/build-lambda.sh` produces the four Lambda zips as one full-backend bundle and prunes runtime-provided/unused `node_modules` to fit Lambda's 250 MB unzipped limit. The orchestrator independently confirmed `terraform plan` is idempotent, 219 resources applied, and `GET /api/health` returns 200 with `database:connected`. + +Watch for: (1) the Lambda bundle relies on the `nodejs20.x` runtime providing all `@aws-sdk/*` packages — including `@aws-sdk/s3-request-presigner`, imported at module-load time by `routes/upload.js` (confirmed; a future runtime that stops bundling the SDK would break cold start). (2) Disabling CloudFront and the WAF association narrows the intended security architecture — the uploads path loses the CDN and the API runs with an unattached WebACL, i.e. no active WAF protection (confirmed; documented, but worth a conscious sign-off). (3) A fresh apply on another account still defaults `enable_cloudfront = true` and will fail on a CloudFront-unverified account until the operator sets the flag (confirmed). + +**Verdict**: APPROVED + +## High-level view + +The conditional-resource work is correct. CloudFront distributions and their bucket policies are gated with `count`, the self-references were updated to `[0]`, and the outputs use `one(...)` with graceful fallbacks — `uploads_distribution_domain_name` degrades to the uploads S3 regional domain so the Lambda's `CDN_DOMAIN` stays valid, while ID/ARN outputs return `""`. The WAF association is gated behind a flag defaulting to `false` (WAFv2 regional WebACLs genuinely cannot attach to API Gateway v2), and DNS failover records are gated on a non-empty `hosted_zone_id`. All guards reference resources that exist, and the root module declares every variable the dev wrapper passes through. + +The Lambda bundle — one identical full-backend zip for all four functions — resolves every handler path regardless of which zip loads, and the only `sharp`/`@img` consumer (`image-processor.js`) is not one of the four deployed functions, so pruning those is safe. Pruning `@aws-sdk`/`@smithy` is safe *today* because `nodejs20.x` bundles AWS SDK v3 and the deployed health path proved the full `server.js` module graph (which statically imports the SDK via `routes/upload.js` and `config/aws.js`) loads. This is a runtime-version-coupled assumption, not an intrinsic property of the code. + +The security posture change should be an explicit decision, not a silent default: with CloudFront off, uploads are served directly from S3 regional domains and the frontend has no CDN/OAC edge; with the WAF association off, the created WebACL protects nothing. Both are documented, and the WebACL-still-created design keeps the resource ready to attach once the API is fronted by CloudFront. + +Secret handling is clean: the two secrets are declared `sensitive`, live only in a gitignored `secret.auto.tfvars` that is not tracked, and no real values appear in committed tfvars or the deployment report. + +
+Issues (4) + +1. **AWS SDK runtime coupling** — the Lambda zips omit `@aws-sdk`/`@smithy` and depend on `nodejs20.x` providing them; `routes/upload.js` imports `@aws-sdk/s3-request-presigner` at load time, so a runtime upgrade that stops bundling the SDK would fail cold start. Pin the runtime or add the SDK packages to the bundle before moving off `nodejs20.x`. +2. **WAF provides no active protection** — with `enable_api_gateway_association = false` (the default) the WebACL exists but is attached to nothing, so the API has no WAF in front of it. Confirm this is acceptable for dev and track fronting the API with CloudFront (CLOUDFRONT-scope WebACL) to restore protection. +3. **CloudFront-off weakens the upload/edge path** — uploads are served from the S3 regional domain and the frontend has no OAC/CDN edge. Fine as a documented temporary state; re-enable once the account is CloudFront-verified. +4. **Fresh-apply default trips on unverified accounts** — `enable_cloudfront` defaults to `true` in both root and dev, so a fresh apply on another CloudFront-unverified account fails until the operator sets it false. Document this as a required pre-apply input for new accounts. + +
+ +
+Details + +### Conditional CloudFront: count guards, self-references, and output fallbacks + +Both distributions and their two `aws_s3_bucket_policy` resources are gated with `count = var.enable_distributions ? 1 : 0`, and the bucket policies' `AWS:SourceArn` self-references were updated to `aws_cloudfront_distribution.frontend[0].arn` / `uploads[0].arn`. The outputs switched to `one(...) != null ? ... : ""`, avoiding an index into an empty list. The load-bearing one is `uploads_distribution_domain_name`, which falls back to `var.uploads_bucket_regional_domain_name` — that value feeds `module.lambda`'s `cdn_domain` / `CDN_DOMAIN`, so the application still gets a resolvable domain instead of `""`. When disabled, the root `cloudfront_frontend_url` output is `""` and the frontend bucket has no OAC policy — the frontend is not edge-served in this state. That is the intended trade-off for an unverified account, and the default remains `true` so verified accounts get the full topology. + +### WAF WebACL created but unattached on HTTP API v2 + +The `aws_wafv2_web_acl_association.api_gateway` resource is gated behind `var.enable_api_gateway_association` (default `false`), and the root `module "waf"` block does not override it. WAFv2 regional WebACLs cannot associate with API Gateway v2 (HTTP API) stages, and Taskly's API is `aws_apigatewayv2_api`. The net posture is that the API runs with no WAF actively in front of it — the created-but-unattached WebACL is inert until the API is fronted by CloudFront (CLOUDFRONT scope) or migrated to a REST API. This is a security reduction versus the stated architecture, so it should be an explicit sign-off rather than an implicit default. + +### DNS failover gated on hosted zone + +`aws_route53_record.api_primary` and `api_secondary` are gated with `count = var.hosted_zone_id != "" ? 1 : 0` (default `""`); the rest of the disaster-recovery module still applies without them, and the guard references only `var.hosted_zone_id`, so there is no dependency on a disabled resource. Supplying a real zone later brings the records back with no other change. + +### Lambda bundle: one full-backend zip, pruned to fit 250 MB + +``` + build-lambda.sh + backend/ ──────────────────────► api-handler.zip ──┬─► taskly-dev-api + (full source + prod node_modules, ├─► achievement-processor (identical copy) + minus @aws-sdk/@smithy/@img/sharp/core-js) ├─► notification-processor (identical copy) + └─► email-processor (identical copy) +``` + +One identical full-backend bundle to all four functions means every handler path (`index.handler`, `lambda/processors/*.handler`) resolves no matter which zip loads. Pruning `sharp`/`@img` is confirmed safe: the only importer is `lambda/processors/image-processor.js`, which is not one of the four deployed handlers and is not transitively imported by `server.js` or `lambda/handler.js`. + +Pruning `@aws-sdk`/`@smithy` is the load-bearing assumption. `server.js` statically imports `routes/upload.js`, which statically imports `@aws-sdk/s3-request-presigner` and `@aws-sdk/client-s3`; `config/aws.js`, `utils/secrets.js`, `services/emailService.js`, and the processors import other `@aws-sdk/client-*` packages. These are top-level ESM imports resolved at cold-start module load, so a missing one would throw `ERR_MODULE_NOT_FOUND` before `/api/health` could return. The observed 200 proves the full module graph loaded and the `nodejs20.x` runtime provided all of them — the prune is verified safe on the current runtime. The residual risk is version coupling: it depends on the runtime bundling AWS SDK v3, which AWS has signaled will not hold on future runtimes. The runtime pin and the prune list are now coupled and must move together. + +The script fetches the DocumentDB global TLS bundle into the archive root, is idempotent, and only zips entries that exist (missing optional dirs warn rather than fail). No `*.zip` is tracked in git. + +### Backend and variable wiring + +Every pass-through the dev wrapper sends into `module.taskly` (`documentdb_master_password`, `jwt_signing_key`, `ses_domain`, `hosted_zone_id`, `domain_name`, `documentdb_instance_class`, `documentdb_instance_count`, `vpc_cidr`, `enable_cloudfront`) is declared in the root module — verified by name, so no pass-through errors on an undeclared variable. The previously-undeclared tuning variables (`api_handler_memory`, throttling/WAF/log/budget knobs) are now declared in the dev `variables.tf` to silence undeclared-variable warnings; the file's own note states the root does not yet consume several of them, so they are inert placeholders, not functional wiring — worth knowing that setting e.g. `api_handler_memory` in tfvars currently has no effect. The dev `outputs.tf` re-exports nine root outputs, all of which exist in the root. The backend bucket rename matches the real bucket; the lock table was created out-of-band. + +### S3 replication and monitoring ordering fixes + +Two apply-time races were fixed with explicit edges. The uploads replication config now `depends_on = [aws_s3_bucket_versioning.uploads_replica]` (versioning must be enabled on the destination before a replication config referencing it can be created) and adds `delete_marker_replication { status = "Enabled" }` (required by the V2 replication schema S3 selects once a `filter {}` is present). Both referenced resources exist in the module. The monitoring module now consumes `module.lambda.api_handler_log_group_name` — sourced from `aws_cloudwatch_log_group.api_handler.name` — instead of an interpolated `/aws/lambda/${name}` string, building a real dependency edge so metric filters are not created before the log group exists. This is a genuine ordering fix, not cosmetic. + +### Secret-leak check + +The two secrets are declared `sensitive`, `documentdb_endpoint` outputs are `sensitive`, and the new `infrastructure/.gitignore` excludes `*.auto.tfvars`, `environments/*/secret.auto.tfvars`, state files, and plan output. `git ls-files` shows no tracked `secret.auto.tfvars`, and the deployment report redacts the password. + +
+ +
+File map + +- `infrastructure/environments/dev/backend.tf` — state bucket reconciled to the real account-suffixed bucket. +- `infrastructure/environments/dev/main.tf` — pass secrets, DNS/email, sizing, and `enable_cloudfront` into `module.taskly`. +- `infrastructure/environments/dev/variables.tf` — declare required secrets, DNS/sizing, and previously-undeclared tuning vars. +- `infrastructure/environments/dev/outputs.tf` — re-export nine root outputs for consumers. +- `infrastructure/environments/dev/terraform.tfvars` — set `enable_cloudfront = false` with rationale. +- `infrastructure/main.tf` — wire `enable_distributions` and switch monitoring to the lambda log-group output. +- `infrastructure/variables.tf` — root `enable_cloudfront` flag (default true). +- `infrastructure/modules/cloudfront/{main,outputs,variables}.tf` — conditional distributions/policies, `one(...)` outputs with S3 fallback, `enable_distributions` var. +- `infrastructure/modules/waf/{main,variables}.tf` — conditional API Gateway association, default false. +- `infrastructure/modules/disaster-recovery/dns-failover.tf` — DNS records gated on non-empty hosted zone. +- `infrastructure/modules/disaster-recovery/main.tf` — replication `depends_on` versioning + `delete_marker_replication`. +- `infrastructure/modules/lambda/outputs.tf` — new `api_handler_log_group_name` output. +- `infrastructure/.gitignore` — exclude state, secrets, plans. +- `scripts/build-lambda.sh` — build four full-backend Lambda zips, prune SDK/native modules, fetch TLS bundle. +- `.tasks/task-deploy-infra/**` — task state and deployment report. + +Full diff: `git diff main -- infrastructure/ scripts/build-lambda.sh` + +
diff --git a/.tasks/task-deploy-infra/DEPLOYMENT_REPORT.md b/.tasks/task-deploy-infra/DEPLOYMENT_REPORT.md new file mode 100644 index 0000000..305a925 --- /dev/null +++ b/.tasks/task-deploy-infra/DEPLOYMENT_REPORT.md @@ -0,0 +1,200 @@ +# Taskly Dev Infrastructure — Deployment Report + +**Task:** task-deploy-infra / FEAT-003 +**Date:** 2026-08-30 +**Environment:** dev +**AWS Account:** 583168584925 +**Region:** us-east-1 +**Terraform:** v1.9.8 +**Branch:** fix/taskly-deployment-issues + +## Result: SUCCESS + +`terraform apply` converged for the dev environment. A follow-up apply reports +**"No changes. Your infrastructure matches the configuration."** (idempotent). +`terraform output` resolves all documented outputs. + +The API is fully live end to end: **`/api/health` returns HTTP 200 with +`"database":"connected"`**, proving API Gateway -> Lambda (in VPC) -> DocumentDB +connectivity all work. + +## Resources created + +**219 resources** managed in state (S3 backend: +`taskly-terraform-state-583168584925`, key `environments/dev/terraform.tfstate`, +lock table `taskly-terraform-locks`). + +Key resources by type: + +| Type | Count | Notes | +|------|-------|-------| +| Lambda functions | 5 | api handler + 3 event processors + secret-rotation | +| API Gateway (HTTP API) | 1 API + 11 routes + stage | `aws_apigatewayv2_*` | +| DocumentDB | 1 cluster + 1 instance (db.t3.medium) | available | +| VPC | 1 VPC, 4 subnets, 6 interface endpoints, NAT GW | | +| SQS queues | 5 | email/notification + DLQs | +| Secrets Manager | 4 secrets (+ rotation) | documentdb creds, jwt, ses smtp | +| Cognito | user pool + app client | | +| WAF | 1 regional WebACL | created; see WAF note below | +| CloudWatch | 7 log groups, 6 metric filters, 5 alarms | | +| S3 | 4 buckets (frontend, uploads, uploads-replica, logs) | + cross-region replication | +| IAM | 10 roles, 16 policies, 29 attachments | | + +## Key outputs + +| Output | Value | +|--------|-------| +| `api_gateway_url` | `https://bvju0gyni7.execute-api.us-east-1.amazonaws.com` | +| `cognito_user_pool_id` | `us-east-1_l0jRjqILW` | +| `cognito_client_id` | `a1i9m0h2tqf5hu2ddpsq6bcdf` | +| `s3_uploads_bucket` | `taskly-dev-uploads-583168584925` | +| `lambda_function_name` | `taskly-dev-api` | +| `cloudfront_frontend_url` | `""` (CloudFront disabled — see manual prerequisites) | +| `documentdb_endpoint` | sensitive — `taskly-dev-docdb-cluster.cluster-c6psyaugmxvj.us-east-1.docdb.amazonaws.com` (not printed by `terraform output`) | + +## Verification evidence + +### Lambda (`taskly-dev-api`) +``` +aws lambda get-function --function-name taskly-dev-api --region us-east-1 \ + --query 'Configuration.[State,LastUpdateStatus,Runtime,Handler,MemorySize,Timeout]' +=> ["Active", "Successful", "nodejs20.x", "index.handler", 512, 29] +``` +All three event processors are also `Active`: +`taskly-dev-achievement-processor`, `taskly-dev-notification-processor`, +`taskly-dev-email-processor`. + +### DocumentDB +``` +aws docdb describe-db-clusters --db-cluster-identifier taskly-dev-docdb-cluster \ + --region us-east-1 --query 'DBClusters[0].Status' +=> "available" +``` + +### API Gateway `/api/health` +``` +curl -s -w 'HTTP %{http_code}' https://bvju0gyni7.execute-api.us-east-1.amazonaws.com/api/health +=> HTTP 200 +{"status":"OK","message":"Taskly API Server is running", + "timestamp":"2026-08-30T14:44:13.334Z","environment":"production", + "version":"1.0.0","database":"connected"} +``` +This is the ideal outcome: **HTTP 200** with **`database:connected`** confirms the +full request path API Gateway -> Lambda (VPC) -> DocumentDB works, including +`kms:Decrypt` on the secret and `AWSLambdaVPCAccessExecutionRole`. + +### DocumentDB credentials secret +``` +aws secretsmanager get-secret-value --secret-id taskly/dev/documentdb-credentials \ + --region us-east-1 --query SecretString +=> host = taskly-dev-docdb-cluster.cluster-c6psyaugmxvj.us-east-1.docdb.amazonaws.com + port = 27017, dbname = taskly, engine = mongo, username = taskly_admin + (password redacted) +``` +Host field is correctly populated with the DocumentDB cluster endpoint. + +### CloudFront +``` +aws cloudfront list-distributions --query "DistributionList.Items[]" +=> None +``` +Intentionally not created on this account (see manual prerequisites). + +## Fixes applied during this feature (committed on the branch) + +1. **Lambda bundle exceeded 250 MB unzipped limit** (`InvalidParameterValueException: + Unzipped size must be smaller than 262144000 bytes`). The full production + `node_modules` was ~310 MB. `scripts/build-lambda.sh` now prunes packages that + are provided by the Node.js 20 Lambda runtime or unused by the four deployed + handlers: `@aws-sdk`, `@smithy` (runtime-provided AWS SDK v3), `@img` + `sharp` + (native image libs used only by `image-processor.js`, which is not deployed), + and `core-js` (no direct import). Bundle now unzips to ~238 MB. Zips rebuilt and + re-uploaded to `s3://taskly-dev-uploads-583168584925/deploy/`. + +2. **S3 cross-region replication ordering + schema.** Added + `depends_on = [aws_s3_bucket_versioning.uploads_replica]` (was racing ahead of + destination versioning) and `delete_marker_replication { status = "Enabled" }` + (required by the V2 replication schema when a `filter` is present). + +3. **WAF association incompatible with HTTP API.** WAFv2 regional WebACLs cannot be + associated with API Gateway v2 (HTTP API) stages. Added + `var.enable_api_gateway_association` (default `false`) guarding + `aws_wafv2_web_acl_association.api_gateway`. The WebACL itself is still created. + +4. **CloudFront distributions made optional.** The account is not verified for + CloudFront (`AccessDenied: Your account must be verified before you can add new + CloudFront resources`). Added `var.enable_cloudfront` (root) / + `var.enable_distributions` (cloudfront module), set to `false` for dev. When + disabled, `cdn_domain` for the API Lambda falls back to the uploads S3 bucket + regional domain name so all downstream resources deploy and function. + +5. **CloudWatch metric-filter / log-group race.** The monitoring module received + the log group name as an interpolated string, so Terraform did not order it after + the log group resource (`ResourceNotFoundException: The specified log group does + not exist`). Added `api_handler_log_group_name` output on the lambda module and + wired the monitoring module to it, creating a proper dependency edge. + +6. **Dev environment outputs.** Added `infrastructure/environments/dev/outputs.tf` + to re-export the root `module.taskly` outputs (api_gateway_url, cognito ids, + uploads bucket, lambda name, documentdb endpoint, cloudfront url) so + `terraform output` works at the dev level. + +## Estimated monthly cost (~$130/month, dev) + +Per the DEPLOYMENT.md cost reference table: + +| Resource | Monthly cost (dev) | +|----------|-------------------| +| DocumentDB (1x db.t3.medium) | ~$60 | +| NAT Gateway | ~$32 | +| VPC Interface Endpoints (4x) | ~$28 | +| Lambda + API Gateway | ~$5 | +| S3 + CloudFront | ~$3 | +| Everything else | ~$5 | +| **Total** | **~$130/month** | + +Cost reduction options (from DEPLOYMENT.md): disable VPC interface endpoints in dev +(`enable_interface_endpoints = false`) and keep the single NAT Gateway (default). +Note CloudFront is currently disabled, so the S3+CloudFront line is closer to ~$1. + +## Remaining manual prerequisites + +1. **CloudFront (account verification).** The AWS account must be verified by AWS + Support before CloudFront distributions can be created. Once verified, set + `enable_cloudfront = true` in + `infrastructure/environments/dev/terraform.tfvars` and re-apply. This will + create the frontend + uploads distributions and re-point the Lambda + `CDN_DOMAIN` at the CloudFront domain. Open a case at + https://console.aws.amazon.com/support/home and reference the error + "Your account must be verified before you can add new CloudFront resources". + +2. **SES domain verification.** The SES identity is created for `taskly.app`, a + domain the account does not own, so it stays in "pending verification". + Verification requires publishing the SES DNS records (TXT/CNAME) in the DNS zone + of a domain you control. Until verified, outbound email is limited to the SES + sandbox / verified identities. + +3. **Route53 DNS failover (disaster recovery).** The DR module's `aws_route53_record` + resources are guarded by `hosted_zone_id != ""` and are intentionally NOT created + (the account has no hosted zone). To enable API DNS failover, register a domain, + create a Route53 hosted zone, then set `hosted_zone_id` and `domain_name` in the + dev tfvars and re-apply. + +4. **WAF protection for the API.** The WebACL exists but is not attached (WAFv2 does + not support HTTP API v2 stages). To protect the API with WAF, either front the + HTTP API with CloudFront and attach a `CLOUDFRONT`-scope WebACL, or migrate to an + API Gateway REST (v1) API and set `enable_api_gateway_association = true`. + +## Teardown + +To stop all charges, run `terraform destroy` from +`infrastructure/environments/dev` (with the same secret `*.auto.tfvars` present). +See the Teardown section of `DEPLOYMENT.md` for the full procedure. Note the S3 +buckets with versioning and the DocumentDB cluster may require emptying / final +snapshot handling. + +## Secrets handling + +Secret values (`documentdb_master_password`, `jwt_signing_key`) live only in +`infrastructure/environments/dev/secret.auto.tfvars`, which is **gitignored** and +was **NOT committed**. No real secrets appear in the repo or in this report. diff --git a/.tasks/task-deploy-infra/context.json b/.tasks/task-deploy-infra/context.json new file mode 100644 index 0000000..eac801d --- /dev/null +++ b/.tasks/task-deploy-infra/context.json @@ -0,0 +1,30 @@ +{ + "project_type": "Serverless AWS application (Node.js Express API on Lambda + API Gateway) with Terraform-managed infrastructure", + "language": "HCL (Terraform) for infra; Node.js 20 (ESM) for Lambda backend", + "build_system": "Terraform >= 1.5.0 (installed v1.9.8 at /usr/local/bin/terraform). Lambda bundles built with npm from backend/ and zipped.", + "test_framework": "n/a for infra deployment (jest exists for backend but not used here)", + "build_command": "cd infrastructure/environments/dev && terraform init && terraform plan && terraform apply", + "test_command": "terraform validate; then post-apply verification via aws CLI + curl on API Gateway /api/health", + "verification_instructions": "1) terraform validate passes. 2) terraform apply completes. 3) terraform output resolves (api_gateway_url, cognito ids, cloudfront url, uploads bucket, lambda name). 4) aws lambda get-function on taskly-dev-api succeeds. 5) curl https:///api/health returns JSON (may be 503 DATABASE_UNAVAILABLE until secret host populated, which is acceptable proof of routing+lambda). 6) aws docdb describe-db-clusters shows taskly-dev-docdb-cluster available.", + "snapshot_or_generated_files": "Lambda deployment zips built from backend/: deploy/api-handler.zip (index.mjs + lambda/ + server.js + config/controllers/middleware/models/routes/services/utils + node_modules + global-bundle.pem at ROOT), deploy/achievement-processor.zip, deploy/notification-processor.zip, deploy/email-processor.zip. Uploaded to the uploads S3 bucket under deploy/. Rebuild command lives in scripts/build-lambda.sh (to be created).", + "setup_instructions": "Terraform CLI installed via releases.hashicorp.com linux_amd64 zip -> /usr/local/bin/terraform. Node 20 available via nvm: source ~/.nvm/nvm.sh && nvm use 20. AWS creds valid (account 583168584925, us-east-1).", + "environment_constraints": "OPEN_INTERNET. AWS creds valid. Provisions REAL BILLABLE resources (~$130/mo dev: DocumentDB db.t3.medium, NAT GW, VPC interface endpoints, Lambda, API GW, WAF, CloudFront, cross-region S3 replica). No registered domain / Route53 hosted zone in account -> DR module DNS failover Route53 records CANNOT apply and must be made conditional. SES domain identity for a domain we don't own will be created but stay unverified (does not block apply).", + "contribution_requirements": "Do NOT commit real secrets. Secret values (documentdb_master_password, jwt_signing_key) supplied via TF_VAR_ env vars or a gitignored .auto.tfvars. Repo changes committed locally on branch fix/taskly-deployment-issues; orchestrator publishes.", + "key_patterns": "Root module in infrastructure/ wired by environments/dev/main.tf (thin wrapper). Root providers.tf defines default + us_east_1 + dr(us-west-2) providers. main.tf lambda module reads zips from module.s3.uploads_bucket_id under deploy/. Backend Lambda uses @vendia/serverless-express; handler=index.handler (index.mjs re-exports lambda/handler.js). Processors handler=lambda/processors/.handler.", + "relevant_files": [ + "infrastructure/backend.tf", + "infrastructure/environments/dev/backend.tf", + "infrastructure/environments/dev/main.tf", + "infrastructure/environments/dev/variables.tf", + "infrastructure/environments/dev/terraform.tfvars", + "infrastructure/variables.tf", + "infrastructure/main.tf", + "infrastructure/modules/disaster-recovery/dns-failover.tf", + "infrastructure/modules/lambda/main.tf", + "backend/index.mjs", + "backend/lambda/handler.js", + "backend/lambda/processors/", + "backend/package.json" + ], + "directory_structure": "infrastructure/{main,providers,variables,outputs,backend}.tf; infrastructure/environments/{dev,staging,prod}/{main,backend,variables}.tf+terraform.tfvars; infrastructure/modules/<15 modules>. backend/ holds Lambda source. .tasks/ holds task state." +} diff --git a/.tasks/task-deploy-infra/features/FEAT-001.json b/.tasks/task-deploy-infra/features/FEAT-001.json new file mode 100644 index 0000000..453d199 --- /dev/null +++ b/.tasks/task-deploy-infra/features/FEAT-001.json @@ -0,0 +1,32 @@ +{ + "id": "FEAT-001", + "type": "chore", + "description": "Prepare the deployment: reconcile the Terraform state backend, create the missing DynamoDB lock table, wire the required root-module variables (secrets + sizing) through the dev wrapper, make the disaster-recovery DNS failover Route53 records conditional (account has no hosted zone), and confirm `terraform init` + `terraform validate` succeed for the dev environment. This is the environment-setup baseline feature: no resources are applied yet, but the working directory must init and validate cleanly.", + "status": "completed", + "steps": [ + "Confirm terraform is on PATH: run `terraform version` (expect >= 1.5.0; v1.9.8 is installed at /usr/local/bin/terraform). If missing, download terraform 1.9.8 linux_amd64 from releases.hashicorp.com and place at /usr/local/bin/terraform.", + "Create the DynamoDB lock table if absent: `aws dynamodb create-table --table-name taskly-terraform-locks --attribute-definitions AttributeName=LockID,AttributeType=S --key-schema AttributeName=LockID,KeyType=HASH --billing-mode PAY_PER_REQUEST --region us-east-1`, then wait for ACTIVE via `aws dynamodb wait table-exists --table-name taskly-terraform-locks --region us-east-1`.", + "Reconcile the state backend to the bucket that ACTUALLY exists (taskly-terraform-state-583168584925). Edit infrastructure/environments/dev/backend.tf: change `bucket = \"taskly-terraform-state\"` to `bucket = \"taskly-terraform-state-583168584925\"`. Keep key=environments/dev/terraform.tfstate, dynamodb_table=taskly-terraform-locks, encrypt=true. Ensure bucket versioning is enabled: `aws s3api put-bucket-versioning --bucket taskly-terraform-state-583168584925 --versioning-configuration Status=Enabled`.", + "Wire required + sizing variables through the dev wrapper. Edit infrastructure/environments/dev/variables.tf to ADD declarations for: documentdb_master_password (string, sensitive, no default), jwt_signing_key (string, sensitive, no default), ses_domain (string, default \"taskly.app\"), hosted_zone_id (string, default \"\"), domain_name (string, default \"api.taskly.app\"), documentdb_instance_class (string, default \"db.t3.medium\"), documentdb_instance_count (number, default 1), vpc_cidr (string, default \"10.0.0.0/16\"). Also add declarations (with defaults matching terraform.tfvars) for the currently-undeclared tfvars so `terraform` does not warn/ignore them and so they can be threaded if needed: api_handler_memory, api_handler_timeout, processor_memory, reserved_concurrency_api, reserved_concurrency_processors, throttling_burst_limit, throttling_rate_limit, waf_rate_limit, waf_rate_limit_action, log_retention_days, monthly_budget_amount, alarm_email_endpoints (list(string) default []), cloudfront_price_class, cors_allowed_origins (list(string)). These extra ones are accepted-but-currently-unconsumed by the root module; declaring them prevents 'value for undeclared variable' warnings. Do NOT invent new root-module wiring for sizing vars the root module does not accept — the root module currently only accepts documentdb_instance_class/count among sizing vars.", + "Edit infrastructure/environments/dev/main.tf to pass the newly-required variables into module \"taskly\": documentdb_master_password = var.documentdb_master_password, jwt_signing_key = var.jwt_signing_key, ses_domain = var.ses_domain, hosted_zone_id = var.hosted_zone_id, domain_name = var.domain_name, documentdb_instance_class = var.documentdb_instance_class, documentdb_instance_count = var.documentdb_instance_count, vpc_cidr = var.vpc_cidr. Keep the existing aws_region/environment/project_name/cost_center/owner passthroughs.", + "Make the DR module's DNS failover records optional. Edit infrastructure/modules/disaster-recovery/dns-failover.tf: add `count = var.hosted_zone_id != \"\" ? 1 : 0` to aws_route53_record.api_primary and aws_route53_record.api_secondary. Guard the aws_route53_health_check.api similarly (count based on hosted_zone_id) OR keep it (health checks don't require a zone) — prefer guarding both records only, and update their internal references (e.g. health_check_id = aws_route53_health_check.api.id stays valid only if health check is unconditional; keep health check unconditional so it always exists). Update the outputs health_check_id/maintenance_url to remain valid. Where a resource becomes a count-indexed list, fix any self-references accordingly. The goal: when hosted_zone_id == \"\" (the dev default), NO aws_route53_record resources are planned, and validate/plan succeed.", + "Create a gitignored secrets tfvars for local convenience OR rely on TF_VAR_ env vars. Add `environments/*/secret.auto.tfvars` and `*.auto.tfvars` (secret-bearing) to infrastructure/.gitignore (create the .gitignore if absent). Do NOT commit real secret values. The actual secret values will be provided at plan/apply time via TF_VAR_documentdb_master_password and TF_VAR_jwt_signing_key environment variables (documented in the deployment report).", + "Run `cd infrastructure/environments/dev && terraform init` (this configures the S3 backend). Then run `terraform validate`. Fix any HCL errors surfaced (undeclared variables, type mismatches, count self-reference issues) until both succeed.", + "Run `terraform plan` with dummy TF_VAR secrets (TF_VAR_documentdb_master_password='DummyPlanPassw0rd123!' TF_VAR_jwt_signing_key='dummy-plan-jwt-key-not-for-apply') to confirm the plan generates with NO route53_record resources and NO errors. Capture the resource count. Do NOT apply in this feature." + ], + "acceptance_criteria": [ + "`aws dynamodb describe-table --table-name taskly-terraform-locks --region us-east-1` returns an ACTIVE table", + "infrastructure/environments/dev/backend.tf references bucket taskly-terraform-state-583168584925", + "`terraform init` in infrastructure/environments/dev succeeds against the S3 backend", + "`terraform validate` in infrastructure/environments/dev returns success", + "`terraform plan` (with dummy TF_VAR secrets and hosted_zone_id default \"\") completes with no errors and plans zero aws_route53_record resources", + "infrastructure/.gitignore excludes secret-bearing tfvars; no real secrets are committed" + ], + "verification": [ + "cd infrastructure/environments/dev && terraform init && terraform validate", + "cd infrastructure/environments/dev && TF_VAR_documentdb_master_password='DummyPlanPassw0rd123!' TF_VAR_jwt_signing_key='dummy-plan-jwt-key' terraform plan -no-color 2>&1 | tail -40", + "aws dynamodb describe-table --table-name taskly-terraform-locks --region us-east-1 --query 'Table.TableStatus'" + ], + "blocked_reason": null, + "findings": "Completed. Terraform v1.9.8 confirmed on PATH. DynamoDB lock table taskly-terraform-locks created (PAY_PER_REQUEST, LockID:S) and is ACTIVE. Backend bucket in environments/dev/backend.tf changed to the existing taskly-terraform-state-583168584925; versioning enabled on that bucket. Wired required secrets (documentdb_master_password, jwt_signing_key), DNS/email (ses_domain, hosted_zone_id, domain_name), and sizing (documentdb_instance_class/count, vpc_cidr) through environments/dev/main.tf into module \"taskly\"; declared all previously-undeclared tfvars in environments/dev/variables.tf with defaults matching terraform.tfvars to silence undeclared-variable warnings (root module only actually consumes documentdb_instance_class/count and vpc_cidr among sizing vars). Made DR route53 records conditional: added `count = var.hosted_zone_id != \"\" ? 1 : 0` to aws_route53_record.api_primary and api_secondary in modules/disaster-recovery/dns-failover.tf; kept aws_route53_health_check.api unconditional so health_check_id output stays valid; no external references to those records exist so no address fixes were needed. Created infrastructure/.gitignore (force-added, since root .gitignore line 213 ignores all .gitignore files) excluding *.auto.tfvars, environments/*/secret.auto.tfvars, .terraform/, *.tfstate. VERIFICATION: `terraform init` succeeded against S3 backend; `terraform validate` succeeded (only benign warnings: backend-ignored when root called as child module, and a pre-existing S3 lifecycle filter warning). `terraform plan` with dummy TF_VAR secrets and hosted_zone_id default \"\" completed with 0 errors: Plan = 211 to add, 0 change, 0 destroy, with ZERO aws_route53_record resources and 1 aws_route53_health_check. Did NOT run apply (per feature scope). NOTE for later features: pre-existing unrelated change to modules/secrets/lambda/rotation.zip is present in the tree but was NOT touched or committed by this feature. Real secrets must be supplied via TF_VAR_documentdb_master_password and TF_VAR_jwt_signing_key at apply time." +} diff --git a/.tasks/task-deploy-infra/features/FEAT-002.json b/.tasks/task-deploy-infra/features/FEAT-002.json new file mode 100644 index 0000000..285bba8 --- /dev/null +++ b/.tasks/task-deploy-infra/features/FEAT-002.json @@ -0,0 +1,27 @@ +{ + "id": "FEAT-002", + "type": "feat", + "description": "Build the four Lambda deployment bundles from backend/ and stage them, resolving the chicken-and-egg ordering (the uploads bucket that stores the zips is created by the same Terraform run). Approach: create a reusable build script that produces the zips, do a targeted `terraform apply -target=module.s3` first to create the uploads bucket, upload the zips to s3:///deploy/, then the full apply in FEAT-003 will find them. This keeps the lambda module unchanged (it already has lifecycle ignore_changes on s3_key).", + "status": "completed", + "steps": [ + "Create scripts/build-lambda.sh (repo root scripts/ dir; create if absent). It must: (a) source ~/.nvm/nvm.sh && nvm use 20; (b) cd backend; (c) npm install --omit=dev (production deps only); (d) download the DocumentDB TLS bundle to backend/global-bundle.pem from https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem; (e) produce build/api-handler.zip containing at ROOT level: index.mjs, server.js, package.json, schemas.js, global-bundle.pem, and directories lambda/ config/ controllers/ middleware/ models/ routes/ services/ utils/ node_modules/ (use `zip -r -q`); (f) produce build/achievement-processor.zip, build/notification-processor.zip, build/email-processor.zip. Because processors use handler path lambda/processors/.handler and import shared code (../server.js, ../utils/secrets.js) plus node_modules, the simplest correct approach is to make all four zips identical to the api-handler bundle (same full backend contents) so every handler path resolves. Document this in the script comments.", + "Make the script idempotent and parameterized by an output dir (default backend/build). Ensure zips have files at archive root (zip from inside backend/, not the parent).", + "Run the build script: `bash scripts/build-lambda.sh`. Confirm the four zips exist under backend/build and that `unzip -l backend/build/api-handler.zip | grep -E 'index.mjs|lambda/handler.js|node_modules/@vendia'` shows those entries at root.", + "Create the uploads bucket via targeted apply: `cd infrastructure/environments/dev && TF_VAR_documentdb_master_password=$TF_VAR_documentdb_master_password TF_VAR_jwt_signing_key=$TF_VAR_jwt_signing_key terraform apply -target=module.taskly.module.s3 -auto-approve`. (Real secret TF_VARs must be exported in the shell; see FEAT-003 for how they are generated. If not yet generated, generate them now and persist in the shell/session so FEAT-003 reuses the SAME values.)", + "Resolve the uploads bucket name: `terraform output -raw s3_uploads_bucket` (or `terraform state show` the s3 module). Upload the zips: `aws s3 cp backend/build/api-handler.zip s3:///deploy/api-handler.zip` and likewise achievement-processor.zip, notification-processor.zip, email-processor.zip under deploy/.", + "Verify all four objects exist: `aws s3 ls s3:///deploy/`." + ], + "acceptance_criteria": [ + "scripts/build-lambda.sh exists, is committed, and regenerates all four zips deterministically", + "backend/build/api-handler.zip has index.mjs and lambda/handler.js and node_modules/@vendia/serverless-express at archive ROOT (verified via unzip -l)", + "The uploads S3 bucket exists (created via targeted apply of module.s3)", + "s3:///deploy/ contains api-handler.zip, achievement-processor.zip, notification-processor.zip, email-processor.zip" + ], + "verification": [ + "unzip -l backend/build/api-handler.zip | grep -E 'index.mjs|lambda/handler.js|@vendia/serverless-express' | head", + "cd infrastructure/environments/dev && terraform output -raw s3_uploads_bucket", + "aws s3 ls s3://$(cd infrastructure/environments/dev && terraform output -raw s3_uploads_bucket)/deploy/" + ], + "blocked_reason": null, + "findings": "Uploads bucket name: taskly-dev-uploads-583168584925 (region us-east-1). All four zips uploaded to s3://taskly-dev-uploads-583168584925/deploy/ (api-handler.zip, achievement-processor.zip, notification-processor.zip, email-processor.zip; each ~101MB, identical full-backend bundle). scripts/build-lambda.sh created and committed; it makes all four zips identical to resolve every handler path. index.mjs and lambda/handler.js and node_modules/@vendia/serverless-express confirmed at archive ROOT. Secrets generated ONCE and persisted to infrastructure/environments/dev/secret.auto.tfvars (gitignored, NOT committed) so FEAT-003 reuses the exact same values for state consistency: documentdb_master_password (35 chars, prefix 'Aa1' + 32 hex, no forbidden / @ \" space chars) and jwt_signing_key (64 hex). openssl is NOT available in this sandbox; used /dev/urandom + od instead. Targeted apply 'terraform apply -target=module.taskly.module.s3 -auto-approve' created 10 resources with only warnings (backend-config-ignored on child-module call, and lifecycle rule filter/prefix deprecation) - no errors. Note: because it was a targeted apply, root outputs are not yet populated (terraform output -raw s3_uploads_bucket returns 'No outputs found'); bucket name read from state show. FEAT-003 full apply will populate outputs. Added backend/build/ to root .gitignore. Left pre-existing unstaged change to infrastructure/modules/secrets/lambda/rotation.zip (not part of this feature) uncommitted." +} diff --git a/.tasks/task-deploy-infra/features/FEAT-003.json b/.tasks/task-deploy-infra/features/FEAT-003.json new file mode 100644 index 0000000..6e9b676 --- /dev/null +++ b/.tasks/task-deploy-infra/features/FEAT-003.json @@ -0,0 +1,35 @@ +{ + "id": "FEAT-003", + "type": "feat", + "description": "Apply the full dev stack and verify it. Generate strong secret values (DocumentDB master password + JWT signing key), export as TF_VAR_ env vars (reused from FEAT-002 if already generated), run terraform apply for the dev environment, then verify outputs resolve, Lambda exists and is invokable, API Gateway responds, and DocumentDB is available. Report any resource that cannot apply with exact remediation.", + "status": "completed", + "steps": [ + "Generate strong secrets ONCE and persist for the session (do NOT commit): DocumentDB master password must satisfy DocumentDB rules (8-100 chars, printable ASCII, no / @ \" or space). Example generation: `export TF_VAR_documentdb_master_password=$(openssl rand -base64 24 | tr -d '/@\" ' | cut -c1-24)` then append a digit+upper to guarantee complexity; `export TF_VAR_jwt_signing_key=$(openssl rand -hex 32)`. Write these VALUES only into infrastructure/environments/dev/secret.auto.tfvars (which is gitignored) as a convenience AND keep them exported. Record that they are stored gitignored, never committed.", + "Ensure FEAT-002 used these exact same secret values for the targeted s3 apply (state consistency). If FEAT-002 used dummy/different values, that's fine for s3 (s3 module doesn't consume secrets), but the full apply must use the final real values.", + "Run the full apply: `cd infrastructure/environments/dev && terraform apply -auto-approve` (TF_VARs exported). Expect ~150-190 resources, 10-15 min (DocumentDB slowest). If Lambda creation fails with 'S3 key does not exist', confirm FEAT-002 uploaded the zips to the correct uploads bucket and re-apply.", + "If apply fails on any resource, capture the error, fix if it is a code/config issue (e.g. a module input mismatch), and re-apply. For resources that genuinely cannot apply because the account lacks a prerequisite (e.g. a real Route53 hosted zone / registered domain), confirm they are already guarded (DR route53 records were guarded in FEAT-001) and note any remaining ones as blocked with exact user remediation rather than fabricating resources.", + "After apply, capture outputs: `terraform output` and individually `terraform output -raw api_gateway_url`, `cognito_user_pool_id`, `cognito_client_id`, `cloudfront_frontend_url`, `s3_uploads_bucket`, `lambda_function_name`.", + "Verify Lambda: `aws lambda get-function --function-name taskly-dev-api --region us-east-1 --query 'Configuration.[State,LastUpdateStatus,Runtime,Handler]'`. Confirm State=Active. Optionally `aws lambda invoke` with a synthetic API GW v2 event for /api/health.", + "Verify DocumentDB: `aws docdb describe-db-clusters --db-cluster-identifier taskly-dev-docdb-cluster --region us-east-1 --query 'DBClusters[0].Status'` == available.", + "Verify API Gateway: `curl -s -o /dev/null -w '%{http_code}' https:///api/health` then `curl -s https:///api/health`. A 200 with database:connected is ideal; a 503 DATABASE_UNAVAILABLE still proves API Gateway->Lambda routing works (DB secret host may need the documentdb endpoint, which main.tf wires via module.secrets documentdb_endpoint — verify the secret's host field is populated: `aws secretsmanager get-secret-value --secret-id taskly/dev/documentdb-credentials --query SecretString --output text | head -c 200`). If host is empty or connectivity fails, document it as a known post-deploy step, not a hard failure of the infra apply.", + "Verify CloudFront: `aws cloudfront list-distributions --query \"DistributionList.Items[?contains(Comment,'taskly-dev')||contains(Origins.Items[0].DomainName,'taskly-dev')].[Id,DomainName,Status]\"` (or read from terraform state). Confirm a distribution exists.", + "Write a deployment report to .tasks/task-deploy-infra/DEPLOYMENT_REPORT.md summarizing: resources created (count + key ones), all key outputs (api endpoint, cloudfront domain, cognito pool/client ids, uploads bucket, lambda name), what was verified and how (with the actual command output values), estimated monthly cost (~$130 dev per DEPLOYMENT.md), and any manual prerequisites still remaining (e.g. SES domain verification requires DNS records for a domain the account must own; enabling DNS failover requires a Route53 hosted zone -> set hosted_zone_id + domain_name and re-apply)." + ], + "acceptance_criteria": [ + "`terraform apply` for dev completes; `terraform output` resolves all documented outputs without error", + "aws lambda get-function on taskly-dev-api returns State=Active", + "aws docdb describe-db-clusters shows taskly-dev-docdb-cluster status=available", + "curl to https:///api/health returns an HTTP response from the Lambda (200 preferred; 503 DATABASE_UNAVAILABLE acceptable as proof of routing, with the DB gap documented)", + "A CloudFront distribution for the frontend exists", + ".tasks/task-deploy-infra/DEPLOYMENT_REPORT.md exists with resources, outputs, verification evidence, cost, and remaining manual steps", + "No real secrets are committed to the repo" + ], + "verification": [ + "cd infrastructure/environments/dev && terraform output", + "aws lambda get-function --function-name taskly-dev-api --region us-east-1 --query 'Configuration.State'", + "aws docdb describe-db-clusters --db-cluster-identifier taskly-dev-docdb-cluster --region us-east-1 --query 'DBClusters[0].Status'", + "curl -s -w '\\nHTTP %{http_code}\\n' https://$(cd infrastructure/environments/dev && terraform output -raw api_gateway_url | sed 's#https://##')/api/health" + ], + "blocked_reason": null, + "findings": "SUCCESS - dev stack fully applied and verified. terraform apply converged (219 resources); a follow-up apply reports 'No changes'. Reused the SAME secrets from secret.auto.tfvars (gitignored, NOT committed) - not regenerated. KEY OUTPUTS: api_gateway_url=https://bvju0gyni7.execute-api.us-east-1.amazonaws.com ; cognito_user_pool_id=us-east-1_l0jRjqILW ; cognito_client_id=a1i9m0h2tqf5hu2ddpsq6bcdf ; s3_uploads_bucket=taskly-dev-uploads-583168584925 ; lambda_function_name=taskly-dev-api ; documentdb_endpoint=taskly-dev-docdb-cluster.cluster-c6psyaugmxvj.us-east-1.docdb.amazonaws.com (sensitive) ; cloudfront_frontend_url='' (disabled). VERIFICATION: Lambda taskly-dev-api State=Active LastUpdateStatus=Successful nodejs20.x index.handler (mem 512, timeout 29); all 3 processor Lambdas Active. DocumentDB taskly-dev-docdb-cluster status=available. /api/health => HTTP 200 with body {\"status\":\"OK\",...,\"database\":\"connected\"} (IDEAL - full API GW->Lambda(VPC)->DocumentDB path works). Secret taskly/dev/documentdb-credentials host field populated with the docdb cluster endpoint. WAF regional WebACL taskly-dev-api-waf exists (not associated: WAFv2 cannot attach to HTTP API v2). CloudFront: none (disabled).\n\nFIXES applied during apply (committed on branch, EXCLUDING secrets/zips):\n1) Lambda 250MB unzipped limit: scripts/build-lambda.sh now prunes @aws-sdk/@smithy (runtime-provided), @img/sharp (image-processor only, not deployed), core-js (unused) -> bundle ~238MB; zips rebuilt+re-uploaded.\n2) S3 replication: added depends_on versioning + delete_marker_replication (V2 schema).\n3) WAF association made conditional via var.enable_api_gateway_association=false (WAFv2 incompatible with HTTP API v2).\n4) CloudFront made optional via var.enable_cloudfront/enable_distributions (account NOT verified for CloudFront: 'AccessDenied: Your account must be verified'); set false for dev; cdn_domain falls back to uploads S3 regional domain. Outputs degrade gracefully.\n5) CloudWatch metric-filter/log-group race: added lambda output api_handler_log_group_name and wired monitoring module to it for a real dependency edge.\n6) Added infrastructure/environments/dev/outputs.tf to re-export module.taskly outputs so terraform output works at dev level.\n\nDEPLOYMENT_REPORT.md written to .tasks/task-deploy-infra/. Est cost ~$130/mo (DocumentDB ~$60, NAT ~$32, VPC endpoints ~$28, per DEPLOYMENT.md). REMAINING MANUAL PREREQS: (a) CloudFront needs AWS Support account verification then set enable_cloudfront=true and re-apply; (b) SES domain taskly.app stays pending (account doesn't own it) - needs DNS records; (c) Route53 DNS failover needs a hosted zone -> set hosted_zone_id+domain_name; (d) WAF-on-API needs CloudFront (CLOUDFRONT-scope ACL) or REST API v1. Teardown: terraform destroy from environments/dev. NO real secrets committed (verified secret.auto.tfvars and backend/build/*.zip are gitignored / not staged). Left pre-existing unrelated modules/secrets/lambda/rotation.zip change uncommitted." +} diff --git a/.tasks/task-deploy-infra/task.json b/.tasks/task-deploy-infra/task.json new file mode 100644 index 0000000..9db0fcf --- /dev/null +++ b/.tasks/task-deploy-infra/task.json @@ -0,0 +1,15 @@ +{ + "task_id": "task-deploy-infra", + "task_description": "Deploy the Taskly AWS infrastructure (dev environment) to a working state via Terraform. Reconcile the state backend, create the missing lock table, wire required root-module variables through the dev wrapper, make the disaster-recovery Route53 DNS failover conditional (account owns no hosted zone), build and stage the four Lambda deployment zips (resolving the uploads-bucket chicken-and-egg via targeted apply), then full apply and verify (outputs, Lambda active, DocumentDB available, API Gateway /api/health responds, CloudFront exists). Provisions real billable resources (~$130/mo). Report the maximal working subset plus any manual prerequisites (SES domain verification, DNS failover) that require a domain the account does not own.", + "status": "completed", + "feature_order": ["FEAT-001", "FEAT-002", "FEAT-003"], + "blocked_reason": null, + "verification": { + "build": "pass", + "tests": "pass", + "test_quality": "pass", + "docker_build": "skipped", + "summary": "Independently verified the deployed dev stack. `terraform validate` succeeds (only pre-existing benign S3 lifecycle deprecation warnings). `terraform plan -detailed-exitcode` returns exit 0 with 'No changes' (fully converged/idempotent). Lambda taskly-dev-api State=Active/Successful (nodejs20.x, index.handler); all three processor Lambdas Active. DocumentDB taskly-dev-docdb-cluster status=available. `terraform output` resolves: api_gateway_url=https://bvju0gyni7.execute-api.us-east-1.amazonaws.com, cognito_user_pool_id=us-east-1_l0jRjqILW, cognito_client_id=a1i9m0h2tqf5hu2ddpsq6bcdf, s3_uploads_bucket=taskly-dev-uploads-583168584925, lambda_function_name=taskly-dev-api, cloudfront_frontend_url=\"\" (disabled). curl https://bvju0gyni7.execute-api.us-east-1.amazonaws.com/api/health => HTTP 200 with {\"status\":\"OK\",...,\"database\":\"connected\"}, proving the full API Gateway -> Lambda(VPC) -> DocumentDB path. 'build/test' here map to terraform validate+plan since this is an infra deployment task (no unit-test suite is exercised). No real secrets committed; working tree clean.", + "notes": "Blocked-off subsets (documented, require account prerequisites the account lacks, not fabricated): CloudFront disabled until AWS Support account verification; SES identity for taskly.app stays pending (account does not own the domain); Route53 DNS failover skipped (no hosted zone); WAF WebACL created but not attached (WAFv2 cannot associate with HTTP API v2). Full remediation in DEPLOYMENT_REPORT.md." + } +} diff --git a/infrastructure/.gitignore b/infrastructure/.gitignore new file mode 100644 index 0000000..5b8b8e9 --- /dev/null +++ b/infrastructure/.gitignore @@ -0,0 +1,16 @@ +# Terraform state and working files +.terraform/ +.terraform.lock.hcl +*.tfstate +*.tfstate.* +crash.log +crash.*.log + +# Secret-bearing variable files — never commit real secrets +*.auto.tfvars +*.auto.tfvars.json +environments/*/secret.auto.tfvars +environments/*/secret.auto.tfvars.json + +# Terraform plan output +*.tfplan diff --git a/infrastructure/environments/dev/backend.tf b/infrastructure/environments/dev/backend.tf index cc53a95..67450ac 100644 --- a/infrastructure/environments/dev/backend.tf +++ b/infrastructure/environments/dev/backend.tf @@ -9,7 +9,7 @@ terraform { } backend "s3" { - bucket = "taskly-terraform-state" + bucket = "taskly-terraform-state-583168584925" key = "environments/dev/terraform.tfstate" region = "us-east-1" dynamodb_table = "taskly-terraform-locks" diff --git a/infrastructure/environments/dev/main.tf b/infrastructure/environments/dev/main.tf index 5ae62f1..7fea898 100644 --- a/infrastructure/environments/dev/main.tf +++ b/infrastructure/environments/dev/main.tf @@ -6,4 +6,21 @@ module "taskly" { project_name = var.project_name cost_center = var.cost_center owner = var.owner + + # Required secrets + documentdb_master_password = var.documentdb_master_password + jwt_signing_key = var.jwt_signing_key + + # DNS / Email + ses_domain = var.ses_domain + hosted_zone_id = var.hosted_zone_id + domain_name = var.domain_name + + # Sizing + documentdb_instance_class = var.documentdb_instance_class + documentdb_instance_count = var.documentdb_instance_count + vpc_cidr = var.vpc_cidr + + # Feature flags + enable_cloudfront = var.enable_cloudfront } diff --git a/infrastructure/environments/dev/outputs.tf b/infrastructure/environments/dev/outputs.tf new file mode 100644 index 0000000..6bb3c8e --- /dev/null +++ b/infrastructure/environments/dev/outputs.tf @@ -0,0 +1,52 @@ +# Dev Environment — Re-exported Outputs +# +# The dev environment is a thin wrapper that instantiates the root Taskly module +# as `module.taskly`. Terraform does not surface a child module's outputs at the +# root automatically, so we re-export the ones consumers (verification scripts, +# CI/CD, the deployment report) need here. + +output "environment" { + description = "Current deployment environment" + value = module.taskly.environment +} + +output "aws_region" { + description = "AWS region where resources are deployed" + value = module.taskly.aws_region +} + +output "api_gateway_url" { + description = "API Gateway endpoint URL" + value = module.taskly.api_gateway_url +} + +output "cloudfront_frontend_url" { + description = "CloudFront distribution URL for the frontend (empty when CloudFront is disabled/unverified)" + value = module.taskly.cloudfront_frontend_url +} + +output "cognito_user_pool_id" { + description = "Cognito User Pool ID" + value = module.taskly.cognito_user_pool_id +} + +output "cognito_client_id" { + description = "Cognito App Client ID" + value = module.taskly.cognito_client_id +} + +output "documentdb_endpoint" { + description = "DocumentDB cluster endpoint" + value = module.taskly.documentdb_endpoint + sensitive = true +} + +output "s3_uploads_bucket" { + description = "S3 uploads bucket name" + value = module.taskly.s3_uploads_bucket +} + +output "lambda_function_name" { + description = "API handler Lambda function name" + value = module.taskly.lambda_function_name +} diff --git a/infrastructure/environments/dev/terraform.tfvars b/infrastructure/environments/dev/terraform.tfvars index 8f3d5aa..297f5c5 100644 --- a/infrastructure/environments/dev/terraform.tfvars +++ b/infrastructure/environments/dev/terraform.tfvars @@ -35,6 +35,12 @@ alarm_email_endpoints = [] # CloudFront cloudfront_price_class = "PriceClass_100" +# This AWS account is not yet verified for CloudFront (the CloudFront API returns +# "AccessDenied: Your account must be verified before you can add new CloudFront +# resources"). Disable distributions so the rest of the stack deploys; the Lambda +# CDN_DOMAIN falls back to the uploads S3 bucket regional domain. Set to true and +# re-apply once AWS Support verifies the account for CloudFront. +enable_cloudfront = false # CORS cors_allowed_origins = ["http://localhost:3000", "http://127.0.0.1:3000"] diff --git a/infrastructure/environments/dev/variables.tf b/infrastructure/environments/dev/variables.tf index 38d9f5e..1eedfbe 100644 --- a/infrastructure/environments/dev/variables.tf +++ b/infrastructure/environments/dev/variables.tf @@ -27,3 +27,152 @@ variable "owner" { type = string default = "platform-team" } + +# ─── Required root-module secrets (no defaults) ─────────────────────────────── + +variable "documentdb_master_password" { + description = "Master password for the DocumentDB cluster" + type = string + sensitive = true +} + +variable "jwt_signing_key" { + description = "JWT signing key for legacy token compatibility" + type = string + sensitive = true +} + +# ─── DNS / Email ────────────────────────────────────────────────────────────── + +variable "ses_domain" { + description = "Domain name for SES identity verification" + type = string + default = "taskly.app" +} + +variable "hosted_zone_id" { + description = "Route 53 hosted zone ID for DNS failover (empty disables DNS records)" + type = string + default = "" +} + +variable "domain_name" { + description = "Domain name for the API" + type = string + default = "api.taskly.app" +} + +# ─── Sizing (consumed by root module) ───────────────────────────────────────── + +variable "documentdb_instance_class" { + description = "DocumentDB instance class" + type = string + default = "db.t3.medium" +} + +variable "documentdb_instance_count" { + description = "Number of DocumentDB instances" + type = number + default = 1 +} + +variable "vpc_cidr" { + description = "CIDR block for the VPC" + type = string + default = "10.0.0.0/16" +} + +# ─── Accepted-but-currently-unconsumed tuning variables ─────────────────────── +# Declared so terraform does not warn about undeclared variables present in +# terraform.tfvars. The root module does not currently wire these; defaults +# match the values in terraform.tfvars. + +variable "api_handler_memory" { + description = "Memory (MB) for the API handler Lambda" + type = number + default = 256 +} + +variable "api_handler_timeout" { + description = "Timeout (seconds) for the API handler Lambda" + type = number + default = 29 +} + +variable "processor_memory" { + description = "Memory (MB) for the processor Lambdas" + type = number + default = 128 +} + +variable "reserved_concurrency_api" { + description = "Reserved concurrency for the API handler Lambda" + type = number + default = 10 +} + +variable "reserved_concurrency_processors" { + description = "Reserved concurrency for the processor Lambdas" + type = number + default = 5 +} + +variable "throttling_burst_limit" { + description = "API Gateway throttling burst limit" + type = number + default = 50 +} + +variable "throttling_rate_limit" { + description = "API Gateway throttling rate limit" + type = number + default = 25 +} + +variable "waf_rate_limit" { + description = "WAF rate limit threshold" + type = number + default = 2000 +} + +variable "waf_rate_limit_action" { + description = "WAF rate limit action (count or block)" + type = string + default = "count" +} + +variable "log_retention_days" { + description = "CloudWatch log retention in days" + type = number + default = 7 +} + +variable "monthly_budget_amount" { + description = "Monthly budget amount (USD)" + type = number + default = 50 +} + +variable "alarm_email_endpoints" { + description = "Email endpoints for alarm notifications" + type = list(string) + default = [] +} + +variable "cloudfront_price_class" { + description = "CloudFront price class" + type = string + default = "PriceClass_100" +} + +variable "cors_allowed_origins" { + description = "Allowed CORS origins" + type = list(string) + default = ["http://localhost:3000", "http://127.0.0.1:3000"] +} + +variable "enable_cloudfront" { + description = "Whether to create CloudFront distributions. Disable on AWS accounts not yet verified for CloudFront; the rest of the stack still deploys." + type = bool + default = true +} diff --git a/infrastructure/main.tf b/infrastructure/main.tf index 0c6bf1d..f276095 100644 --- a/infrastructure/main.tf +++ b/infrastructure/main.tf @@ -43,6 +43,7 @@ module "cloudfront" { project = var.project_name environment = var.environment + enable_distributions = var.enable_cloudfront frontend_bucket_id = module.s3.frontend_bucket_id frontend_bucket_arn = module.s3.frontend_bucket_arn frontend_bucket_regional_domain_name = module.s3.frontend_bucket_regional_domain_name @@ -204,7 +205,7 @@ module "monitoring" { project_name = var.project_name environment = var.environment api_handler_function_name = module.lambda.api_handler_function_name - api_handler_log_group_name = "/aws/lambda/${module.lambda.api_handler_function_name}" + api_handler_log_group_name = module.lambda.api_handler_log_group_name documentdb_cluster_id = module.documentdb.cluster_id tags = local.common_tags } diff --git a/infrastructure/modules/cloudfront/main.tf b/infrastructure/modules/cloudfront/main.tf index 9c463f5..4b7fdd7 100644 --- a/infrastructure/modules/cloudfront/main.tf +++ b/infrastructure/modules/cloudfront/main.tf @@ -141,6 +141,7 @@ resource "aws_cloudfront_response_headers_policy" "frontend_security" { # ----------------------------------------------------------------------------- resource "aws_cloudfront_distribution" "frontend" { + count = var.enable_distributions ? 1 : 0 enabled = true is_ipv6_enabled = true comment = "${local.name_prefix} frontend distribution" @@ -270,6 +271,7 @@ resource "aws_cloudfront_cache_policy" "uploads" { # ----------------------------------------------------------------------------- resource "aws_cloudfront_distribution" "uploads" { + count = var.enable_distributions ? 1 : 0 enabled = true is_ipv6_enabled = true comment = "${local.name_prefix} uploads distribution" @@ -329,6 +331,7 @@ resource "aws_cloudfront_distribution" "uploads" { # ----------------------------------------------------------------------------- resource "aws_s3_bucket_policy" "frontend_cloudfront" { + count = var.enable_distributions ? 1 : 0 bucket = var.frontend_bucket_id policy = jsonencode({ @@ -344,7 +347,7 @@ resource "aws_s3_bucket_policy" "frontend_cloudfront" { Resource = "${var.frontend_bucket_arn}/*" Condition = { StringEquals = { - "AWS:SourceArn" = aws_cloudfront_distribution.frontend.arn + "AWS:SourceArn" = aws_cloudfront_distribution.frontend[0].arn } } } @@ -358,6 +361,7 @@ resource "aws_s3_bucket_policy" "frontend_cloudfront" { # ----------------------------------------------------------------------------- resource "aws_s3_bucket_policy" "uploads_cloudfront" { + count = var.enable_distributions ? 1 : 0 bucket = var.uploads_bucket_id policy = jsonencode({ @@ -373,7 +377,7 @@ resource "aws_s3_bucket_policy" "uploads_cloudfront" { Resource = "${var.uploads_bucket_arn}/*" Condition = { StringEquals = { - "AWS:SourceArn" = aws_cloudfront_distribution.uploads.arn + "AWS:SourceArn" = aws_cloudfront_distribution.uploads[0].arn } } } diff --git a/infrastructure/modules/cloudfront/outputs.tf b/infrastructure/modules/cloudfront/outputs.tf index 053df8c..3972470 100644 --- a/infrastructure/modules/cloudfront/outputs.tf +++ b/infrastructure/modules/cloudfront/outputs.tf @@ -1,5 +1,13 @@ # CloudFront Module - Outputs # Exports distribution identifiers for use by other modules (CI/CD, DNS, application config) +# +# NOTE: The two distributions are conditional on var.enable_distributions. When +# disabled (e.g. on an unverified AWS account that cannot create CloudFront +# resources yet), the ID/ARN/domain outputs degrade gracefully: +# - IDs/ARNs/hosted zone IDs return "" (empty string) +# - uploads_distribution_domain_name falls back to the uploads S3 bucket +# regional domain name so downstream consumers (e.g. the API Lambda's +# CDN_DOMAIN env var) still receive a valid, resolvable domain. # ============================================================================= # FRONTEND DISTRIBUTION @@ -7,22 +15,22 @@ output "frontend_distribution_id" { description = "ID of the frontend CloudFront distribution (used for cache invalidation in CI/CD)" - value = aws_cloudfront_distribution.frontend.id + value = one(aws_cloudfront_distribution.frontend[*].id) != null ? one(aws_cloudfront_distribution.frontend[*].id) : "" } output "frontend_distribution_arn" { description = "ARN of the frontend CloudFront distribution (used for S3 bucket policy and WAF association)" - value = aws_cloudfront_distribution.frontend.arn + value = one(aws_cloudfront_distribution.frontend[*].arn) != null ? one(aws_cloudfront_distribution.frontend[*].arn) : "" } output "frontend_distribution_domain_name" { - description = "Domain name of the frontend CloudFront distribution (e.g., d1234.cloudfront.net)" - value = aws_cloudfront_distribution.frontend.domain_name + description = "Domain name of the frontend CloudFront distribution (e.g., d1234.cloudfront.net). Empty when distributions are disabled." + value = one(aws_cloudfront_distribution.frontend[*].domain_name) != null ? one(aws_cloudfront_distribution.frontend[*].domain_name) : "" } output "frontend_distribution_hosted_zone_id" { description = "Route 53 hosted zone ID for the frontend distribution (for alias records)" - value = aws_cloudfront_distribution.frontend.hosted_zone_id + value = one(aws_cloudfront_distribution.frontend[*].hosted_zone_id) != null ? one(aws_cloudfront_distribution.frontend[*].hosted_zone_id) : "" } # ============================================================================= @@ -31,22 +39,22 @@ output "frontend_distribution_hosted_zone_id" { output "uploads_distribution_id" { description = "ID of the uploads CloudFront distribution (used for cache invalidation)" - value = aws_cloudfront_distribution.uploads.id + value = one(aws_cloudfront_distribution.uploads[*].id) != null ? one(aws_cloudfront_distribution.uploads[*].id) : "" } output "uploads_distribution_arn" { description = "ARN of the uploads CloudFront distribution (used for S3 bucket policy)" - value = aws_cloudfront_distribution.uploads.arn + value = one(aws_cloudfront_distribution.uploads[*].arn) != null ? one(aws_cloudfront_distribution.uploads[*].arn) : "" } output "uploads_distribution_domain_name" { - description = "Domain name of the uploads CloudFront distribution (e.g., d5678.cloudfront.net)" - value = aws_cloudfront_distribution.uploads.domain_name + description = "Domain name of the uploads CloudFront distribution. Falls back to the uploads S3 bucket regional domain name when distributions are disabled." + value = one(aws_cloudfront_distribution.uploads[*].domain_name) != null ? one(aws_cloudfront_distribution.uploads[*].domain_name) : var.uploads_bucket_regional_domain_name } output "uploads_distribution_hosted_zone_id" { description = "Route 53 hosted zone ID for the uploads distribution (for alias records)" - value = aws_cloudfront_distribution.uploads.hosted_zone_id + value = one(aws_cloudfront_distribution.uploads[*].hosted_zone_id) != null ? one(aws_cloudfront_distribution.uploads[*].hosted_zone_id) : "" } # ============================================================================= diff --git a/infrastructure/modules/cloudfront/variables.tf b/infrastructure/modules/cloudfront/variables.tf index 5c6b5cf..2589628 100644 --- a/infrastructure/modules/cloudfront/variables.tf +++ b/infrastructure/modules/cloudfront/variables.tf @@ -16,6 +16,21 @@ variable "environment" { } } +variable "enable_distributions" { + description = <<-EOT + Whether to create the two CloudFront distributions (frontend + uploads) and + their S3 bucket policies. New/unverified AWS accounts return + "AccessDenied: Your account must be verified before you can add new + CloudFront resources" from the CloudFront API until AWS Support verifies the + account. Set this to false to deploy the rest of the stack (API, Lambda, + DocumentDB, etc.) without CloudFront; when disabled, cdn_domain falls back to + the uploads S3 bucket regional domain name. Re-enable and re-apply once the + account is verified. Defaults to true. + EOT + type = bool + default = true +} + # ----------------------------------------------------------------------------- # Frontend Distribution Variables # ----------------------------------------------------------------------------- diff --git a/infrastructure/modules/disaster-recovery/dns-failover.tf b/infrastructure/modules/disaster-recovery/dns-failover.tf index b039961..98b6dcc 100644 --- a/infrastructure/modules/disaster-recovery/dns-failover.tf +++ b/infrastructure/modules/disaster-recovery/dns-failover.tf @@ -130,7 +130,10 @@ resource "aws_s3_object" "maintenance_page" { # ─── DNS Failover Records ───────────────────────────────────────────────────── # Primary record — points to API Gateway +# Only created when a hosted zone is provided (skipped when hosted_zone_id == "") resource "aws_route53_record" "api_primary" { + count = var.hosted_zone_id != "" ? 1 : 0 + zone_id = var.hosted_zone_id name = var.domain_name type = "A" @@ -150,7 +153,10 @@ resource "aws_route53_record" "api_primary" { } # Secondary record — points to maintenance page +# Only created when a hosted zone is provided (skipped when hosted_zone_id == "") resource "aws_route53_record" "api_secondary" { + count = var.hosted_zone_id != "" ? 1 : 0 + zone_id = var.hosted_zone_id name = var.domain_name type = "A" diff --git a/infrastructure/modules/disaster-recovery/main.tf b/infrastructure/modules/disaster-recovery/main.tf index 538c2d4..c1e1b7b 100644 --- a/infrastructure/modules/disaster-recovery/main.tf +++ b/infrastructure/modules/disaster-recovery/main.tf @@ -179,12 +179,27 @@ resource "aws_s3_bucket_replication_configuration" "uploads" { bucket = var.uploads_bucket_id role = aws_iam_role.replication.arn + # S3 requires versioning to be ENABLED on the destination (replica) bucket + # before a replication configuration referencing it can be created. Without + # this explicit dependency Terraform may create the replication config before + # the versioning resource is applied, yielding: + # "Destination bucket must have versioning enabled." + depends_on = [aws_s3_bucket_versioning.uploads_replica] + rule { id = "replicate-all" status = "Enabled" filter {} + # When a `filter` is present, S3 uses the V2 replication schema which + # REQUIRES delete_marker_replication to be specified explicitly, otherwise: + # "DeleteMarkerReplication must be specified for this version of Cross + # Region Replication configuration schema." + delete_marker_replication { + status = "Enabled" + } + destination { bucket = aws_s3_bucket.uploads_replica.arn storage_class = "STANDARD_IA" diff --git a/infrastructure/modules/lambda/outputs.tf b/infrastructure/modules/lambda/outputs.tf index bd7774e..b033b86 100644 --- a/infrastructure/modules/lambda/outputs.tf +++ b/infrastructure/modules/lambda/outputs.tf @@ -24,6 +24,11 @@ output "api_handler_qualified_arn" { value = aws_lambda_function.api_handler.qualified_arn } +output "api_handler_log_group_name" { + description = "Name of the API handler Lambda CloudWatch log group. Sourced from the log group resource so consumers (e.g. the monitoring module's metric filters) build a proper dependency edge and are not created before the log group exists." + value = aws_cloudwatch_log_group.api_handler.name +} + # ─── Achievement Processor ──────────────────────────────────────────────────── output "achievement_processor_arn" { diff --git a/infrastructure/modules/waf/main.tf b/infrastructure/modules/waf/main.tf index 867bc3a..e1caa2e 100644 --- a/infrastructure/modules/waf/main.tf +++ b/infrastructure/modules/waf/main.tf @@ -137,7 +137,11 @@ resource "aws_wafv2_web_acl" "api" { # ─── WAF Association with API Gateway ───────────────────────────────────────── +# NOTE: WAFv2 regional WebACLs cannot be associated with API Gateway v2 (HTTP API) +# stages. Taskly's API Gateway is an HTTP API, so this association is disabled by +# default (see var.enable_api_gateway_association). The WebACL itself is still created. resource "aws_wafv2_web_acl_association" "api_gateway" { + count = var.enable_api_gateway_association ? 1 : 0 resource_arn = var.api_gateway_stage_arn web_acl_arn = aws_wafv2_web_acl.api.arn } diff --git a/infrastructure/modules/waf/variables.tf b/infrastructure/modules/waf/variables.tf index 87c2566..886dd22 100644 --- a/infrastructure/modules/waf/variables.tf +++ b/infrastructure/modules/waf/variables.tf @@ -25,6 +25,20 @@ variable "api_gateway_stage_arn" { type = string } +variable "enable_api_gateway_association" { + description = <<-EOT + Whether to associate the WAF WebACL with the API Gateway stage. + WAFv2 regional WebACLs can only be associated with API Gateway v1 (REST) stages, + Application Load Balancers, AppSync, Cognito user pools, and App Runner services. + They CANNOT be associated with API Gateway v2 (HTTP API) stages. Taskly uses an + HTTP API (aws_apigatewayv2_api, protocol_type = "HTTP"), so this defaults to false. + The WebACL is still created and can be attached to a compatible resource, or the + API can be fronted by CloudFront (which supports the CLOUDFRONT-scope WebACL). + EOT + type = bool + default = false +} + variable "rate_limit" { description = "Maximum requests per IP per 5-minute window before rate limiting" type = number diff --git a/infrastructure/variables.tf b/infrastructure/variables.tf index 640a683..87c2cf0 100644 --- a/infrastructure/variables.tf +++ b/infrastructure/variables.tf @@ -40,6 +40,21 @@ variable "vpc_cidr" { default = "10.0.0.0/16" } +variable "enable_cloudfront" { + description = <<-EOT + Whether to create the CloudFront distributions (frontend + uploads). Set to + false on AWS accounts that have not been verified for CloudFront (the + CloudFront API returns "AccessDenied: Your account must be verified before + you can add new CloudFront resources" until AWS Support verifies the + account). When false, the rest of the stack (API Gateway, Lambda, DocumentDB, + etc.) still deploys, and the Lambda CDN_DOMAIN falls back to the uploads S3 + bucket regional domain name. Re-enable and re-apply once the account is + verified. Defaults to true. + EOT + type = bool + default = true +} + # ─── Database ───────────────────────────────────────────────────────────────── variable "documentdb_master_password" { diff --git a/scripts/build-lambda.sh b/scripts/build-lambda.sh new file mode 100755 index 0000000..697b47e --- /dev/null +++ b/scripts/build-lambda.sh @@ -0,0 +1,143 @@ +#!/usr/bin/env bash +# +# build-lambda.sh — Build the four Lambda deployment bundles from backend/. +# +# Taskly runs four Lambda functions from a single backend codebase: +# - api-handler (handler = index.handler; index.mjs re-exports lambda/handler.js) +# - achievement-processor (handler = lambda/processors/achievement-processor.handler) +# - notification-processor (handler = lambda/processors/notification-processor.handler) +# - email-processor (handler = lambda/processors/email-processor.handler) +# +# All four handlers import shared backend code (../server.js, ../utils/secrets.js, etc.) +# and depend on node_modules (@vendia/serverless-express, mongoose, aws-sdk, ...). +# The simplest correct approach — and the one used here — is to make ALL FOUR zips +# an IDENTICAL bundle containing the full backend contents. Every handler path then +# resolves regardless of which zip a given Lambda loads. Files are placed at the +# ARCHIVE ROOT (we zip from INSIDE backend/, never from the parent directory) so that +# handler paths like "index.handler" and "lambda/processors/email-processor.handler" +# resolve correctly. +# +# The script is idempotent: it cleans/recreates the output dir each run and rebuilds +# production dependencies. Output directory defaults to backend/build and can be +# overridden with the first positional arg or the BUILD_DIR env var. +# +# Usage: +# bash scripts/build-lambda.sh [output_dir] +# +set -euo pipefail + +# --- Resolve paths ----------------------------------------------------------- +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" +BACKEND_DIR="${REPO_ROOT}/backend" +BUILD_DIR="${1:-${BUILD_DIR:-${BACKEND_DIR}/build}}" + +PEM_URL="https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem" +PEM_PATH="${BACKEND_DIR}/global-bundle.pem" + +echo "==> Repo root: ${REPO_ROOT}" +echo "==> Backend dir: ${BACKEND_DIR}" +echo "==> Build dir: ${BUILD_DIR}" + +# --- Activate Node 20 via nvm (best effort) ---------------------------------- +if [ -s "${HOME}/.nvm/nvm.sh" ]; then + # shellcheck disable=SC1091 + source "${HOME}/.nvm/nvm.sh" + nvm use 20 >/dev/null 2>&1 || echo "WARN: 'nvm use 20' failed; using current node ($(node -v 2>/dev/null || echo 'none'))" +else + echo "WARN: ~/.nvm/nvm.sh not found; using current node ($(node -v 2>/dev/null || echo 'none'))" +fi + +# --- Install production dependencies only ------------------------------------ +cd "${BACKEND_DIR}" +echo "==> Installing production dependencies (npm install --omit=dev)" +npm install --omit=dev + +# --- Prune bundle bloat to stay under Lambda's 250MB unzipped limit ---------- +# The full production node_modules is ~310MB unzipped, which exceeds Lambda's +# hard limit of 262144000 bytes (250 MiB). We prune packages that are either +# provided by the Node.js 20 Lambda runtime or unused by the four deployed +# handlers: +# - @aws-sdk / @smithy : AWS SDK for JavaScript v3 is bundled in the nodejs20.x +# runtime (client-s3, client-ses, client-sqs, +# client-eventbridge, client-secrets-manager, +# s3-request-presigner, ...). Safe to exclude from the zip. +# - @img / sharp : native image libs imported ONLY by +# lambda/processors/image-processor.js, which is NOT one +# of the four deployed functions. Not loaded by the +# api/achievement/notification/email handlers. +# - core-js : polyfill with no direct import in backend source. +# This reduces the unzipped bundle to ~238MB. +PRUNE_MODULES=( + "@aws-sdk" + "@smithy" + "@img" + "sharp" + "core-js" +) +echo "==> Pruning runtime-provided / unused modules to fit the 250MB limit" +for m in "${PRUNE_MODULES[@]}"; do + if [ -e "${BACKEND_DIR}/node_modules/${m}" ]; then + echo " - removing node_modules/${m}" + rm -rf "${BACKEND_DIR}/node_modules/${m:?}" + fi +done + +# --- Download the DocumentDB TLS certificate bundle -------------------------- +echo "==> Downloading DocumentDB global TLS bundle -> ${PEM_PATH}" +curl -fsSL "${PEM_URL}" -o "${PEM_PATH}" + +# --- Prepare a clean output directory ---------------------------------------- +rm -rf "${BUILD_DIR}" +mkdir -p "${BUILD_DIR}" + +# --- Build the canonical bundle ---------------------------------------------- +# Files/dirs included at the ARCHIVE ROOT. We zip from inside backend/ so paths +# like "index.mjs" and "lambda/handler.js" sit at the top of the archive. +API_ZIP="${BUILD_DIR}/api-handler.zip" + +ROOT_FILES=( + index.mjs + server.js + package.json + schemas.js + global-bundle.pem +) + +ROOT_DIRS=( + lambda + config + controllers + middleware + models + routes + services + utils + node_modules +) + +# Collect only the entries that actually exist so the script stays robust. +INCLUDE=() +for f in "${ROOT_FILES[@]}"; do + [ -e "${BACKEND_DIR}/${f}" ] && INCLUDE+=("${f}") || echo "WARN: missing file ${f}, skipping" +done +for d in "${ROOT_DIRS[@]}"; do + [ -d "${BACKEND_DIR}/${d}" ] && INCLUDE+=("${d}") || echo "WARN: missing dir ${d}, skipping" +done + +echo "==> Creating ${API_ZIP} (full backend bundle at archive root)" +( cd "${BACKEND_DIR}" && zip -r -q "${API_ZIP}" "${INCLUDE[@]}" ) + +# --- The processor bundles are identical copies of the api bundle ------------ +for name in achievement-processor notification-processor email-processor; do + echo "==> Creating ${BUILD_DIR}/${name}.zip (copy of api bundle)" + cp "${API_ZIP}" "${BUILD_DIR}/${name}.zip" +done + +# --- Summary ----------------------------------------------------------------- +echo "" +echo "==> Build complete. Artifacts:" +ls -lh "${BUILD_DIR}"/*.zip +echo "" +echo "==> api-handler.zip key entries:" +unzip -l "${API_ZIP}" | grep -E 'index.mjs|lambda/handler.js|lambda/processors|node_modules/@vendia/serverless-express' | head -n 10 || true