From 4951709c50aa19b6ef3c0e95646c2ea44946f670 Mon Sep 17 00:00:00 2001 From: Tom Scanlan Date: Fri, 16 Oct 2026 21:05:26 -0400 Subject: [PATCH 1/8] DEVNET_DOMAIN: the domain every name sits under, devnet.test by default The PDS names, handle domains, service DIDs, relay hosts, nginx server names, the CA's name constraints and the scripts' defaults all come from one setting, read from the environment or .env. Unset, the stack renders as before: every service's config is unchanged except nginx, which now fills its config in from a template at start (the output matches the old file), and init, which gets the domain for its seed emails. plc.directory and .devnet.internal stay as they are. https-ca.sh issues the leaf beside the current one and replaces it only once it checks out, so a CA made for another domain no longer leaves nginx with a leaf it cannot vouch for. data/devnet.env carries DEVNET_DOMAIN for apps. --- .env.example | 3 ++ README.md | 19 ++++++++--- docker-compose.https.yml | 28 +++++++++------- docker-compose.multi-pds.yml | 12 +++---- docker-compose.yml | 13 ++++---- https/app-names.cjs | 6 ++-- https/{nginx.conf => nginx.conf.template} | 10 +++--- scripts/create-account.sh | 2 +- scripts/https-ca.sh | 40 +++++++++++++++-------- scripts/https-lib.sh | 19 ++++++----- scripts/https-probe.mjs | 21 +++++++----- scripts/https-run | 6 ++-- scripts/https-up.sh | 1 + scripts/init.sh | 4 +-- scripts/lexicon-authority.sh | 4 +-- 15 files changed, 116 insertions(+), 72 deletions(-) rename https/{nginx.conf => nginx.conf.template} (87%) diff --git a/.env.example b/.env.example index f5da713..da0c2c3 100644 --- a/.env.example +++ b/.env.example @@ -11,6 +11,9 @@ DEVNET_DB_NAME=plc DEVNET_SMTP_URL=smtp://maildev:1025 # Optional: Override defaults +# The domain every name sits under: the PDS (alpha. on the https devnet) and +# the handles (.). The two below default to it. +# DEVNET_DOMAIN=devnet.test # DEVNET_PDS_HOSTNAME=devnet.test # DEVNET_PDS_ADMIN_PASSWORD=devnet-admin-password # DEVNET_HANDLE_DOMAIN=.devnet.test diff --git a/README.md b/README.md index fa43a6a..a3cb385 100644 --- a/README.md +++ b/README.md @@ -598,6 +598,13 @@ the three PDSes as `wss://` hosts. On the machine, the names resolve to `127.0.0 commands run under `scripts/https-run`, so every other program still reaches the real `plc.directory`. +`devnet.test` is the default of one setting, `DEVNET_DOMAIN`, and every name in the table above +(and in the rest of this README) uses it. Set it in the environment or in `.env` and the same +stack comes up under that domain: `alpha.` with handles at `.`, +`regular.`, `prod.`. `plc.directory` and `.devnet.internal` stay as they are. +A checkout keeps one domain, because its CA can vouch only for the domain it was made for +([The CA](#the-ca)); for a second domain, use a second checkout. + The devnet knows no app. It has no app's accounts, invite codes, lexicons or ports. It gives an app tools to make its own (below), and using them is optional. @@ -646,7 +653,8 @@ a new DID. - `data/devnet.env`: the devnet's own facts, nothing secret. `ALPHA_PDS_URL`, `REGULAR_PDS_URL`, `PROD_PDS_URL`, `PLC_URL` (`https://plc.directory`), `JETSTREAM_URL` (`ws://127.0.0.1:`), - `MAILDEV_URL`, `LEX_AUTHORITY_DID`, `LEX_AUTHORITY_HANDLE`, `DEVNET_PROJECT` and `DEVNET_CA_FILE`. + `MAILDEV_URL`, `LEX_AUTHORITY_DID`, `LEX_AUTHORITY_HANDLE`, `DEVNET_PROJECT`, `DEVNET_DOMAIN` and + `DEVNET_CA_FILE`. An app can source it: `set -a; . /data/devnet.env; set +a`. - `data/accounts.env` (mode 600): logins. `DEVNET_INVITE_CODE`, `ALICE_*` and `BOB_*` from init, `LEX_AUTHORITY_*`, and whatever the tools below add. @@ -789,8 +797,8 @@ deeper names with `HTTPS_EXTRA_NAMES="x.mine.devnet.test"`, then restart nginx. under `devnet.test` is already covered. To start over, delete `data/https/ca.*` and trust the new `ca.crt` wherever the old one was trusted. -The CA can only vouch for servers under `devnet.test`, `devnet.internal` and `plc.directory` (name -constraints, server auth only, no IP addresses), so trusting it puts no other site at risk, even if +The CA can only vouch for servers under `devnet.test` (the `DEVNET_DOMAIN` it was made for), +`devnet.internal` and `plc.directory` (name constraints, server auth only, no IP addresses), so trusting it puts no other site at risk, even if `data/https/ca.key` leaks. Keep that file on the dev machine, and remove the CA from a browser or OS when you're done with it. @@ -867,12 +875,13 @@ All settings have sensible defaults. Override via `.env` or environment variable | Variable | Default | Description | |----------|---------|-------------| -| `DEVNET_PDS_HOSTNAME` | `devnet.test` | PDS service hostname | +| `DEVNET_DOMAIN` | `devnet.test` | The domain every name sits under ([The https devnet](#the-https-devnet)) | +| `DEVNET_PDS_HOSTNAME` | `DEVNET_DOMAIN` | PDS service hostname (the https devnet names its PDSes itself) | | `DEVNET_PDS_IMAGE` | `ghcr.io/bluesky-social/pds:0.4` | PDS image (see [Choosing a PDS version](#choosing-a-pds-version)) | | `DEVNET_PDS_ADMIN_PASSWORD` | `devnet-admin-password` | PDS admin password | | `DEVNET_APPVIEW_URL` / `DEVNET_APPVIEW_DID` | `https://appview.invalid` / `did:example:invalid` | Where the PDS forwards app.bsky.* reads and methods it doesn't implement. Unresolvable, so nothing reaches Bluesky | | `DEVNET_REPORT_SERVICE_URL` / `DEVNET_REPORT_SERVICE_DID` | `https://moderator.invalid` / `did:example:invalid` | Where reports go | -| `DEVNET_HANDLE_DOMAIN` | `.devnet.test` | Handle suffix for accounts | +| `DEVNET_HANDLE_DOMAIN` | `.` + `DEVNET_DOMAIN` | Handle suffix for accounts | | `DEVNET_SEED_ACCOUNTS` | `true` | Create alice/bob on startup | | `DEVNET_PDS_REGULAR_IMAGE` / `DEVNET_PDS_PROD_IMAGE` | `ghcr.io/bluesky-social/pds:0.4.5037` / a pinned production digest | The two extra PDS builds ([Several PDS builds side by side](#several-pds-builds-side-by-side)) | | `DEVNET_LEXICON_AUTHORITY_DID` | empty | The spaces PDS's lexicon authority ([Spaces PDS and unpublished lexicons](#spaces-pds-and-unpublished-lexicons)) | diff --git a/docker-compose.https.yml b/docker-compose.https.yml index 666dc8c..20f77cb 100644 --- a/docker-compose.https.yml +++ b/docker-compose.https.yml @@ -10,7 +10,8 @@ # certificates, the lexicon authority's second pass and data/devnet.env. Use it. # # Names, all answered by nginx on 127.0.0.1:443 with one leaf from data/https/ -# (scripts/https-ca.sh), signed by a local CA that can vouch for nothing else: +# (scripts/https-ca.sh), signed by a local CA that can vouch for nothing else. +# devnet.test is DEVNET_DOMAIN's default; set it and every .devnet.test name follows: # # alpha.devnet.test *.devnet.test pds the spaces alpha, invites on # regular.devnet.test *.regular.devnet.test pds-regular a current release @@ -48,10 +49,10 @@ services: pds: environment: <<: *https-pds-env - PDS_HOSTNAME: alpha.devnet.test + PDS_HOSTNAME: alpha.${DEVNET_DOMAIN:-devnet.test} # Explicit: the default would be .alpha.devnet.test. - PDS_SERVICE_HANDLE_DOMAINS: .devnet.test - PDS_HANDLE_DOMAINS: .devnet.test + PDS_SERVICE_HANDLE_DOMAINS: .${DEVNET_DOMAIN:-devnet.test} + PDS_HANDLE_DOMAINS: .${DEVNET_DOMAIN:-devnet.test} # The public URL is https, so the alpha runs as it would on the real network. # ("false", not !reset: compose keeps a key an earlier overlay set.) PDS_DEV_MODE: "false" @@ -63,7 +64,7 @@ services: pds-regular: environment: <<: *https-pds-env - PDS_HOSTNAME: regular.devnet.test + PDS_HOSTNAME: regular.${DEVNET_DOMAIN:-devnet.test} volumes: - *devnet-ca - *app-names @@ -71,7 +72,7 @@ services: pds-prod: environment: <<: *https-pds-env - PDS_HOSTNAME: prod.devnet.test + PDS_HOSTNAME: prod.${DEVNET_DOMAIN:-devnet.test} volumes: - *devnet-ca - *app-names @@ -94,7 +95,7 @@ services: condition: service_healthy environment: # Bare names are https/wss hosts to the relay. - RELAY_HOSTS: alpha.devnet.test regular.devnet.test prod.devnet.test + RELAY_HOSTS: alpha.${DEVNET_DOMAIN:-devnet.test} regular.${DEVNET_DOMAIN:-devnet.test} prod.${DEVNET_DOMAIN:-devnet.test} # Resync fetches repos through an SSRF-safe transport, which refuses nginx's address, # so TAP follows the relay's stream but cannot resync a repo here. @@ -135,8 +136,13 @@ services: extra_hosts: # This machine, where an app registered with scripts/https-app.sh listens. - host.docker.internal:host-gateway + environment: + # The image fills ${DEVNET_DOMAIN} into the template at start, and only names + # starting DEVNET_, so nginx's own $host and the like are left alone. + DEVNET_DOMAIN: ${DEVNET_DOMAIN:-devnet.test} + NGINX_ENVSUBST_FILTER: ^DEVNET_ volumes: - - ./https/nginx.conf:/etc/nginx/conf.d/default.conf:ro + - ./https/nginx.conf.template:/etc/nginx/templates/default.conf.template:ro - ./data/https/leaf.crt:/etc/nginx/https/leaf.crt:ro - ./data/https/leaf.key:/etc/nginx/https/leaf.key:ro # The app routes scripts/https-app.sh writes (a directory, so a rewrite shows up). @@ -144,9 +150,9 @@ services: networks: atproto-devnet: aliases: - - alpha.devnet.test - - regular.devnet.test - - prod.devnet.test + - alpha.${DEVNET_DOMAIN:-devnet.test} + - regular.${DEVNET_DOMAIN:-devnet.test} + - prod.${DEVNET_DOMAIN:-devnet.test} healthcheck: test: ["CMD", "nginx", "-t", "-q"] interval: 5s diff --git a/docker-compose.multi-pds.yml b/docker-compose.multi-pds.yml index 742262f..297f2b0 100644 --- a/docker-compose.multi-pds.yml +++ b/docker-compose.multi-pds.yml @@ -42,7 +42,7 @@ x-extra-pds-env: &extra-pds-env PDS_JWT_SECRET: devnet-jwt-secret-not-for-production PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef" PDS_EMAIL_SMTP_URL: ${DEVNET_SMTP_URL:-smtp://maildev:1025} - PDS_EMAIL_FROM_ADDRESS: noreply@devnet.test + PDS_EMAIL_FROM_ADDRESS: noreply@${DEVNET_DOMAIN:-devnet.test} PDS_LEXICON_AUTHORITY_DID: ${DEVNET_LEXICON_AUTHORITY_DID:-} LOG_ENABLED: "true" NODE_ENV: development @@ -51,7 +51,7 @@ services: pds: environment: # All three would otherwise be did:web:localhost. - PDS_SERVICE_DID: did:web:alpha.devnet.test + PDS_SERVICE_DID: did:web:alpha.${DEVNET_DOMAIN:-devnet.test} ports: !override - "${DEVNET_SPACES_PDS_PORT:-3010}:${DEVNET_SPACES_PDS_PORT:-3010}" - "3020:3020" @@ -64,8 +64,8 @@ services: environment: <<: *extra-pds-env PDS_PORT: "3020" - PDS_SERVICE_DID: did:web:regular.devnet.test - PDS_SERVICE_HANDLE_DOMAINS: .regular.devnet.test + PDS_SERVICE_DID: did:web:regular.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_HANDLE_DOMAINS: .regular.${DEVNET_DOMAIN:-devnet.test} volumes: - pds-regular-data:/pds/data healthcheck: @@ -81,8 +81,8 @@ services: environment: <<: *extra-pds-env PDS_PORT: "3030" - PDS_SERVICE_DID: did:web:prod.devnet.test - PDS_SERVICE_HANDLE_DOMAINS: .prod.devnet.test + PDS_SERVICE_DID: did:web:prod.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_HANDLE_DOMAINS: .prod.${DEVNET_DOMAIN:-devnet.test} volumes: - pds-prod-data:/pds/data healthcheck: diff --git a/docker-compose.yml b/docker-compose.yml index a9cb1d2..b44d493 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -58,7 +58,7 @@ services: condition: service_healthy environment: PDS_PORT: 3000 - PDS_HOSTNAME: ${DEVNET_PDS_HOSTNAME:-devnet.test} + PDS_HOSTNAME: ${DEVNET_PDS_HOSTNAME:-${DEVNET_DOMAIN:-devnet.test}} PDS_DATA_DIRECTORY: /pds/data PDS_BLOBSTORE_DISK_LOCATION: /pds/data/blocks PDS_DID_PLC_URL: http://plc:2582 @@ -81,9 +81,9 @@ services: PDS_JWT_SECRET: devnet-jwt-secret-not-for-production PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef" PDS_EMAIL_SMTP_URL: ${DEVNET_SMTP_URL:-smtp://maildev:1025} - PDS_EMAIL_FROM_ADDRESS: noreply@devnet.test - PDS_HANDLE_DOMAINS: ${DEVNET_HANDLE_DOMAIN:-.devnet.test} - PDS_SERVICE_HANDLE_DOMAINS: ${DEVNET_HANDLE_DOMAIN:-.devnet.test} + PDS_EMAIL_FROM_ADDRESS: noreply@${DEVNET_DOMAIN:-devnet.test} + PDS_HANDLE_DOMAINS: ${DEVNET_HANDLE_DOMAIN:-.${DEVNET_DOMAIN:-devnet.test}} + PDS_SERVICE_HANDLE_DOMAINS: ${DEVNET_HANDLE_DOMAIN:-.${DEVNET_DOMAIN:-devnet.test}} LOG_ENABLED: "true" NODE_ENV: development volumes: @@ -159,9 +159,10 @@ services: - ./scripts:/scripts:ro - ./data:/devnet-data environment: - PDS_HOSTNAME: ${DEVNET_PDS_HOSTNAME:-devnet.test} + PDS_HOSTNAME: ${DEVNET_PDS_HOSTNAME:-${DEVNET_DOMAIN:-devnet.test}} PDS_ADMIN_PASSWORD: ${DEVNET_PDS_ADMIN_PASSWORD:-devnet-admin-password} - HANDLE_DOMAIN: ${DEVNET_HANDLE_DOMAIN:-.devnet.test} + HANDLE_DOMAIN: ${DEVNET_HANDLE_DOMAIN:-.${DEVNET_DOMAIN:-devnet.test}} + DEVNET_DOMAIN: ${DEVNET_DOMAIN:-devnet.test} SEED_ACCOUNTS: ${DEVNET_SEED_ACCOUNTS:-true} entrypoint: /bin/sh command: ["/scripts/init.sh"] diff --git a/https/app-names.cjs b/https/app-names.cjs index e0ea888..091b93c 100644 --- a/https/app-names.cjs +++ b/https/app-names.cjs @@ -1,9 +1,9 @@ // Preloaded into each PDS (NODE_OPTIONS=--require, docker-compose.https.yml), so a PDS can // reach an app at https://.devnet.internal: it fetches a confidential client's // metadata and keys from there. Docker's DNS has no wildcard names, so every such name is -// looked up as nginx instead, which routes it by name (https/nginx.conf). Only the address -// lookup changes: the request still carries the app's name, and TLS still checks the -// certificate for it against the devnet CA. Every other name resolves as usual. +// looked up as nginx instead, which routes it by name (https/nginx.conf.template). Only +// the address lookup changes: the request still carries the app's name, and TLS still +// checks the certificate for it against the devnet CA. Every other name resolves as usual. 'use strict'; const dns = require('dns'); diff --git a/https/nginx.conf b/https/nginx.conf.template similarity index 87% rename from https/nginx.conf rename to https/nginx.conf.template index 15eb1e0..bf8540e 100644 --- a/https/nginx.conf +++ b/https/nginx.conf.template @@ -1,6 +1,8 @@ # nginx for the https devnet (docker-compose.https.yml). One leaf certificate names # every host below; data/https/ca.crt is the CA that signed it (scripts/https-ca.sh). -# Mounted as conf.d/default.conf, so this is the http block's whole server list. +# A template: at start the image writes it to conf.d/default.conf with the DEVNET_DOMAIN +# setting filled in, so this is the http block's whole server list. The comments below +# name its default, devnet.test. # Docker's embedded DNS, re-asked every 10 s, so a recreated container's new # address is picked up without reloading nginx. @@ -40,7 +42,7 @@ server { # *.devnet.test. server { listen 443 ssl; - server_name alpha.devnet.test *.devnet.test; + server_name alpha.${DEVNET_DOMAIN} *.${DEVNET_DOMAIN}; location / { set $pds http://pds:3010; proxy_pass $pds; @@ -49,7 +51,7 @@ server { server { listen 443 ssl; - server_name regular.devnet.test *.regular.devnet.test; + server_name regular.${DEVNET_DOMAIN} *.regular.${DEVNET_DOMAIN}; location / { set $pds http://pds:3020; proxy_pass $pds; @@ -58,7 +60,7 @@ server { server { listen 443 ssl; - server_name prod.devnet.test *.prod.devnet.test; + server_name prod.${DEVNET_DOMAIN} *.prod.${DEVNET_DOMAIN}; location / { set $pds http://pds:3030; proxy_pass $pds; diff --git a/scripts/create-account.sh b/scripts/create-account.sh index 117eac9..de11c03 100755 --- a/scripts/create-account.sh +++ b/scripts/create-account.sh @@ -24,7 +24,7 @@ PDS_URL="${DEVNET_PDS_URL:-http://localhost:${DEVNET_PDS_PORT:-3000}}" PDS_ADMIN_PASSWORD="${DEVNET_PDS_ADMIN_PASSWORD:-devnet-admin-password}" HANDLE="${1:?Usage: create-account.sh [email] [password]}" -EMAIL="${2:-${HANDLE}@devnet.test}" +EMAIL="${2:-${HANDLE}@${DEVNET_DOMAIN:-devnet.test}}" PASSWORD="${3:-$(head -c 16 /dev/urandom | base64 | tr -dc 'a-zA-Z0-9' | head -c 16)}" # Get or create invite code diff --git a/scripts/https-ca.sh b/scripts/https-ca.sh index 05bb2b7..87697c3 100755 --- a/scripts/https-ca.sh +++ b/scripts/https-ca.sh @@ -3,7 +3,8 @@ set -eu # The https devnet's certificates: one local CA, made once, and one leaf that nginx # serves for every devnet name: the three PDSes and their handles, plc.directory, and -# every app at .devnet.internal. +# every app at .devnet.internal. The PDS names sit under DEVNET_DOMAIN +# (devnet.test by default; scripts/https-lib.sh), and the names below use the default. # # Usage: ./scripts/https-ca.sh # @@ -12,11 +13,11 @@ set -eu # devnet; only the leaf is reissued. Delete data/https/ca.* to start over, and then # trust the new ca.crt again everywhere the old one was trusted. # -# The CA can only vouch for servers under devnet.test, devnet.internal and -# plc.directory (name constraints, serverAuth only, no IP addresses), so trusting it -# on a host cannot put any other site at risk, even if ca.key leaks from the pod. -# A name outside those makes the leaf fail its own check below; widening the list -# means a new CA, trusted again. +# The CA can only vouch for servers under the domain it was made for, devnet.internal +# and plc.directory (name constraints, serverAuth only, no IP addresses), so trusting +# it on a host cannot put any other site at risk, even if ca.key leaks from the pod. +# A name outside those makes the leaf fail its own check below; widening the list, or +# a new DEVNET_DOMAIN, means a new CA, trusted again. # # A TLS wildcard covers one label, so *.devnet.test does not cover a handle on the # regular PDS (.regular.devnet.test); each PDS's handle domain has its own. @@ -25,11 +26,13 @@ set -eu # (space-separated), for instance when a PDS is added. SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +. "${SCRIPT_DIR}/https-lib.sh" +D="${DEVNET_DOMAIN}" OUT="${SCRIPT_DIR}/../data/https" mkdir -p "${OUT}" cd "${OUT}" -NAMES="alpha.devnet.test regular.devnet.test prod.devnet.test *.devnet.test *.regular.devnet.test *.prod.devnet.test plc.directory *.devnet.internal ${HTTPS_EXTRA_NAMES:-}" +NAMES="alpha.${D} regular.${D} prod.${D} *.${D} *.regular.${D} *.prod.${D} plc.directory *.devnet.internal ${HTTPS_EXTRA_NAMES:-}" umask 077 @@ -47,7 +50,7 @@ else -addext "basicConstraints=critical,CA:TRUE,pathlen:0" \ -addext "keyUsage=critical,keyCertSign,cRLSign" \ -addext "extendedKeyUsage=serverAuth" \ - -addext "nameConstraints=critical,permitted;DNS:devnet.test,permitted;DNS:devnet.internal,permitted;DNS:plc.directory,excluded;IP:0.0.0.0/0.0.0.0,excluded;IP:0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0" + -addext "nameConstraints=critical,permitted;DNS:${D},permitted;DNS:devnet.internal,permitted;DNS:plc.directory,excluded;IP:0.0.0.0/0.0.0.0,excluded;IP:0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0" chmod 644 ca.crt echo "Made a new CA: $(openssl x509 -in ca.crt -noout -subject)" >&2 fi @@ -68,19 +71,30 @@ authorityKeyIdentifier=keyid,issuer subjectAltName=${SAN} EOF +# Issued beside the current leaf, which is replaced only once the new one checks out. openssl req -new -nodes -sha256 \ -newkey ec -pkeyopt ec_paramgen_curve:P-256 \ - -keyout leaf.key -out leaf.csr \ - -subj "/O=atproto-devnet/CN=alpha.devnet.test" + -keyout leaf.key.new -out leaf.csr \ + -subj "/O=atproto-devnet/CN=alpha.${D}" # 397 days: under the 398-day ceiling browsers apply to server certificates. openssl x509 -req -sha256 -days 397 \ -in leaf.csr -CA ca.crt -CAkey ca.key -CAcreateserial \ - -extfile leaf.ext -out leaf.crt + -extfile leaf.ext -out leaf.crt.new rm -f leaf.csr leaf.ext + +if ! openssl verify -CAfile ca.crt leaf.crt.new >&2; then + rm -f leaf.crt.new leaf.key.new + echo "The CA cannot vouch for every leaf name, so the leaf is unchanged. If" >&2 + echo "DEVNET_DOMAIN (${D}) changed since the CA was made, use a separate checkout, or" >&2 + echo "delete data/https/ca.* to make a new CA for it (and trust that one again)." >&2 + exit 1 +fi +# Written in place, not renamed: nginx's single-file mounts follow the file, not the name. +cat leaf.key.new > leaf.key +cat leaf.crt.new > leaf.crt +rm -f leaf.crt.new leaf.key.new chmod 644 leaf.crt # nginx reads the leaf key as root inside its container. chmod 600 leaf.key ca.key - -openssl verify -CAfile ca.crt leaf.crt >&2 echo "Leaf names: $(openssl x509 -in leaf.crt -noout -ext subjectAltName | tail -n +2 | sed 's/^ *//')" >&2 echo "Leaf valid until $(openssl x509 -in leaf.crt -noout -enddate | cut -d= -f2)" >&2 diff --git a/scripts/https-lib.sh b/scripts/https-lib.sh index d28ae3f..d9b2aba 100644 --- a/scripts/https-lib.sh +++ b/scripts/https-lib.sh @@ -10,20 +10,18 @@ CA="${HTTPS_DIR}/ca.crt" # authority, and accounts made by https-account.sh). DEVNET_ACCOUNTS="${DATA_DIR}/accounts.env" -# The three PDSes: short name, https name, handle domain. +# The three PDSes: short name, https name, handle domain. Each is under DEVNET_DOMAIN +# (set below), devnet.test by default. pds_name() { case "$1" in - alpha) echo alpha.devnet.test ;; - regular) echo regular.devnet.test ;; - prod) echo prod.devnet.test ;; + alpha | regular | prod) echo "$1.${DEVNET_DOMAIN}" ;; *) return 1 ;; esac } pds_domain() { case "$1" in - alpha) echo .devnet.test ;; - regular) echo .regular.devnet.test ;; - prod) echo .prod.devnet.test ;; + alpha) echo ".${DEVNET_DOMAIN}" ;; + regular | prod) echo ".$1.${DEVNET_DOMAIN}" ;; *) return 1 ;; esac } @@ -55,6 +53,11 @@ env_get() { sed -n "s/^$1=//p" "$2" | tail -1 } +# The domain every devnet name sits under: from the environment, else from .env, else +# devnet.test. Compose works it out the same way itself, so it is not exported. +DEVNET_DOMAIN="${DEVNET_DOMAIN:-$(env_get DEVNET_DOMAIN "${ROOT_DIR}/.env" | tr -d "\"'")}" +DEVNET_DOMAIN="${DEVNET_DOMAIN:-devnet.test}" + # env_put : replace those lines with stdin's lines. The # file stays mode 600 and is never readable by anyone else, even briefly. env_put() { @@ -88,7 +91,7 @@ add_name() { # render_apps: nginx's app routes, from data/https/apps (lines of " "), into # data/https/nginx/: one upstream per app at host.docker.internal: (apps.conf) and -# one map line per name (apps.map). See https/nginx.conf. +# one map line per name (apps.map). See https/nginx.conf.template. render_apps() { mkdir -p "${HTTPS_DIR}/nginx" _apps="${HTTPS_DIR}/apps" diff --git a/scripts/https-probe.mjs b/scripts/https-probe.mjs index 0b9d83a..b3f2a1e 100644 --- a/scripts/https-probe.mjs +++ b/scripts/https-probe.mjs @@ -22,7 +22,8 @@ // or a plain `npm i` of the two) // PROBE_DID the account to resolve and name in the PAR (default: the lexicon // authority, LEX_AUTHORITY_DID in data/devnet.env) -// PROBE_PDS the PDS that DID must name (default https://alpha.devnet.test) +// PROBE_PDS the PDS that DID must name (default https://alpha., the +// domain being DEVNET_DOMAIN in data/devnet.env, else devnet.test) // PROBE_APP the app name (default probe: https://probe.devnet.internal/) // PROBE_CLIENT_PORT where the client's documents are served (default 5480) // PROBE_CLIENT_HOST the address to serve on (default: the docker0 address, which is @@ -43,22 +44,24 @@ if (!ATCUTE_DIR) { console.log('FAIL setup: set ATCUTE_DIR to a directory node resolves @atcute/* from'); process.exit(2); } -const PDS = process.env.PROBE_PDS ?? 'https://alpha.devnet.test'; +const devnetEnv = () => { + try { + return readFileSync(join(ROOT, 'data/devnet.env'), 'utf8'); + } catch { + return ''; + } +}; +const DOMAIN = (devnetEnv().match(/^DEVNET_DOMAIN=(.+)$/m)?.[1] ?? 'devnet.test').trim(); +const PDS = process.env.PROBE_PDS ?? `https://alpha.${DOMAIN}`; const APP = process.env.PROBE_APP ?? 'probe'; const SITE = `https://${APP}.devnet.internal`; +// Always a .test name, whatever the domain: check (c) is about the .test suffix. const TEST_SITE = `https://${APP}.devnet.test`; const PORT = Number(process.env.PROBE_CLIENT_PORT ?? 5480); const HOST = process.env.PROBE_CLIENT_HOST ?? os.networkInterfaces().docker0?.find((a) => a.family === 'IPv4')?.address ?? '127.0.0.1'; -const devnetEnv = () => { - try { - return readFileSync(join(ROOT, 'data/devnet.env'), 'utf8'); - } catch { - return ''; - } -}; const did = ( process.env.PROBE_DID ?? devnetEnv().match(/^LEX_AUTHORITY_DID=(.+)$/m)?.[1] ?? diff --git a/scripts/https-run b/scripts/https-run index 47341ed..002851b 100755 --- a/scripts/https-run +++ b/scripts/https-run @@ -3,7 +3,8 @@ set -eu # Run a command as if the https devnet were the real network, for that command's # process tree only: -# - the three PDS names (alpha.devnet.test, regular.devnet.test, prod.devnet.test), +# - the three PDS names (alpha.devnet.test, regular.devnet.test, prod.devnet.test, +# under DEVNET_DOMAIN if it is set; scripts/https-lib.sh), # plc.directory, every handle in data/https/names (scripts/https-account.sh and # https-up.sh add them) and every app in data/https/apps, as .devnet.internal # (scripts/https-app.sh), resolve to 127.0.0.1, where nginx answers on 443; @@ -21,6 +22,7 @@ set -eu # or app added later needs a restart of the command. SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +. "${SCRIPT_DIR}/https-lib.sh" DATA="$(cd "${SCRIPT_DIR}/.." && pwd)/data/https" CA="${DATA}/ca.crt" @@ -37,7 +39,7 @@ if ! command -v bwrap >/dev/null 2>&1; then exit 1 fi -NAMES="alpha.devnet.test regular.devnet.test prod.devnet.test plc.directory" +NAMES="$(pds_name alpha) $(pds_name regular) $(pds_name prod) plc.directory" if [ -f "${DATA}/names" ]; then NAMES="${NAMES} $(grep -E '^[A-Za-z0-9.-]+$' "${DATA}/names" | tr '\n' ' ')" fi diff --git a/scripts/https-up.sh b/scripts/https-up.sh index 3d1fa83..2957eb4 100755 --- a/scripts/https-up.sh +++ b/scripts/https-up.sh @@ -108,6 +108,7 @@ DENV="${DATA_DIR}/devnet.env" echo "# The https devnet's own facts, written by scripts/https-up.sh. Nothing secret:" echo "# logins are in data/accounts.env. Run clients under scripts/https-run." echo "DEVNET_PROJECT=${DEVNET_PROJECT}" + echo "DEVNET_DOMAIN=${DEVNET_DOMAIN}" echo "ALPHA_PDS_URL=https://$(pds_name alpha)" echo "REGULAR_PDS_URL=https://$(pds_name regular)" echo "PROD_PDS_URL=https://$(pds_name prod)" diff --git a/scripts/init.sh b/scripts/init.sh index 55e6af3..f57006f 100755 --- a/scripts/init.sh +++ b/scripts/init.sh @@ -95,13 +95,13 @@ if [ "${SEED_ACCOUNTS}" = "true" ]; then create_account \ "alice${HANDLE_DOMAIN}" \ - "alice@devnet.test" \ + "alice@${DEVNET_DOMAIN:-devnet.test}" \ "alice-devnet-pass" \ "ALICE" create_account \ "bob${HANDLE_DOMAIN}" \ - "bob@devnet.test" \ + "bob@${DEVNET_DOMAIN:-devnet.test}" \ "bob-devnet-pass" \ "BOB" diff --git a/scripts/lexicon-authority.sh b/scripts/lexicon-authority.sh index f10af9a..dfdf543 100755 --- a/scripts/lexicon-authority.sh +++ b/scripts/lexicon-authority.sh @@ -28,7 +28,7 @@ if [ -f "${ROOT_DIR}/.env" ]; then fi PDS_URL="${DEVNET_PDS_URL:-http://localhost:${DEVNET_SPACES_PDS_PORT:-3010}}" -HANDLE="${DEVNET_LEX_AUTHORITY_HANDLE:-lex-authority${DEVNET_HANDLE_DOMAIN:-.devnet.test}}" +HANDLE="${DEVNET_LEX_AUTHORITY_HANDLE:-lex-authority${DEVNET_HANDLE_DOMAIN:-.${DEVNET_DOMAIN:-devnet.test}}}" PASSWORD="${DEVNET_LEX_AUTHORITY_PASSWORD:-lex-authority-devnet-pass}" DID=$(curl -s -X POST "${PDS_URL}/xrpc/com.atproto.server.createSession" \ @@ -45,7 +45,7 @@ if [ -z "${DID}" ]; then -H "Content-Type: application/json" \ -d "{ \"handle\": \"${HANDLE}\", - \"email\": \"lex-authority@devnet.test\", + \"email\": \"lex-authority@${DEVNET_DOMAIN:-devnet.test}\", \"password\": \"${PASSWORD}\", \"inviteCode\": \"${INVITE_CODE}\" }") From b05ae41d2e55fd84ffb3ba73b3a65c33aee84e50 Mon Sep 17 00:00:00 2001 From: Tom Scanlan Date: Fri, 16 Oct 2026 21:05:26 -0400 Subject: [PATCH 2/8] README: the regular-PDS account example uses bookreader; the PDS reserves reader --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a3cb385..ef90655 100644 --- a/README.md +++ b/README.md @@ -692,7 +692,7 @@ to the file named. ```bash scripts/https-account.sh bookclubowner BOOKCLUB_OWNER # bookclubowner.devnet.test on the alpha -scripts/https-account.sh reader BOOKCLUB_READER regular # reader.regular.devnet.test +scripts/https-account.sh bookreader BOOKCLUB_READER regular # bookreader.regular.devnet.test ``` It prints the DID alone, and a rerun finds the account instead of making another. The login goes to From c5f6aee0e9229d029e72a24a58a4842bc6b2627c Mon Sep 17 00:00:00 2001 From: Tom Scanlan Date: Fri, 16 Oct 2026 21:28:17 -0400 Subject: [PATCH 3/8] DEVNET_DOMAIN reaches only the https devnet; docker-compose.yml is as on main The plain stack (docker-compose.yml, with or without the test overlay) renders exactly as on main again, so its users see no change and no compose version has to read a default inside a default. The https overlay now sets the alpha's mail sender and init's handle domain from DEVNET_DOMAIN itself. --- .env.example | 5 +++-- README.md | 9 +++++---- docker-compose.https.yml | 9 +++++++++ docker-compose.yml | 13 ++++++------- 4 files changed, 23 insertions(+), 13 deletions(-) diff --git a/.env.example b/.env.example index da0c2c3..fc432f2 100644 --- a/.env.example +++ b/.env.example @@ -11,8 +11,9 @@ DEVNET_DB_NAME=plc DEVNET_SMTP_URL=smtp://maildev:1025 # Optional: Override defaults -# The domain every name sits under: the PDS (alpha. on the https devnet) and -# the handles (.). The two below default to it. +# The https devnet's domain: its PDSes are alpha., regular. and +# prod., and the alpha's handles .. The plain stack uses +# DEVNET_PDS_HOSTNAME and DEVNET_HANDLE_DOMAIN instead. # DEVNET_DOMAIN=devnet.test # DEVNET_PDS_HOSTNAME=devnet.test # DEVNET_PDS_ADMIN_PASSWORD=devnet-admin-password diff --git a/README.md b/README.md index ef90655..ab6abdc 100644 --- a/README.md +++ b/README.md @@ -599,7 +599,8 @@ commands run under `scripts/https-run`, so every other program still reaches the `plc.directory`. `devnet.test` is the default of one setting, `DEVNET_DOMAIN`, and every name in the table above -(and in the rest of this README) uses it. Set it in the environment or in `.env` and the same +(and in the rest of this README) uses it. Only the https devnet reads it; the plain stack keeps +`DEVNET_PDS_HOSTNAME` and `DEVNET_HANDLE_DOMAIN`. Set it in the environment or in `.env` and the same stack comes up under that domain: `alpha.` with handles at `.`, `regular.`, `prod.`. `plc.directory` and `.devnet.internal` stay as they are. A checkout keeps one domain, because its CA can vouch only for the domain it was made for @@ -875,13 +876,13 @@ All settings have sensible defaults. Override via `.env` or environment variable | Variable | Default | Description | |----------|---------|-------------| -| `DEVNET_DOMAIN` | `devnet.test` | The domain every name sits under ([The https devnet](#the-https-devnet)) | -| `DEVNET_PDS_HOSTNAME` | `DEVNET_DOMAIN` | PDS service hostname (the https devnet names its PDSes itself) | +| `DEVNET_DOMAIN` | `devnet.test` | The https devnet's domain, which every name there sits under ([The https devnet](#the-https-devnet)). The plain stack does not read it | +| `DEVNET_PDS_HOSTNAME` | `devnet.test` | PDS service hostname (the https devnet names its PDSes itself) | | `DEVNET_PDS_IMAGE` | `ghcr.io/bluesky-social/pds:0.4` | PDS image (see [Choosing a PDS version](#choosing-a-pds-version)) | | `DEVNET_PDS_ADMIN_PASSWORD` | `devnet-admin-password` | PDS admin password | | `DEVNET_APPVIEW_URL` / `DEVNET_APPVIEW_DID` | `https://appview.invalid` / `did:example:invalid` | Where the PDS forwards app.bsky.* reads and methods it doesn't implement. Unresolvable, so nothing reaches Bluesky | | `DEVNET_REPORT_SERVICE_URL` / `DEVNET_REPORT_SERVICE_DID` | `https://moderator.invalid` / `did:example:invalid` | Where reports go | -| `DEVNET_HANDLE_DOMAIN` | `.` + `DEVNET_DOMAIN` | Handle suffix for accounts | +| `DEVNET_HANDLE_DOMAIN` | `.devnet.test` | Handle suffix for accounts (the https devnet uses `.` + `DEVNET_DOMAIN`) | | `DEVNET_SEED_ACCOUNTS` | `true` | Create alice/bob on startup | | `DEVNET_PDS_REGULAR_IMAGE` / `DEVNET_PDS_PROD_IMAGE` | `ghcr.io/bluesky-social/pds:0.4.5037` / a pinned production digest | The two extra PDS builds ([Several PDS builds side by side](#several-pds-builds-side-by-side)) | | `DEVNET_LEXICON_AUTHORITY_DID` | empty | The spaces PDS's lexicon authority ([Spaces PDS and unpublished lexicons](#spaces-pds-and-unpublished-lexicons)) | diff --git a/docker-compose.https.yml b/docker-compose.https.yml index 20f77cb..99b49c9 100644 --- a/docker-compose.https.yml +++ b/docker-compose.https.yml @@ -53,6 +53,7 @@ services: # Explicit: the default would be .alpha.devnet.test. PDS_SERVICE_HANDLE_DOMAINS: .${DEVNET_DOMAIN:-devnet.test} PDS_HANDLE_DOMAINS: .${DEVNET_DOMAIN:-devnet.test} + PDS_EMAIL_FROM_ADDRESS: noreply@${DEVNET_DOMAIN:-devnet.test} # The public URL is https, so the alpha runs as it would on the real network. # ("false", not !reset: compose keeps a key an earlier overlay set.) PDS_DEV_MODE: "false" @@ -89,6 +90,14 @@ services: nginx: condition: service_healthy + # init seeds alice and bob on the alpha, so under its handle domain. docker-compose.yml + # itself does not read DEVNET_DOMAIN: the plain stack keeps DEVNET_HANDLE_DOMAIN. + init: + environment: + PDS_HOSTNAME: alpha.${DEVNET_DOMAIN:-devnet.test} + HANDLE_DOMAIN: .${DEVNET_DOMAIN:-devnet.test} + DEVNET_DOMAIN: ${DEVNET_DOMAIN:-devnet.test} + relay-init: depends_on: nginx: diff --git a/docker-compose.yml b/docker-compose.yml index b44d493..a9cb1d2 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -58,7 +58,7 @@ services: condition: service_healthy environment: PDS_PORT: 3000 - PDS_HOSTNAME: ${DEVNET_PDS_HOSTNAME:-${DEVNET_DOMAIN:-devnet.test}} + PDS_HOSTNAME: ${DEVNET_PDS_HOSTNAME:-devnet.test} PDS_DATA_DIRECTORY: /pds/data PDS_BLOBSTORE_DISK_LOCATION: /pds/data/blocks PDS_DID_PLC_URL: http://plc:2582 @@ -81,9 +81,9 @@ services: PDS_JWT_SECRET: devnet-jwt-secret-not-for-production PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef" PDS_EMAIL_SMTP_URL: ${DEVNET_SMTP_URL:-smtp://maildev:1025} - PDS_EMAIL_FROM_ADDRESS: noreply@${DEVNET_DOMAIN:-devnet.test} - PDS_HANDLE_DOMAINS: ${DEVNET_HANDLE_DOMAIN:-.${DEVNET_DOMAIN:-devnet.test}} - PDS_SERVICE_HANDLE_DOMAINS: ${DEVNET_HANDLE_DOMAIN:-.${DEVNET_DOMAIN:-devnet.test}} + PDS_EMAIL_FROM_ADDRESS: noreply@devnet.test + PDS_HANDLE_DOMAINS: ${DEVNET_HANDLE_DOMAIN:-.devnet.test} + PDS_SERVICE_HANDLE_DOMAINS: ${DEVNET_HANDLE_DOMAIN:-.devnet.test} LOG_ENABLED: "true" NODE_ENV: development volumes: @@ -159,10 +159,9 @@ services: - ./scripts:/scripts:ro - ./data:/devnet-data environment: - PDS_HOSTNAME: ${DEVNET_PDS_HOSTNAME:-${DEVNET_DOMAIN:-devnet.test}} + PDS_HOSTNAME: ${DEVNET_PDS_HOSTNAME:-devnet.test} PDS_ADMIN_PASSWORD: ${DEVNET_PDS_ADMIN_PASSWORD:-devnet-admin-password} - HANDLE_DOMAIN: ${DEVNET_HANDLE_DOMAIN:-.${DEVNET_DOMAIN:-devnet.test}} - DEVNET_DOMAIN: ${DEVNET_DOMAIN:-devnet.test} + HANDLE_DOMAIN: ${DEVNET_HANDLE_DOMAIN:-.devnet.test} SEED_ACCOUNTS: ${DEVNET_SEED_ACCOUNTS:-true} entrypoint: /bin/sh command: ["/scripts/init.sh"] From 346ebde74629fd388d73bc204f1ed652b4bbf8c7 Mon Sep 17 00:00:00 2001 From: Tom Scanlan Date: Fri, 16 Oct 2026 21:49:42 -0400 Subject: [PATCH 4/8] lexicon-authority.sh and create-account.sh are plain-stack tools, as on main: DEVNET_DOMAIN stays the https devnet's --- scripts/create-account.sh | 2 +- scripts/lexicon-authority.sh | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/create-account.sh b/scripts/create-account.sh index de11c03..117eac9 100755 --- a/scripts/create-account.sh +++ b/scripts/create-account.sh @@ -24,7 +24,7 @@ PDS_URL="${DEVNET_PDS_URL:-http://localhost:${DEVNET_PDS_PORT:-3000}}" PDS_ADMIN_PASSWORD="${DEVNET_PDS_ADMIN_PASSWORD:-devnet-admin-password}" HANDLE="${1:?Usage: create-account.sh [email] [password]}" -EMAIL="${2:-${HANDLE}@${DEVNET_DOMAIN:-devnet.test}}" +EMAIL="${2:-${HANDLE}@devnet.test}" PASSWORD="${3:-$(head -c 16 /dev/urandom | base64 | tr -dc 'a-zA-Z0-9' | head -c 16)}" # Get or create invite code diff --git a/scripts/lexicon-authority.sh b/scripts/lexicon-authority.sh index dfdf543..f10af9a 100755 --- a/scripts/lexicon-authority.sh +++ b/scripts/lexicon-authority.sh @@ -28,7 +28,7 @@ if [ -f "${ROOT_DIR}/.env" ]; then fi PDS_URL="${DEVNET_PDS_URL:-http://localhost:${DEVNET_SPACES_PDS_PORT:-3010}}" -HANDLE="${DEVNET_LEX_AUTHORITY_HANDLE:-lex-authority${DEVNET_HANDLE_DOMAIN:-.${DEVNET_DOMAIN:-devnet.test}}}" +HANDLE="${DEVNET_LEX_AUTHORITY_HANDLE:-lex-authority${DEVNET_HANDLE_DOMAIN:-.devnet.test}}" PASSWORD="${DEVNET_LEX_AUTHORITY_PASSWORD:-lex-authority-devnet-pass}" DID=$(curl -s -X POST "${PDS_URL}/xrpc/com.atproto.server.createSession" \ @@ -45,7 +45,7 @@ if [ -z "${DID}" ]; then -H "Content-Type: application/json" \ -d "{ \"handle\": \"${HANDLE}\", - \"email\": \"lex-authority@${DEVNET_DOMAIN:-devnet.test}\", + \"email\": \"lex-authority@devnet.test\", \"password\": \"${PASSWORD}\", \"inviteCode\": \"${INVITE_CODE}\" }") From e885c829bec2037616211957b44e75e6ff3d910e Mon Sep 17 00:00:00 2001 From: Tom Scanlan Date: Fri, 16 Oct 2026 22:19:41 -0400 Subject: [PATCH 5/8] DEVNET_DOMAIN leaves the multi-PDS overlay as on main; the https overlay names its PDSes The plain multi-PDS stack shares docker-compose.multi-pds.yml with the https devnet, so a DEVNET_DOMAIN set for https changed the plain stack's service DIDs, handle domains and mail senders too. That file is byte-identical to main again, and the https overlay sets the three PDSes' service DIDs, handle domains and mail sender itself. The six-file https stack renders the same services as before, default and custom. --- docker-compose.https.yml | 9 ++++++++- docker-compose.multi-pds.yml | 12 ++++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/docker-compose.https.yml b/docker-compose.https.yml index 99b49c9..57bebee 100644 --- a/docker-compose.https.yml +++ b/docker-compose.https.yml @@ -44,16 +44,19 @@ x-https-pds-env: &https-pds-env NODE_EXTRA_CA_CERTS: /https/ca.crt # .devnet.internal resolves to nginx inside each PDS (https/app-names.cjs). NODE_OPTIONS: --require /https/app-names.cjs + PDS_EMAIL_FROM_ADDRESS: noreply@${DEVNET_DOMAIN:-devnet.test} services: pds: environment: <<: *https-pds-env PDS_HOSTNAME: alpha.${DEVNET_DOMAIN:-devnet.test} + # docker-compose.multi-pds.yml names all three PDSes under devnet.test, as on the + # plain stack, which does not read DEVNET_DOMAIN; here they follow it. + PDS_SERVICE_DID: did:web:alpha.${DEVNET_DOMAIN:-devnet.test} # Explicit: the default would be .alpha.devnet.test. PDS_SERVICE_HANDLE_DOMAINS: .${DEVNET_DOMAIN:-devnet.test} PDS_HANDLE_DOMAINS: .${DEVNET_DOMAIN:-devnet.test} - PDS_EMAIL_FROM_ADDRESS: noreply@${DEVNET_DOMAIN:-devnet.test} # The public URL is https, so the alpha runs as it would on the real network. # ("false", not !reset: compose keeps a key an earlier overlay set.) PDS_DEV_MODE: "false" @@ -66,6 +69,8 @@ services: environment: <<: *https-pds-env PDS_HOSTNAME: regular.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_DID: did:web:regular.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_HANDLE_DOMAINS: .regular.${DEVNET_DOMAIN:-devnet.test} volumes: - *devnet-ca - *app-names @@ -74,6 +79,8 @@ services: environment: <<: *https-pds-env PDS_HOSTNAME: prod.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_DID: did:web:prod.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_HANDLE_DOMAINS: .prod.${DEVNET_DOMAIN:-devnet.test} volumes: - *devnet-ca - *app-names diff --git a/docker-compose.multi-pds.yml b/docker-compose.multi-pds.yml index 297f2b0..742262f 100644 --- a/docker-compose.multi-pds.yml +++ b/docker-compose.multi-pds.yml @@ -42,7 +42,7 @@ x-extra-pds-env: &extra-pds-env PDS_JWT_SECRET: devnet-jwt-secret-not-for-production PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX: "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef" PDS_EMAIL_SMTP_URL: ${DEVNET_SMTP_URL:-smtp://maildev:1025} - PDS_EMAIL_FROM_ADDRESS: noreply@${DEVNET_DOMAIN:-devnet.test} + PDS_EMAIL_FROM_ADDRESS: noreply@devnet.test PDS_LEXICON_AUTHORITY_DID: ${DEVNET_LEXICON_AUTHORITY_DID:-} LOG_ENABLED: "true" NODE_ENV: development @@ -51,7 +51,7 @@ services: pds: environment: # All three would otherwise be did:web:localhost. - PDS_SERVICE_DID: did:web:alpha.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_DID: did:web:alpha.devnet.test ports: !override - "${DEVNET_SPACES_PDS_PORT:-3010}:${DEVNET_SPACES_PDS_PORT:-3010}" - "3020:3020" @@ -64,8 +64,8 @@ services: environment: <<: *extra-pds-env PDS_PORT: "3020" - PDS_SERVICE_DID: did:web:regular.${DEVNET_DOMAIN:-devnet.test} - PDS_SERVICE_HANDLE_DOMAINS: .regular.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_DID: did:web:regular.devnet.test + PDS_SERVICE_HANDLE_DOMAINS: .regular.devnet.test volumes: - pds-regular-data:/pds/data healthcheck: @@ -81,8 +81,8 @@ services: environment: <<: *extra-pds-env PDS_PORT: "3030" - PDS_SERVICE_DID: did:web:prod.${DEVNET_DOMAIN:-devnet.test} - PDS_SERVICE_HANDLE_DOMAINS: .prod.${DEVNET_DOMAIN:-devnet.test} + PDS_SERVICE_DID: did:web:prod.devnet.test + PDS_SERVICE_HANDLE_DOMAINS: .prod.devnet.test volumes: - pds-prod-data:/pds/data healthcheck: From 5e7cd3da86a66c719a4fb0a2a9b3ebdc07ae135f Mon Sep 17 00:00:00 2001 From: Tom Scanlan Date: Fri, 16 Oct 2026 22:19:41 -0400 Subject: [PATCH 6/8] https-run: the PDS names come from https-lib.sh in a subshell Sourcing https-lib.sh in https-run set ROOT_DIR, DATA_DIR, HTTPS_DIR and DEVNET_ACCOUNTS, and where the caller had exported one of those, the wrapped command got the devnet's value instead of its own. The command's environment is as on main. --- scripts/https-run | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/https-run b/scripts/https-run index 002851b..23607c8 100755 --- a/scripts/https-run +++ b/scripts/https-run @@ -22,7 +22,6 @@ set -eu # or app added later needs a restart of the command. SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" -. "${SCRIPT_DIR}/https-lib.sh" DATA="$(cd "${SCRIPT_DIR}/.." && pwd)/data/https" CA="${DATA}/ca.crt" @@ -39,7 +38,10 @@ if ! command -v bwrap >/dev/null 2>&1; then exit 1 fi -NAMES="$(pds_name alpha) $(pds_name regular) $(pds_name prod) plc.directory" +# In a subshell: https-lib.sh sets ROOT_DIR, DATA_DIR and the like, and where the caller +# exported one of those, the command would see ours instead. +NAMES="$(. "${SCRIPT_DIR}/https-lib.sh" && echo "$(pds_name alpha) $(pds_name regular) $(pds_name prod)")" +NAMES="${NAMES} plc.directory" if [ -f "${DATA}/names" ]; then NAMES="${NAMES} $(grep -E '^[A-Za-z0-9.-]+$' "${DATA}/names" | tr '\n' ' ')" fi From 39cff5948eb702c0a10c7d6a9f84fe2856aef744 Mon Sep 17 00:00:00 2001 From: Tom Scanlan Date: Fri, 16 Oct 2026 22:19:57 -0400 Subject: [PATCH 7/8] https-lib: DEVNET_DOMAIN is read from .env as compose reads it, and must be a domain name The scripts took only an exact DEVNET_DOMAIN= line and fell back to .env when the variable was set but empty, so `export DEVNET_DOMAIN=...`, a trailing comment or a CRLF gave the scripts one domain and nginx another. They now accept those forms, let a variable set in the environment win even when empty (then devnet.test), and refuse a value that is not a domain name, which includes a ${...} compose would expand. --- README.md | 6 ++++-- scripts/https-lib.sh | 26 +++++++++++++++++++++++--- 2 files changed, 27 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index ab6abdc..0351bf2 100644 --- a/README.md +++ b/README.md @@ -603,8 +603,10 @@ commands run under `scripts/https-run`, so every other program still reaches the `DEVNET_PDS_HOSTNAME` and `DEVNET_HANDLE_DOMAIN`. Set it in the environment or in `.env` and the same stack comes up under that domain: `alpha.` with handles at `.`, `regular.`, `prod.`. `plc.directory` and `.devnet.internal` stay as they are. -A checkout keeps one domain, because its CA can vouch only for the domain it was made for -([The CA](#the-ca)); for a second domain, use a second checkout. +In `.env`, write the domain itself (quotes, `export` and a trailing comment are fine); the scripts +refuse a `${...}` there, which compose would expand and they would not. A checkout keeps one domain, +because its CA can vouch only for the domain it was made for ([The CA](#the-ca)); for a second +domain, use a second checkout. The devnet knows no app. It has no app's accounts, invite codes, lexicons or ports. It gives an app tools to make its own (below), and using them is optional. diff --git a/scripts/https-lib.sh b/scripts/https-lib.sh index d9b2aba..17533f0 100644 --- a/scripts/https-lib.sh +++ b/scripts/https-lib.sh @@ -53,10 +53,30 @@ env_get() { sed -n "s/^$1=//p" "$2" | tail -1 } -# The domain every devnet name sits under: from the environment, else from .env, else -# devnet.test. Compose works it out the same way itself, so it is not exported. -DEVNET_DOMAIN="${DEVNET_DOMAIN:-$(env_get DEVNET_DOMAIN "${ROOT_DIR}/.env" | tr -d "\"'")}" +# env_domain : DEVNET_DOMAIN's last value in an env file, read as compose reads +# it: an export prefix, quotes, a trailing " # comment" and a CRLF line end are allowed. +env_domain() { + [ -f "$1" ] || return 0 + sed -n -E 's/\r$//; s/^[[:space:]]*(export[[:space:]]+)?DEVNET_DOMAIN[[:space:]]*=[[:space:]]*//p' "$1" | + tail -1 | + sed -E "s/^\"([^\"]*)\".*/\1/; t; s/^'([^']*)'.*/\1/; t; s/[[:space:]]+#.*//; s/[[:space:]]+\$//" +} + +# The domain every devnet name sits under, as compose works out ${DEVNET_DOMAIN:-devnet.test}: +# set in the environment (even empty), that wins; else .env; empty means devnet.test. +# Compose works it out itself, so it is not exported. +if [ -z "${DEVNET_DOMAIN+set}" ]; then + DEVNET_DOMAIN="$(env_domain "${ROOT_DIR}/.env")" +fi DEVNET_DOMAIN="${DEVNET_DOMAIN:-devnet.test}" +# A domain name and nothing else. This also refuses ${...} in .env, which compose would +# expand and these scripts do not. +case "${DEVNET_DOMAIN}" in + *[!A-Za-z0-9.-]* | .* | *. | *..*) + echo "DEVNET_DOMAIN=${DEVNET_DOMAIN} is not a domain name (set in the environment or .env)" >&2 + exit 1 + ;; +esac # env_put : replace those lines with stdin's lines. The # file stays mode 600 and is never readable by anyone else, even briefly. From 05259ef7a16b8a658871444e599bc0a6543f5517 Mon Sep 17 00:00:00 2001 From: Tom Scanlan Date: Fri, 16 Oct 2026 22:19:58 -0400 Subject: [PATCH 8/8] README: rerun https-up.sh after pulling the nginx template; atmo fixtures follow the domain An https nginx container made before the pull still mounts https/nginx.conf, and once the file is gone it cannot start again until https-up.sh recreates it; the upgrade notes say so, and where an overlay's own nginx config goes now. The atmo groups e2e recipe wrote devnet.test as the handle domain under any DEVNET_DOMAIN; it reads it from data/devnet.env. --- README.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 0351bf2..59695e5 100644 --- a/README.md +++ b/README.md @@ -76,6 +76,15 @@ Changes to the default stack that existing setups will notice: `data/accounts.env` gets new DIDs and a new invite code. - **The PDS runs as root.** Release images already did. This lets `DEVNET_PDS_IMAGE` take monorepo images, which default to `node`. +- **The https devnet's nginx config is a template now: rerun `scripts/https-up.sh` after pulling.** + `https/nginx.conf` became `https/nginx.conf.template`, which the nginx image fills in with + `DEVNET_DOMAIN` when it starts. An nginx container made before the pull still mounts the old file, + and once the file is gone that container cannot start again (Docker leaves an empty directory in + its place). `https-up.sh` recreates it on the new mount. Checking out an older version needs the + same rerun, the other way round. An overlay that mounts its own config over + `/etc/nginx/conf.d/default.conf` should mount it at `/etc/nginx/templates/default.conf.template` + instead, and one that replaces nginx's command must keep the image's entrypoint, which writes the + template out. ## Integrating into your project @@ -415,7 +424,7 @@ unset ACCOUNTS_FILE { grep -E '^E2E_[A-Z]+_DID=' data/atmo-e2e.env echo "E2E_PDS=$(sed -n 's/^ALPHA_PDS_URL=//p' data/devnet.env)" echo "E2E_PLC_URL=$(sed -n 's/^PLC_URL=//p' data/devnet.env)" - echo "E2E_HANDLE_DOMAIN=devnet.test" + echo "E2E_HANDLE_DOMAIN=$(sed -n 's/^DEVNET_DOMAIN=//p' data/devnet.env)" echo "E2E_CREDENTIALS=$PWD/data/atmo-e2e.env"; } > data/atmo-e2e.fixtures.env ```