The https devnet takes its domain as a setting, DEVNET_DOMAIN

.env.example

@@ -11,6 +11,10 @@ DEVNET_DB_NAME=plc
 DEVNET_SMTP_URL=smtp://maildev:1025
 
 # Optional: Override defaults
+# The https devnet's domain: its PDSes are alpha.<domain>, regular.<domain> and
+# prod.<domain>, and the alpha's handles <name>.<domain>. The plain stack uses
+# DEVNET_PDS_HOSTNAME and DEVNET_HANDLE_DOMAIN instead.
+# DEVNET_DOMAIN=devnet.test
 # DEVNET_PDS_HOSTNAME=devnet.test
 # DEVNET_PDS_ADMIN_PASSWORD=devnet-admin-password
 # DEVNET_HANDLE_DOMAIN=.devnet.test

README.md

@@ -76,6 +76,15 @@ Changes to the default stack that existing setups will notice:
   `data/accounts.env` gets new DIDs and a new invite code.
 - **The PDS runs as root.** Release images already did. This lets `DEVNET_PDS_IMAGE` take monorepo
   images, which default to `node`.
+- **The https devnet's nginx config is a template now: rerun `scripts/https-up.sh` after pulling.**
+  `https/nginx.conf` became `https/nginx.conf.template`, which the nginx image fills in with
+  `DEVNET_DOMAIN` when it starts. An nginx container made before the pull still mounts the old file,
+  and once the file is gone that container cannot start again (Docker leaves an empty directory in
+  its place). `https-up.sh` recreates it on the new mount. Checking out an older version needs the
+  same rerun, the other way round. An overlay that mounts its own config over
+  `/etc/nginx/conf.d/default.conf` should mount it at `/etc/nginx/templates/default.conf.template`
+  instead, and one that replaces nginx's command must keep the image's entrypoint, which writes the
+  template out.
 
 ## Integrating into your project
 
@@ -415,7 +424,7 @@ unset ACCOUNTS_FILE
 { grep -E '^E2E_[A-Z]+_DID=' data/atmo-e2e.env
   echo "E2E_PDS=$(sed -n 's/^ALPHA_PDS_URL=//p' data/devnet.env)"
   echo "E2E_PLC_URL=$(sed -n 's/^PLC_URL=//p' data/devnet.env)"
-  echo "E2E_HANDLE_DOMAIN=devnet.test"
+  echo "E2E_HANDLE_DOMAIN=$(sed -n 's/^DEVNET_DOMAIN=//p' data/devnet.env)"
   echo "E2E_CREDENTIALS=$PWD/data/atmo-e2e.env"; } > data/atmo-e2e.fixtures.env
 ```
 
@@ -598,6 +607,16 @@ the three PDSes as `wss://` hosts. On the machine, the names resolve to `127.0.0
 commands run under `scripts/https-run`, so every other program still reaches the real
 `plc.directory`.
 
+`devnet.test` is the default of one setting, `DEVNET_DOMAIN`, and every name in the table above
+(and in the rest of this README) uses it. Only the https devnet reads it; the plain stack keeps
+`DEVNET_PDS_HOSTNAME` and `DEVNET_HANDLE_DOMAIN`. Set it in the environment or in `.env` and the same
+stack comes up under that domain: `alpha.<domain>` with handles at `<name>.<domain>`,
+`regular.<domain>`, `prod.<domain>`. `plc.directory` and `<name>.devnet.internal` stay as they are.
+In `.env`, write the domain itself (quotes, `export` and a trailing comment are fine); the scripts
+refuse a `${...}` there, which compose would expand and they would not. A checkout keeps one domain,
+because its CA can vouch only for the domain it was made for ([The CA](#the-ca)); for a second
+domain, use a second checkout.
+
 The devnet knows no app. It has no app's accounts, invite codes, lexicons or ports. It gives an app
 tools to make its own (below), and using them is optional.
 
@@ -646,7 +665,8 @@ a new DID.
 
 - `data/devnet.env`: the devnet's own facts, nothing secret. `ALPHA_PDS_URL`, `REGULAR_PDS_URL`,
   `PROD_PDS_URL`, `PLC_URL` (`https://plc.directory`), `JETSTREAM_URL` (`ws://127.0.0.1:<port>`),
-  `MAILDEV_URL`, `LEX_AUTHORITY_DID`, `LEX_AUTHORITY_HANDLE`, `DEVNET_PROJECT` and `DEVNET_CA_FILE`.
+  `MAILDEV_URL`, `LEX_AUTHORITY_DID`, `LEX_AUTHORITY_HANDLE`, `DEVNET_PROJECT`, `DEVNET_DOMAIN` and
+  `DEVNET_CA_FILE`.
   An app can source it: `set -a; . <devnet>/data/devnet.env; set +a`.
 - `data/accounts.env` (mode 600): logins. `DEVNET_INVITE_CODE`, `ALICE_*` and `BOB_*` from init,
   `LEX_AUTHORITY_*`, and whatever the tools below add.
@@ -684,7 +704,7 @@ to the file named.
 
 ```bash
 scripts/https-account.sh bookclubowner BOOKCLUB_OWNER            # bookclubowner.devnet.test on the alpha
-scripts/https-account.sh reader BOOKCLUB_READER regular          # reader.regular.devnet.test
+scripts/https-account.sh bookreader BOOKCLUB_READER regular      # bookreader.regular.devnet.test
 ```
 
 It prints the DID alone, and a rerun finds the account instead of making another. The login goes to
@@ -789,8 +809,8 @@ deeper names with `HTTPS_EXTRA_NAMES="x.mine.devnet.test"`, then restart nginx.
 under `devnet.test` is already covered. To start over, delete `data/https/ca.*` and trust the new
 `ca.crt` wherever the old one was trusted.
 
-The CA can only vouch for servers under `devnet.test`, `devnet.internal` and `plc.directory` (name
-constraints, server auth only, no IP addresses), so trusting it puts no other site at risk, even if
+The CA can only vouch for servers under `devnet.test` (the `DEVNET_DOMAIN` it was made for),
+`devnet.internal` and `plc.directory` (name constraints, server auth only, no IP addresses), so trusting it puts no other site at risk, even if
 `data/https/ca.key` leaks. Keep that file on the dev machine, and remove the CA from a browser or OS
 when you're done with it.
 
@@ -867,12 +887,13 @@ All settings have sensible defaults. Override via `.env` or environment variable
 
 | Variable | Default | Description |
 |----------|---------|-------------|
-| `DEVNET_PDS_HOSTNAME` | `devnet.test` | PDS service hostname |
+| `DEVNET_DOMAIN` | `devnet.test` | The https devnet's domain, which every name there sits under ([The https devnet](#the-https-devnet)). The plain stack does not read it |
+| `DEVNET_PDS_HOSTNAME` | `devnet.test` | PDS service hostname (the https devnet names its PDSes itself) |
 | `DEVNET_PDS_IMAGE` | `ghcr.io/bluesky-social/pds:0.4` | PDS image (see [Choosing a PDS version](#choosing-a-pds-version)) |
 | `DEVNET_PDS_ADMIN_PASSWORD` | `devnet-admin-password` | PDS admin password |
 | `DEVNET_APPVIEW_URL` / `DEVNET_APPVIEW_DID` | `https://appview.invalid` / `did:example:invalid` | Where the PDS forwards app.bsky.* reads and methods it doesn't implement. Unresolvable, so nothing reaches Bluesky |
 | `DEVNET_REPORT_SERVICE_URL` / `DEVNET_REPORT_SERVICE_DID` | `https://moderator.invalid` / `did:example:invalid` | Where reports go |
-| `DEVNET_HANDLE_DOMAIN` | `.devnet.test` | Handle suffix for accounts |
+| `DEVNET_HANDLE_DOMAIN` | `.devnet.test` | Handle suffix for accounts (the https devnet uses `.` + `DEVNET_DOMAIN`) |
 | `DEVNET_SEED_ACCOUNTS` | `true` | Create alice/bob on startup |
 | `DEVNET_PDS_REGULAR_IMAGE` / `DEVNET_PDS_PROD_IMAGE` | `ghcr.io/bluesky-social/pds:0.4.5037` / a pinned production digest | The two extra PDS builds ([Several PDS builds side by side](#several-pds-builds-side-by-side)) |
 | `DEVNET_LEXICON_AUTHORITY_DID` | empty | The spaces PDS's lexicon authority ([Spaces PDS and unpublished lexicons](#spaces-pds-and-unpublished-lexicons)) |

docker-compose.https.yml

@@ -10,7 +10,8 @@
 # certificates, the lexicon authority's second pass and data/devnet.env. Use it.
 #
 # Names, all answered by nginx on 127.0.0.1:443 with one leaf from data/https/
-# (scripts/https-ca.sh), signed by a local CA that can vouch for nothing else:
+# (scripts/https-ca.sh), signed by a local CA that can vouch for nothing else.
+# devnet.test is DEVNET_DOMAIN's default; set it and every .devnet.test name follows:
 #
 #   alpha.devnet.test    *.devnet.test           pds          the spaces alpha, invites on
 #   regular.devnet.test  *.regular.devnet.test   pds-regular  a current release
@@ -43,15 +44,19 @@ x-https-pds-env: &https-pds-env
   NODE_EXTRA_CA_CERTS: /https/ca.crt
   # <name>.devnet.internal resolves to nginx inside each PDS (https/app-names.cjs).
   NODE_OPTIONS: --require /https/app-names.cjs
+  PDS_EMAIL_FROM_ADDRESS: noreply@${DEVNET_DOMAIN:-devnet.test}
 
 services:
   pds:
     environment:
       <<: *https-pds-env
-      PDS_HOSTNAME: alpha.devnet.test
+      PDS_HOSTNAME: alpha.${DEVNET_DOMAIN:-devnet.test}
+      # docker-compose.multi-pds.yml names all three PDSes under devnet.test, as on the
+      # plain stack, which does not read DEVNET_DOMAIN; here they follow it.
+      PDS_SERVICE_DID: did:web:alpha.${DEVNET_DOMAIN:-devnet.test}
       # Explicit: the default would be .alpha.devnet.test.
-      PDS_SERVICE_HANDLE_DOMAINS: .devnet.test
-      PDS_HANDLE_DOMAINS: .devnet.test
+      PDS_SERVICE_HANDLE_DOMAINS: .${DEVNET_DOMAIN:-devnet.test}
+      PDS_HANDLE_DOMAINS: .${DEVNET_DOMAIN:-devnet.test}
       # The public URL is https, so the alpha runs as it would on the real network.
       # ("false", not !reset: compose keeps a key an earlier overlay set.)
       PDS_DEV_MODE: "false"
@@ -63,15 +68,19 @@ services:
   pds-regular:
     environment:
       <<: *https-pds-env
-      PDS_HOSTNAME: regular.devnet.test
+      PDS_HOSTNAME: regular.${DEVNET_DOMAIN:-devnet.test}
+      PDS_SERVICE_DID: did:web:regular.${DEVNET_DOMAIN:-devnet.test}
+      PDS_SERVICE_HANDLE_DOMAINS: .regular.${DEVNET_DOMAIN:-devnet.test}
     volumes:
       - *devnet-ca
       - *app-names
 
   pds-prod:
     environment:
       <<: *https-pds-env
-      PDS_HOSTNAME: prod.devnet.test
+      PDS_HOSTNAME: prod.${DEVNET_DOMAIN:-devnet.test}
+      PDS_SERVICE_DID: did:web:prod.${DEVNET_DOMAIN:-devnet.test}
+      PDS_SERVICE_HANDLE_DOMAINS: .prod.${DEVNET_DOMAIN:-devnet.test}
     volumes:
       - *devnet-ca
       - *app-names
@@ -88,13 +97,21 @@ services:
       nginx:
         condition: service_healthy
 
+  # init seeds alice and bob on the alpha, so under its handle domain. docker-compose.yml
+  # itself does not read DEVNET_DOMAIN: the plain stack keeps DEVNET_HANDLE_DOMAIN.
+  init:
+    environment:
+      PDS_HOSTNAME: alpha.${DEVNET_DOMAIN:-devnet.test}
+      HANDLE_DOMAIN: .${DEVNET_DOMAIN:-devnet.test}
+      DEVNET_DOMAIN: ${DEVNET_DOMAIN:-devnet.test}
+
   relay-init:
     depends_on:
       nginx:
         condition: service_healthy
     environment:
       # Bare names are https/wss hosts to the relay.
-      RELAY_HOSTS: alpha.devnet.test regular.devnet.test prod.devnet.test
+      RELAY_HOSTS: alpha.${DEVNET_DOMAIN:-devnet.test} regular.${DEVNET_DOMAIN:-devnet.test} prod.${DEVNET_DOMAIN:-devnet.test}
 
   # Resync fetches repos through an SSRF-safe transport, which refuses nginx's address,
   # so TAP follows the relay's stream but cannot resync a repo here.
@@ -135,18 +152,23 @@ services:
     extra_hosts:
       # This machine, where an app registered with scripts/https-app.sh listens.
       - host.docker.internal:host-gateway
+    environment:
+      # The image fills ${DEVNET_DOMAIN} into the template at start, and only names
+      # starting DEVNET_, so nginx's own $host and the like are left alone.
+      DEVNET_DOMAIN: ${DEVNET_DOMAIN:-devnet.test}
+      NGINX_ENVSUBST_FILTER: ^DEVNET_
     volumes:
-      - ./https/nginx.conf:/etc/nginx/conf.d/default.conf:ro
+      - ./https/nginx.conf.template:/etc/nginx/templates/default.conf.template:ro
       - ./data/https/leaf.crt:/etc/nginx/https/leaf.crt:ro
       - ./data/https/leaf.key:/etc/nginx/https/leaf.key:ro
       # The app routes scripts/https-app.sh writes (a directory, so a rewrite shows up).
       - ./data/https/nginx:/etc/nginx/devnet:ro
     networks:
       atproto-devnet:
         aliases:
-          - alpha.devnet.test
-          - regular.devnet.test
-          - prod.devnet.test
+          - alpha.${DEVNET_DOMAIN:-devnet.test}
+          - regular.${DEVNET_DOMAIN:-devnet.test}
+          - prod.${DEVNET_DOMAIN:-devnet.test}
     healthcheck:
       test: ["CMD", "nginx", "-t", "-q"]
       interval: 5s

https/app-names.cjs

@@ -1,9 +1,9 @@
 // Preloaded into each PDS (NODE_OPTIONS=--require, docker-compose.https.yml), so a PDS can
 // reach an app at https://<name>.devnet.internal: it fetches a confidential client's
 // metadata and keys from there. Docker's DNS has no wildcard names, so every such name is
-// looked up as nginx instead, which routes it by name (https/nginx.conf). Only the address
-// lookup changes: the request still carries the app's name, and TLS still checks the
-// certificate for it against the devnet CA. Every other name resolves as usual.
+// looked up as nginx instead, which routes it by name (https/nginx.conf.template). Only
+// the address lookup changes: the request still carries the app's name, and TLS still
+// checks the certificate for it against the devnet CA. Every other name resolves as usual.
 'use strict';
 const dns = require('dns');
 

https/nginx.conf → https/nginx.conf.template

@@ -1,6 +1,8 @@
 # nginx for the https devnet (docker-compose.https.yml). One leaf certificate names
 # every host below; data/https/ca.crt is the CA that signed it (scripts/https-ca.sh).
-# Mounted as conf.d/default.conf, so this is the http block's whole server list.
+# A template: at start the image writes it to conf.d/default.conf with the DEVNET_DOMAIN
+# setting filled in, so this is the http block's whole server list. The comments below
+# name its default, devnet.test.
 
 # Docker's embedded DNS, re-asked every 10 s, so a recreated container's new
 # address is picked up without reloading nginx.
@@ -40,7 +42,7 @@ server {
 # *.devnet.test.
 server {
     listen 443 ssl;
-    server_name alpha.devnet.test *.devnet.test;
+    server_name alpha.${DEVNET_DOMAIN} *.${DEVNET_DOMAIN};
     location / {
         set $pds http://pds:3010;
         proxy_pass $pds;
@@ -49,7 +51,7 @@ server {
 
 server {
     listen 443 ssl;
-    server_name regular.devnet.test *.regular.devnet.test;
+    server_name regular.${DEVNET_DOMAIN} *.regular.${DEVNET_DOMAIN};
     location / {
         set $pds http://pds:3020;
         proxy_pass $pds;
@@ -58,7 +60,7 @@ server {
 
 server {
     listen 443 ssl;
-    server_name prod.devnet.test *.prod.devnet.test;
+    server_name prod.${DEVNET_DOMAIN} *.prod.${DEVNET_DOMAIN};
     location / {
         set $pds http://pds:3030;
         proxy_pass $pds;

scripts/https-ca.sh

@@ -3,7 +3,8 @@ set -eu
 
 # The https devnet's certificates: one local CA, made once, and one leaf that nginx
 # serves for every devnet name: the three PDSes and their handles, plc.directory, and
-# every app at <name>.devnet.internal.
+# every app at <name>.devnet.internal. The PDS names sit under DEVNET_DOMAIN
+# (devnet.test by default; scripts/https-lib.sh), and the names below use the default.
 #
 # Usage: ./scripts/https-ca.sh
 #
@@ -12,11 +13,11 @@ set -eu
 # devnet; only the leaf is reissued. Delete data/https/ca.* to start over, and then
 # trust the new ca.crt again everywhere the old one was trusted.
 #
-# The CA can only vouch for servers under devnet.test, devnet.internal and
-# plc.directory (name constraints, serverAuth only, no IP addresses), so trusting it
-# on a host cannot put any other site at risk, even if ca.key leaks from the pod.
-# A name outside those makes the leaf fail its own check below; widening the list
-# means a new CA, trusted again.
+# The CA can only vouch for servers under the domain it was made for, devnet.internal
+# and plc.directory (name constraints, serverAuth only, no IP addresses), so trusting
+# it on a host cannot put any other site at risk, even if ca.key leaks from the pod.
+# A name outside those makes the leaf fail its own check below; widening the list, or
+# a new DEVNET_DOMAIN, means a new CA, trusted again.
 #
 # A TLS wildcard covers one label, so *.devnet.test does not cover a handle on the
 # regular PDS (<name>.regular.devnet.test); each PDS's handle domain has its own.
@@ -25,11 +26,13 @@ set -eu
 # (space-separated), for instance when a PDS is added.
 
 SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+. "${SCRIPT_DIR}/https-lib.sh"
+D="${DEVNET_DOMAIN}"
 OUT="${SCRIPT_DIR}/../data/https"
 mkdir -p "${OUT}"
 cd "${OUT}"
 
-NAMES="alpha.devnet.test regular.devnet.test prod.devnet.test *.devnet.test *.regular.devnet.test *.prod.devnet.test plc.directory *.devnet.internal ${HTTPS_EXTRA_NAMES:-}"
+NAMES="alpha.${D} regular.${D} prod.${D} *.${D} *.regular.${D} *.prod.${D} plc.directory *.devnet.internal ${HTTPS_EXTRA_NAMES:-}"
 
 umask 077
 
@@ -47,7 +50,7 @@ else
     -addext "basicConstraints=critical,CA:TRUE,pathlen:0" \
     -addext "keyUsage=critical,keyCertSign,cRLSign" \
     -addext "extendedKeyUsage=serverAuth" \
-    -addext "nameConstraints=critical,permitted;DNS:devnet.test,permitted;DNS:devnet.internal,permitted;DNS:plc.directory,excluded;IP:0.0.0.0/0.0.0.0,excluded;IP:0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0"
+    -addext "nameConstraints=critical,permitted;DNS:${D},permitted;DNS:devnet.internal,permitted;DNS:plc.directory,excluded;IP:0.0.0.0/0.0.0.0,excluded;IP:0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0"
   chmod 644 ca.crt
   echo "Made a new CA: $(openssl x509 -in ca.crt -noout -subject)" >&2
 fi
@@ -68,19 +71,30 @@ authorityKeyIdentifier=keyid,issuer
 subjectAltName=${SAN}
 EOF
 
+# Issued beside the current leaf, which is replaced only once the new one checks out.
 openssl req -new -nodes -sha256 \
   -newkey ec -pkeyopt ec_paramgen_curve:P-256 \
-  -keyout leaf.key -out leaf.csr \
-  -subj "/O=atproto-devnet/CN=alpha.devnet.test"
+  -keyout leaf.key.new -out leaf.csr \
+  -subj "/O=atproto-devnet/CN=alpha.${D}"
 # 397 days: under the 398-day ceiling browsers apply to server certificates.
 openssl x509 -req -sha256 -days 397 \
   -in leaf.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
-  -extfile leaf.ext -out leaf.crt
+  -extfile leaf.ext -out leaf.crt.new
 rm -f leaf.csr leaf.ext
+
+if ! openssl verify -CAfile ca.crt leaf.crt.new >&2; then
+  rm -f leaf.crt.new leaf.key.new
+  echo "The CA cannot vouch for every leaf name, so the leaf is unchanged. If" >&2
+  echo "DEVNET_DOMAIN (${D}) changed since the CA was made, use a separate checkout, or" >&2
+  echo "delete data/https/ca.* to make a new CA for it (and trust that one again)." >&2
+  exit 1
+fi
+# Written in place, not renamed: nginx's single-file mounts follow the file, not the name.
+cat leaf.key.new > leaf.key
+cat leaf.crt.new > leaf.crt
+rm -f leaf.crt.new leaf.key.new
 chmod 644 leaf.crt
 # nginx reads the leaf key as root inside its container.
 chmod 600 leaf.key ca.key
-
-openssl verify -CAfile ca.crt leaf.crt >&2
 echo "Leaf names: $(openssl x509 -in leaf.crt -noout -ext subjectAltName | tail -n +2 | sed 's/^ *//')" >&2
 echo "Leaf valid until $(openssl x509 -in leaf.crt -noout -enddate | cut -d= -f2)" >&2