Skip to content

fix: make a VM update survive its first start - #209

Merged
catinspace-au merged 1 commit into
mainfrom
fix/updater-tops-up-secrets
Oct 6, 2026
Merged

catinspace-au merged 1 commit into
mainfrom
fix/updater-tops-up-secrets

Conversation

@catinspace-au

Copy link
Copy Markdown
Contributor

An update on an existing VM failed for two reasons.

  • The updater never ran make init, so an .env from an older version was missing the three secrets 2.2.0 generates and the self test refused them. It now runs make init first, which only tops up missing keys and never overwrites one.
  • On an empty volume the engine creates every ClickHouse table before /readyz answers. That took about 76s against an 80s healthcheck budget, so compose could give up seconds before the engine was ready. start_period goes from 30s to 180s. A warm start still reports healthy on its first passing probe.

Local: check-compose, check-hardfail, check-python and check-tests green (772 passed, two new tests pin the update command order).

Done when the 2.2.0 update on the dfe-docker VM passes make ci end to end.

An update on an existing VM failed for two reasons.

- The updater never ran `make init`, so an .env from an older version was missing the three secrets 2.2.0 generates (DFE_HUNT_RUNNER_CLICKHOUSE_PASSWORD, HYPERDX_EXPRESS_SESSION_SECRET, HYPERDX_TOKEN_ENCRYPTION_KEY) and the self test refused them. It now runs `make init` first, which only tops up missing keys and never overwrites one.
- On an empty volume the engine creates every ClickHouse table before /readyz answers. That took about 76s against an 80s healthcheck budget, so compose could give up seconds before the engine was ready. start_period goes from 30s to 180s. A warm start still reports healthy on its first passing probe.
@catinspace-au
catinspace-au merged commit 37a8817 into main Oct 6, 2026
7 checks passed
@catinspace-au
catinspace-au deleted the fix/updater-tops-up-secrets branch October 6, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant