Skip to content

ci: run tests against the Cosmos vNext emulator instead of a deployed account - #147

Merged
wemogy-admin merged 3 commits into
mainfrom
feat/cosmos-vnext-emulator
Aug 22, 2026
Merged

wemogy-admin merged 3 commits into
mainfrom
feat/cosmos-vnext-emulator

Conversation

@SebastianKuesters

Copy link
Copy Markdown
Member

Why

The test workflow ran the Cosmos integration tests against a deployed Cosmos account via secrets.COSMOS_CONNECTION_STRING, because the legacy emulator image (mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator) is amd64-only, needs 3 GB of memory and is unreliable.

To make that secret reachable from fork PRs the workflow used pull_request_target — which runs untrusted PR code with access to the secret. The in-file comment acknowledged this:

Only enable this if you trust that contributors cannot exfiltrate secrets through test code.

The Linux-based vNext emulator went GA on 2026-06-02. It is multi-arch (x64 + ARM64), boots in ~5s and covers everything this test suite needs, so the tests need no credentials at all.

Changes

env/cosmos/docker-compose.yaml

  • image: …:vnext-latest
  • PROTOCOL=https — the vNext emulator defaults to HTTP and the .NET SDK does not support HTTP mode
  • Ports 8081 (gateway), 8080 (health probe), 1234 (Data Explorer); the 10251-10254 direct-mode ports are gone since vNext is gateway-only
  • Removed mem_limit: 3g, cpus: 2.0 and the AZURE_COSMOS_EMULATOR_* vars — none apply to vNext
  • Added a healthcheck on /ready so up --wait blocks until the emulator is usable

env/cosmos/init-scripts/01-create-containers.csh (new)

  • Replaces the robinmanuelthiel/cosmos-db-setup:2.0.0 service, which was an amd64-only third-party image. vNext runs .csh scripts from /init through its built-in Cosmos Shell before accepting requests. Same topology as before: infrastructuredbtests with users on /tenantId and datacenters on /partitionKey.

env/mongo/docker-compose.yaml

  • Added a mongosh ping healthcheck so up --wait waits for a usable server rather than just a started container.

.github/workflows/test.yaml

  • Starts both compose stacks with --wait
  • Dropped COSMOS_CONNECTION_STRING / COSMOS_DATABASE_NAME — both test projects read their committed appsettings.json, which already points at the containers
  • pull_request_target → pull_request, and dropped the ref: …head.sha checkout override that went with it
  • Coverage comment is skipped for fork PRs, which only get a read-only GITHUB_TOKEN on pull_request
  • actions/checkout@v2 → @v5, consistent with the other workflows

.github/workflows/release.yaml

  • Dropped the cosmos-connection-string input — .github/actions/dotnet never declared or used it.

Verification

Local run on arm64, no COSMOS_* / MONGO_* env vars set, using the exact CI command (dotnet test -c Release /p:CollectCoverage=true -m:1):

Core.UnitTests        7 passed
Cosmos.UnitTests    283 passed, 1 skipped
InMemory.UnitTests  167 passed
Mongo.UnitTests      69 passed
─────────────────────────────────
                    526 passed, 1 skipped, 0 failed

The skip is pre-existing (GetLambdaExpression_ShouldBuildAComplexAnyFilter carries a Skip attribute).

Cold start of both stacks to healthy: 5.4s. Cosmos suite runs in 15s, including the three ETag tests — so real _etag and HTTP 412 optimistic-concurrency semantics work against the emulator.

Follow-up

The COSMOS_CONNECTION_STRING repository secret is now unused and can be deleted.

Note that env/mongo still uses the unpinned image: mongo tag — worth pinning separately for reproducible CI.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DPLcLmnYSJwFVB2b7bVbCK

… account

The legacy Cosmos emulator image is amd64-only, needs 3 GB of memory and is
unreliable enough that the test workflow ran against a deployed Cosmos account
via COSMOS_CONNECTION_STRING. That forced the workflow onto pull_request_target
so fork PRs could reach the secret, which means untrusted PR code executed with
access to it.

The Linux-based vNext emulator (GA 2026-06-02) is multi-arch, boots in ~5s and
covers everything the test suite needs, so the tests no longer need any
credentials:

- env/cosmos: switch to the vnext-latest image and configure it via PROTOCOL,
  ENABLE_INIT_DATA and a /ready healthcheck. The .NET SDK does not support the
  emulator's default HTTP mode, hence PROTOCOL=https.
- env/cosmos: replace the amd64-only robinmanuelthiel/cosmos-db-setup container
  with a Cosmos Shell init script, which the emulator runs before it accepts
  requests. It provisions the same database and containers as before.
- env/mongo: add a healthcheck so `up --wait` waits for a usable server.
- test.yaml: start both compose stacks with --wait, drop the
  COSMOS_CONNECTION_STRING and COSMOS_DATABASE_NAME env vars (the test projects
  read their appsettings.json, which already points at the containers) and move
  back from pull_request_target to pull_request.
- test.yaml: skip the coverage comment for fork PRs, which only get a read-only
  GITHUB_TOKEN on pull_request.
- release.yaml: drop the cosmos-connection-string input, which the dotnet
  action never declared or used.

The COSMOS_CONNECTION_STRING repository secret can now be deleted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPLcLmnYSJwFVB2b7bVbCK
- env/mongo: the healthcheck probed 127.0.0.1, but the mongo entrypoint runs a
  temporary mongod bound to loopback while it creates the root user and runs the
  init scripts. The probe went green against that one, so `up --wait` returned
  ~1.9s before the real server accepted connections and before the admin user
  and collections existed. Probing the routable address instead cannot reach the
  temporary server. Verified: an authenticated connection made immediately after
  `up --wait` failed before the fix and succeeds on 3/3 cold starts after it.
- test.yaml: Dependabot PRs come from branches in this repo, not forks, so the
  fork guard did not cover them, yet GitHub still forces a read-only
  GITHUB_TOKEN for them on pull_request. The coverage comment would have 403'd
  and failed every Dependabot PR, which this repo gets regularly.
- test.yaml: split the teardown into two steps. Both `down` calls shared one
  `run` block, so a non-zero exit from the Cosmos teardown would have skipped
  the Mongo one despite `if: always()`.
- test.yaml: drop the reference to the non-existent `release-version` step,
  which actionlint flagged. It resolved to an empty string and the composite
  action never used the input.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPLcLmnYSJwFVB2b7bVbCK
@SebastianKuesters

Copy link
Copy Markdown
Member Author

Self-review (/code-review) — 3 findings, all fixed in 0e92f16

1. MEDIUM — the Mongo healthcheck I added in this PR reported healthy against the wrong server.

The official mongo entrypoint starts a temporary mongod bound to 127.0.0.1 with auth stripped while it creates the root user and runs /docker-entrypoint-initdb.d. The healthcheck ran inside the container and mongosh defaults to 127.0.0.1:27017, so it went green against that temp instance — up --wait returned ~1.9s before the real server accepted connections, and before the admin user, users and datacenters existed. start_period does not help: Docker marks a container healthy on the first success, even during the start period.

That is exactly the race the healthcheck was added to prevent. Reproduced it directly — an authenticated connection issued immediately after up --wait:

before fix: FAILED
after fix:  OK on 3/3 cold starts

Fixed by probing the routable address (--host mongodb) instead of loopback; the temporary mongod binds loopback only, so it cannot be reached that way.

2. MEDIUM — Dependabot PRs would have failed on the coverage-comment step.

My fork guard (head.repo.full_name == github.repository) does not cover Dependabot, whose PRs come from dependabot/… branches in this repo. GitHub still forces a read-only GITHUB_TOKEN for Dependabot on pull_request, so sticky-pull-request-comment would 403 and fail the job. Under the old pull_request_target trigger these ran in base context with a writable token, so this was a regression introduced by this PR — and this repo gets Dependabot PRs regularly (#143–#146, #107). Added github.actor != 'dependabot[bot]'.

3. LOW — chained teardown partially defeated if: always().

Both down commands shared one run: block, which runs under bash -e. A non-zero exit from the Cosmos teardown would have skipped the Mongo one. Split into two steps, each with if: always().

Also addressed

test.yaml:28 referenced steps.release-version.outputs.next-version, but no such step exists in this workflow — actionlint flagged it as an error. It resolved to an empty string and .github/actions/dotnet never uses the input. Replaced with a literal. test.yaml is now actionlint-clean.

Re-verified after the fixes

Cold start of both stacks, no COSMOS_*/MONGO_* env vars, CI's exact command:

Core       7 passed
Cosmos   283 passed, 1 skipped
InMemory 167 passed
Mongo     69 passed
──────────────────────────────
         526 passed, 1 skipped, 0 failed

Knowingly left alone (pre-existing, out of scope)

  • Code Coverage Summary Report has no if: always(), so it is skipped whenever tests fail.
  • build.yaml:17 has the same dead release-version reference.
  • release.yaml has several stale action versions (setup-node@v2, action-gh-release@v1) that actionlint flags as too old to run.
  • env/mongo uses the unpinned image: mongo tag.

@github-actions

Copy link
Copy Markdown

Code Coverage

Package Line Rate Branch Rate Complexity
Wemogy.Infrastructure.Database.Core.UnitTests 94% 90% 255
Wemogy.Infrastructure.Database.Core 87% 76% 346
Wemogy.Infrastructure.Database.Mongo 83% 68% 64
Wemogy.Infrastructure.Database.InMemory 50% 63% 199
Wemogy.Infrastructure.Database.Cosmos 44% 35% 287
Summary 75% (3157 / 4222) 60% (446 / 740) 1151

marocchino/sticky-pull-request-comment ran with recreate: true, which deletes
the previous comment and posts a new one, so every run moved the coverage report
to the bottom of the PR and generated a fresh notification.

mshick/add-pr-comment defaults to allow-repeats: false and
refresh-message-position: false, so a stable message-id makes it edit the
existing comment in place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPLcLmnYSJwFVB2b7bVbCK
@github-actions

Copy link
Copy Markdown

Code Coverage

Package Line Rate Branch Rate Complexity
Wemogy.Infrastructure.Database.Core.UnitTests 94% 90% 255
Wemogy.Infrastructure.Database.Core 87% 76% 346
Wemogy.Infrastructure.Database.Mongo 83% 68% 64
Wemogy.Infrastructure.Database.InMemory 50% 63% 199
Wemogy.Infrastructure.Database.Cosmos 44% 35% 287
Summary 75% (3157 / 4222) 60% (446 / 740) 1151

@wemogy-admin
wemogy-admin merged commit c5ddeb7 into main Aug 22, 2026
4 of 5 checks passed
@wemogy-admin
wemogy-admin deleted the feat/cosmos-vnext-emulator branch August 22, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants