Repository navigation
ci: run the tests weekly on the Windows emulator - #45
Open
xperiandri wants to merge 1 commit into
Open
xperiandri wants to merge 1 commit into
xperiandri wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The workflow and scripts consistently implement the described nightly Windows integration-test lane without changing existing build behavior.
0 open findings
What changed in this PR
Adds nightly Windows Cosmos DB Emulator testing to catch platform-specific query differences.
Changes:
- Runs Debug and Release tests nightly and on demand.
- Adds required failure behavior to emulator install/start scripts.
- Preserves warning-only behavior for existing build jobs.
| File | Description |
|---|---|
.github/workflows/windows-emulator-nightly.yml |
Defines the Windows test lane. |
.github/scripts/windows/install-cosmos-emulator.ps1 |
Adds required installation mode. |
.github/scripts/windows/start-cosmos-emulator.ps1 |
Adds required startup mode. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
xperiandri
force-pushed
the
ci/windows-emulator-nightly
branch
from
October 10, 2026 01:20
7c78256 to
4bf2103
Compare
xperiandri
commented
Oct 10, 2026
xperiandri
force-pushed
the
ci/windows-emulator-nightly
branch
2 times, most recently
from
October 10, 2026 17:08
98acc26 to
97bafd4
Compare
xperiandri
changed the title
xperiandri
added a commit
that referenced
this pull request
Oct 10, 2026
The maintainer's reviews of 2026-10-10 changed several things the record describes, and it now says so where each belongs. The Hedgehog MSTest adapter (#43): the in-repository copy follows the coding guidelines of this repository, while the upstream pull request keeps upstream's style in a copy of its own, so a change of behaviour goes into both; an invocation that MSTest could not run stops the run instead of being shrunk; the cancellation token is checked after every invocation, whatever its outcome, which the automatic review of #50 asked for; Odd and Even stay inside their range; DateOnly and TimeOnly get generators, upstream through hedgehogqa/fsharp-hedgehog#488. The open question of the adapter's code style is answered, formatting and nullness checking included: the two projects are formatted with Fantomas and build with nullness checking on, and only the verbatim port keeps upstream's settings. Collections (section 4): the shared collection functions of #47 that return voption and struct tuples, tested with properties in #50, and the rule of one module per pipeline of #49. Build (section 5): why FAKE's DotNet.test cannot run the test applications, and fsprojects/FAKE#2903, which adds DotNet.testMTP. The spec model (sections 6 and 16): EquatableArray and the ImmutableArray patterns are #48 and the model itself #42; the validator returns an ImmutableArray of struct errors. Golden baselines (section 13): a literal type provider was evaluated on the question asked in #46 and rejected, with the reasons. Tests (section 22): test category attributes sorted by name, in #51 with an instruction scoped to those files, Pascal-case assertion functions and ImmutableArray for shared test data. Two more answers of the same day: the Windows emulator lane of #45 runs weekly instead of nightly (section 2), and the shared collection functions of #47 are in the namespace FSharp.Azure.Cosmos instead of the global one (section 4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
xperiandri
force-pushed
the
ci/windows-emulator-nightly
branch
2 times, most recently
from
October 10, 2026 20:54
97bafd4 to
fbe0c58
Compare
xperiandri
added a commit
that referenced
this pull request
Oct 10, 2026
The maintainer's reviews of 2026-10-10 changed several things the record describes, and it now says so where each belongs. The Hedgehog MSTest adapter (#43): the in-repository copy follows the coding guidelines of this repository, while the upstream pull request keeps upstream's style in a copy of its own, so a change of behaviour goes into both; an invocation that MSTest could not run stops the run instead of being shrunk; the cancellation token is checked after every invocation, whatever its outcome, which the automatic review of #50 asked for; Odd and Even stay inside their range; DateOnly and TimeOnly get generators, upstream through hedgehogqa/fsharp-hedgehog#488. The open question of the adapter's code style is answered, formatting and nullness checking included: the two projects are formatted with Fantomas and build with nullness checking on, and only the verbatim port keeps upstream's settings. Collections (section 4): the shared collection functions of #47 that return voption and struct tuples, tested with properties in #50, and the rule of one module per pipeline of #49. Build (section 5): why FAKE's DotNet.test cannot run the test applications, and fsprojects/FAKE#2903, which adds DotNet.testMTP. The spec model (sections 6 and 16): EquatableArray and the ImmutableArray patterns are #48 and the model itself #42; the validator returns an ImmutableArray of struct errors. Golden baselines (section 13): a literal type provider was evaluated on the question asked in #46 and rejected, with the reasons. Tests (section 22): test category attributes sorted by name, in #51 with an instruction scoped to those files, Pascal-case assertion functions and ImmutableArray for shared test data. Two more answers of the same day: the Windows emulator lane of #45 runs weekly instead of nightly (section 2), and the shared collection functions of #47 are in the namespace FSharp.Azure.Cosmos instead of the global one (section 4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
xperiandri
force-pushed
the
ci/windows-emulator-nightly
branch
from
October 11, 2026 00:41
fbe0c58 to
a49b481
Compare
xperiandri
added a commit
that referenced
this pull request
Oct 11, 2026
The maintainer's reviews of 2026-10-10 changed several things the record describes, and it now says so where each belongs. The Hedgehog MSTest adapter (#43): the in-repository copy follows the coding guidelines of this repository, while the upstream pull request keeps upstream's style in a copy of its own, so a change of behaviour goes into both; an invocation that MSTest could not run stops the run instead of being shrunk; the cancellation token is checked after every invocation, whatever its outcome, which the automatic review of #50 asked for; Odd and Even stay inside their range; DateOnly and TimeOnly get generators, upstream through hedgehogqa/fsharp-hedgehog#488. The open question of the adapter's code style is answered, formatting and nullness checking included: the two projects are formatted with Fantomas and build with nullness checking on, and only the verbatim port keeps upstream's settings. Collections (section 4): the shared collection functions of #47 that return voption and struct tuples, tested with properties in #50, and the rule of one module per pipeline of #49. Build (section 5): why FAKE's DotNet.test cannot run the test applications, and fsprojects/FAKE#2903, which adds DotNet.testMTP. The spec model (sections 6 and 16): EquatableArray and the ImmutableArray patterns are #48 and the model itself #42; the validator returns an ImmutableArray of struct errors. Golden baselines (section 13): a literal type provider was evaluated on the question asked in #46 and rejected, with the reasons. Tests (section 22): test category attributes sorted by name, in #51 with an instruction scoped to those files, Pascal-case assertion functions and ImmutableArray for shared test data. Two more answers of the same day: the Windows emulator lane of #45 runs weekly instead of nightly (section 2), and the shared collection functions of #47 are in the namespace FSharp.Azure.Cosmos instead of the global one (section 4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pull request workflow runs the integration tests on the Linux vNext emulator only. The two emulators answer some queries differently, and tests that state the answers of both (#44) would otherwise check the Windows half on developer machines alone. ADR 0001 plans a non-blocking Windows lane for this. It planned a nightly one; the review of this pull request found that too often, so the lane is weekly. windows-emulator-weekly runs ./build.cmd DotnetTest in Debug and Release against the Windows emulator early every Monday and on demand. It also runs for a pull request that changes the lane itself, so that a change to it is checked before it merges, and for no other pull request, so it holds none up. Its first run showed that the Windows emulator never became ready on the hosted runners. The start script passed /AllowNetworkAccess, which the emulator's documentation allows only together with /Key or /KeyFile, and merely warned when the emulator stayed silent. It now starts the emulator through its PowerShell module with Start-CosmosDbEmulator, as the documentation shows for GitHub Actions, and both Windows scripts fail instead of warning. The build-only Windows jobs of the pull request workflow never used the emulator, so they no longer install or start it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
xperiandri
added a commit
that referenced
this pull request
Oct 11, 2026
The maintainer's reviews of 2026-10-10 changed several things the record describes, and it now says so where each belongs. The Hedgehog MSTest adapter (#43): the in-repository copy follows the coding guidelines of this repository, while the upstream pull request keeps upstream's style in a copy of its own, so a change of behaviour goes into both; an invocation that MSTest could not run stops the run instead of being shrunk; the cancellation token is checked after every invocation, whatever its outcome, which the automatic review of #50 asked for; Odd and Even stay inside their range; DateOnly and TimeOnly get generators, upstream through hedgehogqa/fsharp-hedgehog#488. The open question of the adapter's code style is answered, formatting and nullness checking included: the two projects are formatted with Fantomas and build with nullness checking on, and only the verbatim port keeps upstream's settings. Collections (section 4): the shared collection functions of #47 that return voption and struct tuples, tested with properties in #50, and the rule of one module per pipeline of #49. Build (section 5): why FAKE's DotNet.test cannot run the test applications, and fsprojects/FAKE#2903, which adds DotNet.testMTP. The spec model (sections 6 and 16): EquatableArray and the ImmutableArray patterns are #48 and the model itself #42; the validator returns an ImmutableArray of struct errors. Golden baselines (section 13): a literal type provider was evaluated on the question asked in #46 and rejected, with the reasons. Tests (section 22): test category attributes sorted by name, in #51 with an instruction scoped to those files, Pascal-case assertion functions and ImmutableArray for shared test data. Two more answers of the same day: the Windows emulator lane of #45 runs weekly instead of nightly (section 2), and the shared collection functions of #47 are in the namespace FSharp.Azure.Cosmos instead of the global one (section 4). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
xperiandri
force-pushed
the
ci/windows-emulator-nightly
branch
from
October 11, 2026 11:03
a49b481 to
14e93ad
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Changes
CI runs the integration tests on one emulator only: the Linux "vNext" preview emulator, in the Linux job of every pull request. #44 showed that the two emulators answer a few queries differently, such as which letters a case-insensitive comparison treats as equal, and its tests state the expected answer of each. So far the Windows half of those expectations was checked only on developer machines. ADR 0001 plans a non-blocking Windows lane for this, and this pull request adds it. The plan said nightly; the review here found that too often, so the lane runs once a week.
.github/workflows/windows-emulator-weekly.ymlruns./build.cmd DotnetTest, a build plus every test, in Debug and Release against the Windows emulator. It runs every Monday at 02:23 UTC and on demand (Run workflow). It also runs for a pull request that changes the lane itself, so this pull request checks it before it merges. For any other pull request it does not run, so it never holds one up.What the first run found. The Windows emulator never became ready on GitHub's hosted runners, neither in this lane nor in the existing build workflow, whose Windows jobs only warned and went on, because they build without running tests. The start script passed
/AllowNetworkAccess, which the emulator's documentation allows only together with/Keyor/KeyFile. So:start-cosmos-emulator.ps1now starts the emulator through its PowerShell module (Import-Module …\PSModules\Microsoft.Azure.CosmosDB.Emulator, thenStart-CosmosDbEmulator), as Microsoft's documentation shows for GitHub Actions, without/AllowNetworkAccess, and fails unless the emulator then answers.install-cosmos-emulator.ps1fails as well when the emulator cannot be installed. The hosted runners come with it preinstalled.build.ymlno longer install or start the emulator. They never ran tests against it, and with a start that works they would wait minutes for it on every pull request.Types of changes
CI only; the package is unchanged.
Checklist
Further comments
COSMOS_EMULATOR_PARTITION_COUNT).main).🤖 Generated with Claude Code