test: add a Hedgehog adapter for MSTest property tests

[xperiandri]

Proposed Changes

#47 is merged. The adapter uses the collection functions that #47 added (Seq.tryHead and others returning a voption, Seq.zip returning struct tuples). The branch is rebased onto main with #47 in it, so this pull request shows the seven commits of the adapter alone.

The query translator planned in ADR 0001 (#34) is meant to be checked with property-based tests: the SQL printer against generated syntax trees, and generated predicates against both F# and the emulator. Hedgehog, the property-testing library the plan uses, ships adapters for xUnit and NUnit but not for MSTest, which this repository uses. Without one, a property has to be a [<TestMethod>] that calls Property.check in its body: MSTest sees one opaque test, a failure is a single exception with no way to replay it, and [<TestInitialize>], [<TestCleanup>], data rows and timeouts do not apply to the individual generated cases.

This pull request adds that adapter, tests/Hedgehog.MSTest, and its own suite, tests/Hedgehog.MSTest.Tests.

A property is a test that must hold for every input Hedgehog generates. When an input fails, Hedgehog shrinks it: it searches for the smallest input that still fails, the counterexample. With the adapter, a property is an ordinary test method whose parameters Hedgehog generates:

[<TestClass>]
type ReverseTests () =

    [<Property(200<tests>)>]
    member _.``reversing a list twice gives the list back`` (xs : int list) =
        Assert.AreEqual (xs, List.rev (List.rev xs), "reversing twice must give the list back")

[<Property>] derives from MSTest's TestMethodAttribute. MSTest invokes the method once for every generated case and every shrink step, each time with a new test class instance, its [<TestInitialize>] and [<TestCleanup>], and its own timeout. The whole run is reported as one test result. A failure shows:

• Hedgehog's report with the shrunk arguments;
• the original exception as the inner exception;
• the output of the failing invocation;
• a line such as [<Recheck("33_7178138842248401496_3704456211830240771_4-4-4")>], to add next to the method's [<Property>].

That string is the recheck data: the size, seed and shrink path of the counterexample. Added to the method, next to its [<Property>] so that settings such as its AutoGenConfig stay, it replays exactly that one case.

The adapter is a port of Hedgehog.NUnit and Hedgehog.Xunit from hedgehogqa/fsharp-hedgehog at the Hedgehog 2.0.4 release commit a469772. The first commit copies the nine framework-neutral files unchanged except for the namespace, so the port can be compared with upstream line by line. The adapter is also proposed upstream, as src/Hedgehog.MSTest in hedgehogqa/fsharp-hedgehog#487, where it has its own copy in upstream's style. Both projects here:

• follow the coding guidelines of this repository, its Fantomas formatting and nullness checking included (voption, struct tuples, arrays, the collection functions of refactor: return voption and struct tuples from the collection functions #47), applied in a commit of its own after the port;
• reference no project of this repository;
• carry an Apache-2.0 header in every copied file, which records every change made to it, and are listed in the new THIRD-PARTY-NOTICES.md.

Once upstream publishes the package, it replaces this copy.

The adapter keeps upstream's attribute surface:

• [<Property>] with its six constructor overloads and the AutoGenConfig, AutoGenConfigArgs, Tests, Shrinks and Size settings;
• class-level [<Properties>] and [<Recheck>];
• GenAttribute<'T> with its 18 built-in subclasses ([<Int>], [<Email>], …), which set the generator of one parameter;
• AutoGenConfig types: a type whose single static member returns Hedgehog's IAutoGenConfig, the generators to use per parameter type.

On top of that:

• a Seed setting, so a suite can fix the cases it generates;
• the values of a [<DataRow>] or [<DynamicData>] row are the leading arguments, and only the remaining parameters are generated;
• Assert.Inconclusive discards a case (its precondition does not hold), and a give-up (Hedgehog stopping after 100 discards) fails the test;
• settings come from the test class MSTest runs, so a property inherited from an abstract base gets each derived class's [<Properties>], and a derived class's settings win over its base's;
• a GenAttribute is found through the whole inheritance chain; upstream silently ignores one derived from a built-in attribute;
• a generic method, or a return type other than unit, Task or ValueTask, gives an error result before anything is generated;
• each shrink step is invoked once; Hedgehog 2.0.4 alone would invoke the final counterexample a second time;
• once the TestContext token is cancelled, by a [<Timeout>] (which applies to each invocation) or by the test run, the property stops without shrinking; the token is checked before every invocation and after it, whatever its outcome;
• an invocation that MSTest could not run as a test – one that ends with an outcome such as Error or NotFound instead of Passed, Failed, Inconclusive or Timeout, or an exception of InvokeAsync itself – stops the property without shrinking, and the property reports the outcome and the exception of that invocation instead of a counterexample;
• DateOnly and TimeOnly: Hedgehog 2.0.4 cannot generate these two types on its own (Gen.auto<DateOnly> passes arbitrary integers to the constructor and throws ArgumentOutOfRangeException). Generators, auto-generation in the core and the attributes [<DateOnly>] and [<TimeOnly>] for all three adapters are proposed upstream: Generate DateOnly and TimeOnly: Gen functions, auto-generation and attributes hedgehogqa/fsharp-hedgehog#488 and, for MSTest, Add Hedgehog.MSTest: Property attributes for MSTest 4 hedgehogqa/fsharp-hedgehog#487. Until a release of Hedgehog has them, this adapter brings both halves itself:
  • the two attributes, which bound the value of a parameter;
  • the two default generators of Generate DateOnly and TimeOnly: Gen functions, auto-generation and attributes hedgehogqa/fsharp-hedgehog#488, registered in the configuration that every property starts from. A parameter of one of the two types, or of a type that holds such a value, is generated without an attribute, like any other parameter. A configuration that a test names is merged over it as before, so a generator of the test wins.

The build gains a DotnetListTests target that runs before DotnetTest. MSTest 4 fails the discovery of a whole assembly when one test method has a signature it cannot run, and a [<Property>] returning bool compiles without a warning. Listing the tests reports that before the emulator run starts. The agent instructions now explain how to write property tests.

What the suite covers

149 tests:

• Upstream's F# xUnit and NUnit adapter tests, rewritten for MSTest:

  • generation and shrinking, recheck data and Size, shrink limits;
  • AutoGenConfig types and their error messages, AutoGenConfigArgs;
  • class-level settings, attribute subclasses, tuples, IDisposable arguments;
  • GenAttribute with the 18 built-in attributes.

  Cases MSTest 4 cannot discover (bool, Async or Property returns, for example) are left out.

• Every addition and departure above, each pinning the MSTest behaviour the adapter relies on:

  • a new instance, TestContext, [<TestInitialize>], [<TestCleanup>] and Dispose for every case;
  • data rows and inherited properties;
  • the caller information of all six constructors;
  • Task and ValueTask bodies;
  • Assert.Inconclusive and give-ups;
  • generic methods and unsupported return types;
  • seed reproducibility and a single invocation of the counterexample;
  • cancellation and per-invocation timeouts;
  • invocations that MSTest could not run;
  • [<DateOnly>] and [<TimeOnly>]: default and given ranges, both ends of a range reached;
  • DateOnly and TimeOnly without an attribute: as parameters and inside a record, how the values spread with the size of a case, with a configuration of the test that has a generator of its own and with one that has none. One test states that Hedgehog cannot generate the two types by itself; it fails when a release of Hedgehog can, and the adapter's own generators go then.

• Failure reporting through the real MSTest pipeline: a failing property reports its counterexample, recheck data and output in exactly one result, and [<Recheck>] replays the counterexample and passes once the bug is fixed.

Properties that fail by design never fail the suite. They run in one of two ways:

• through the adapter's entry point, with a harness that inspects the report;
• through a [<FailingProperty>] attribute that passes when MSTest's single result has the expected outcome and message.

Twenty behaviours of the adapter were each reverted on their own, and every one makes at least one test fail. So do seven one-line defects planted in the default generators of DateOnly and TimeOnly: a generator that is not registered, a property that starts from Hedgehog's own defaults, a range that does not grow with the size of a case or grows the wrong way, a time cut to whole seconds.

Verification

• Build: dotnet build FSharp.Azure.Cosmos.slnx, Debug and Release, gives 0 errors. The warnings are the same as on main: nullness warnings in existing test code, none from the adapter projects.
• Adapter suite: 149 of 149 passed in Debug and in Release. --list-tests discovers the tests of both test applications.
• Existing tests on the Windows emulator: 190 of 190 passed in Debug and in Release (this includes the tests of refactor: return voption and struct tuples from the collection functions #47).
• Fantomas: clean.
• Commits: every commit builds in Debug on its own.
• Listing: build.cmd DotnetListTests -s lists the tests of both test applications.
• Example: the one above was compiled against the built adapter.

Changes after the Copilot review

• Generic AutoGenConfig methods: a generic argument is now inferred from every place its parameter occurs, also nested ('a[]), and all places must agree. Upstream takes one argument type per parameter that is itself a generic parameter, which fails for 'a list or a repeated 'a.
• NonZeroInt skips a 0 at either end of its range ([0, 5], [-5, 0]) instead of generating from an empty part, and rejects [0, 0].
• A case that a cancelled token keeps from running now disposes its generated IDisposable arguments too.
• The recheck hint is a standalone [<Recheck("…")>] to add next to the [<Property>], so the property's own settings stay and the replay uses the same generators.
• The misspelled test names "overriden", "properites" and "arg get disposed" are corrected.

Each fix has a test (7 new tests); the disposal test fails without its fix.

Changes after the maintainer's review

The branch was rebuilt on top of #47; the review threads are answered one by one.

• voption instead of option for every setting that may be absent: the members of IPropertyAttribute, the fields of PropertyContext, the private constructors of both attributes, the generic arguments that AutoGenConfig infers.
• The collection functions of refactor: return voption and struct tuples from the collection functions #47 (Seq.tryHead, Seq.tryFind, Seq.tryExactlyOne, Seq.zip) replace FSharp.Core's and the two helpers that upstream's Prelude had for this.
• Arrays instead of lists for the generated arguments. Hedgehog 2.0.4 has Gen.sequence (sequences) and Gen.sequenceList (lists) but no Gen.sequenceArray, so the adapter's Prelude adds one over Gen.sequence. Both existing functions share one traversal, so values shrink as before: the pinned recheck data and the seed-42 expectations of the suite pass unchanged.
• Array and sequence expressions instead of Array.append and Seq.append, everywhere in the adapter.
• KnownTypes, one internal module with the types the adapter compares with (Task, ValueTask, Void, the generic definitions of ValueTask<'T>, Async<'T>, Result<'T, 'E>, IAutoGenConfig), compared through Type.(=).
• Functions instead of function values for withTests, withShrinks and withSeed.
• An invocation that MSTest could not run stops the property and gives it its outcome (Copilot's finding): before, an Error outcome was shrunk like a counterexample and reported as Failed. Three new tests fail when that handling is reverted.
• [<DateOnly>] and [<TimeOnly>] with six tests.
• [<Odd>] and [<Even>] stay inside their range. This is upstream's code, and the review of the upstream pull request flagged it: Odd(-1, 0) could give 1 and Even(1, 2) could give 0. Both now generate from the odd or even values inside the range and reject a range without one. Eight tests.

Changes after the automatic review of #50

#50 is built on this branch, so its Copilot review read these commits too. It found three things, all fixed here in the commits they belong to:

• Cancellation during a passing case. MSTest reports a test method that returns without observing its cancelled token as passed (probed with a cooperative [<Timeout>]). Such a case stopped the property only when a later case read the token, so a property failed when its first case overran the timeout and passed when its last one did. The token is now checked after every invocation, whatever its outcome. A new test with two rows cancels the token during the third and during the last of twenty passing cases; the second row fails without the fix. The upstream pull request has the same change.
• Two broken documentation comments. A slash had moved from the comment of AlphaNumStringAttribute to that of DateOnlyAttribute, so neither started with /// <summary> and the compiler escaped their tags without a warning. The generated XML has no escaped tag left.
• "member", not "property", in the error for an AutoGenConfig type without exactly one config member: a static method that takes AutoGenConfigArgs is accepted as well.

Formatting and nullness checking

Asked for after the second review round: the two projects no longer keep upstream's formatting and Nullable=disable. The branch is also rebased onto the current main.

• Fantomas: .fantomasignore no longer lists the two projects, and their 17 F# files are formatted with the settings of this repository.
• Nullness checking is on, and neither project gives a warning:
  • a data row and the arguments of an invocation or of a config method may hold null, so they are arrays of objnull;
  • a null array is checked for where it enters: the setters of AutoGenConfigArgs take it as no arguments (one new test), and the entry point takes the data row as MSTest hands it in;
  • nested exceptions and base types are matched on null instead of tested with isNull, and reflection lookups that cannot fail are asserted with nonNull;
  • a config method that returns no IAutoGenConfig is an error with a message of its own;
  • one scoped #nowarn "3261" remains, in a helper of its own (InternalLogic.invokerOf). MSTest declares the elements of the arguments of ITestMethod.InvokeAsync as non-null, although it passes ITestMethod.Arguments, whose elements it declares as nullable, to that method itself.
• The pipeline that builds the journal ends in Seq.toArray, after the rule of one module per pipeline that main has now.
• The collection functions of refactor: return voption and struct tuples from the collection functions #47 are in the namespace FSharp.Azure.Cosmos now (review of refactor: return voption and struct tuples from the collection functions #47), so the three adapter files that use them open it, with a comment that says what for.
• The first two commits stay as they were: the verbatim port and the attribute keep upstream's style, with .fantomasignore and Nullable=disable, so that the port can still be compared with upstream line by line. The commit that applies the conventions turns both on, and the suite is added formatted and with nullness checking.
• The copy in Add Hedgehog.MSTest: Property attributes for MSTest 4 hedgehogqa/fsharp-hedgehog#487 keeps upstream's style.

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

• Build and tests pass locally
• I have added tests that prove my fix is effective or that my feature works (if appropriate)
• I have added necessary documentation (if appropriate)

Further comments

• Nothing ships: both projects are test projects, so no package, public API or changelog entry changes.

• Framework layer: MSTest has no counterpart to NUnit's test builder or xUnit's discoverer, so this layer is a single attribute that overrides TestMethodAttribute.ExecuteAsync.

• MSTest behaviour the API does not promise. The adapter relies on:

  • the per-invocation lifecycle of ITestMethod.InvokeAsync;
  • the experimental TestContext.Current (warning FS0057, suppressed in one helper);
  • the per-invocation timeout;
  • the reflected type of an inherited test method;
  • the unpacking of data rows.

  Each is pinned by a test, so an MSTest update that changes one fails the suite.

• Departures from upstream: every one is recorded in the header of the changed file.

Changes after the maintainer's question about DateOnly and TimeOnly

The agent instructions said that a DateOnly or TimeOnly parameter needs an attribute or a generator of its own, and did not say that the missing generators are contributed upstream. Both are corrected, in a commit of its own (9736d79) and in the commit of the instructions:

• The adapter generates the two types without an attribute. It registers the two default generators of Generate DateOnly and TimeOnly: Gen functions, auto-generation and attributes hedgehogqa/fsharp-hedgehog#488 in the configuration that a property starts from (AutoGenConfig.propertyDefaults, internal): a date around 2000-01-01, further away the larger a case is, and a time of day anywhere in the day, later the larger a case is. The precedence does not change: an attribute on the parameter, then the configuration of the method, then that of the class, then this base.
• It is a stand-in with an end. The code names the upstream pull request, and a test fails as soon as Gen.auto generates the two types by itself, which is when the stand-in goes.
• The instructions and the documentation comments of the two attributes link Generate DateOnly and TimeOnly: Gen functions, auto-generation and attributes hedgehogqa/fsharp-hedgehog#488 and say what a parameter gets without an attribute. Gen.auto outside a property does not have the generators before a release of Hedgehog does.
• Six tests; seven planted defects, and each fails at least one test.

Changes after the automatic review of 2026-10-11

• The open declarations are in the order of the repository in the two files of the adapter and the four files of the suite that had another one: System, then Microsoft, then the other libraries, then the namespaces of this solution, of which Hedgehog.MSTest is one here.
• Seq.toArray for a sequence instead of Array.ofSeq, in the suite and in the Prelude of the adapter.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟡 Changes recommended

Generator edge cases, replay guidance, generic configuration inference, and cancellation cleanup contain correctness defects.

Review effort: Balanced
Findings: 4 · 1

Open (5)

• Infer generic arguments from nested and repeated parameter types · New
• Handle zero endpoints without creating empty ranges · New
• Dispose generated arguments when cancellation skips invocation · New
• Preserve Property settings in ready-to-paste recheck hints · New
• Correct misspelling of overridden in target names and tests · New

What changed in this PR

Adds a Hedgehog-to-MSTest adapter and test suite to support replayable property tests for planned quotation/query translation work.

Changes:

• Adds property generation, shrinking, replay, cancellation, and MSTest lifecycle integration.
• Adds comprehensive adapter tests and third-party attribution.
• Integrates test discovery into the build pipeline.

File	Description
.fantomasignore	Excludes upstream-ported code from formatting.
.github/​copilot-instructions.md	Documents property-test conventions.
Directory.Packages.props	Adds Hedgehog dependency version.
FSharp.Azure.Cosmos.slnf	Adds adapter projects to the filter.
FSharp.Azure.Cosmos.slnx	Adds projects and notices.
THIRD-PARTY-NOTICES.md	Records copied code and license.
build/​build.fs	Adds test-discovery target.
tests/​Directory.Build.props	Excludes adapter projects from Cosmos infrastructure.
tests/​Hedgehog.MSTest/​AutoGenConfig.fs	Instantiates generator configurations.
tests/​Hedgehog.MSTest/​GenAttribute.Prelude.fs	Provides built-in generator attributes.
tests/​Hedgehog.MSTest/​GenAttribute.fs	Defines generator attribute abstractions.
tests/​Hedgehog.MSTest/​Hedgehog.MSTest.fsproj	Defines the adapter library.
tests/​Hedgehog.MSTest/​InternalLogic.fs	Implements generation, invocation, and reporting.
tests/​Hedgehog.MSTest/​IPropertyAttribute.fs	Defines shared property settings.
tests/​Hedgehog.MSTest/​Prelude.fs	Adds internal helpers.
tests/​Hedgehog.MSTest/​PropertiesAttribute.fs	Adds class-level defaults.
tests/​Hedgehog.MSTest/​PropertyAttribute.fs	Integrates Hedgehog with MSTest.
tests/​Hedgehog.MSTest/​PropertyContext.fs	Resolves effective property settings.
tests/​Hedgehog.MSTest/​RecheckAttribute.fs	Adds counterexample replay metadata.
tests/​Hedgehog.MSTest/​ReflectionHelpers.fs	Adds return-type inspection helpers.
tests/​Hedgehog.MSTest.Tests/​Common.fs	Defines shared test generators.
tests/​Hedgehog.MSTest.Tests/​GenAttributePreludeTests.fs	Tests built-in generator attributes.
tests/​Hedgehog.MSTest.Tests/​Harness.fs	Provides an adapter test harness.
tests/​Hedgehog.MSTest.Tests/​Hedgehog.MSTest.Tests.fsproj	Defines the adapter test application.
tests/​Hedgehog.MSTest.Tests/​MSTestTests.fs	Tests MSTest-specific behavior.
tests/​Hedgehog.MSTest.Tests/​PropertyTests.fs	Tests generation, shrinking, and settings.
tests/​Hedgehog.MSTest.Tests/​testconfig.json	Enables method-level parallel testing.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Invocation errors are currently shrunk and converted from Error to Failed, obscuring infrastructure failures.

1 open finding

• Preserve infrastructure errors instead of treating them as falsified cases · New

5 resolved since last review

• Preserve Property settings in ready-to-paste recheck hints
• Dispose generated arguments when cancellation skips invocation
• Handle zero endpoints without creating empty ranges
• Infer generic arguments from nested and repeated parameter types
• Correct misspelling of overridden in target names and tests

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The version-sensitive custom MSTest execution layer warrants final human validation.

7 open findings

• Move Microsoft namespace after System · New
• Move Microsoft namespace after System · New
• Move Microsoft namespace after System · New
• Move Microsoft namespace after System · New
• Use Seq.toArray for pipeline materialization · New
• Reorder open block to match repository convention · New
• Reorder open declarations by namespace group · New

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

The large adapter relies on implementation-sensitive and experimental MSTest lifecycle behavior, warranting final human review.

0 open findings

7 resolved since last review

• Reorder open declarations by namespace group
• Reorder open block to match repository convention
• Use Seq.toArray for pipeline materialization
• Move Microsoft namespace after System
• Move Microsoft namespace after System
• Move Microsoft namespace after System
• Move Microsoft namespace after System

🧠 Review effort: Balanced