Repository navigation
Conversation
The default spec path was `Path("spec/v0.1.0/memory-cell.schema.json")`, relative
to the working directory. Run the suite the way it is meant to be run - against
somebody else's server, from wherever they happen to be standing:
amp-conformance --base-url https://their-server.example.com
FAIL schema: schema-spec-file -- spec not found at spec\v0.1.0\...; pass --spec
That is the suite failing to find its own fixture, reported in the same red as a
server failing a vector. A third party reading it sees a failure about a file they
have never heard of, naming neither their server nor their mistake. The goal of
this suite is to be runnable by implementations that are not this one, and a
directory-dependent fixture works against exactly that.
- The normative schema now ships inside the wheel (a hatch force-include from
`spec/v0.1.0/`, so the source of truth stays in one place and the copy is
generated, not duplicated in the tree).
- The default is resolved by searching the working directory, then each of its
parents, then the packaged copy. Running from a checkout subdirectory now works
too - that case was silently failing as well.
- When no schema can be found, the schema vectors report `skip` with the flag to
pass, and the summary counts them separately. `--only schema` exits non-zero,
because asking for those vectors by name and getting a skip that exits 0 is the
exact failure mode this suite exists to catch in other people's servers.
- `--spec` is unchanged and still wins when given.
Two existing tests pinned the old behaviour; both are rewritten to assert the new
one rather than deleted - one of them existed precisely to check that a missing
spec was visible, and that requirement survives as "a skip is counted and named".
Verified: 19 conformance tests pass, the suite run from `conformance/` now reports
14 schema vectors passing where it previously reported a failure, and the built
wheel contains `amp_conformance/spec/memory-cell.schema.json`.
Owner
Author
|
Landed on master in the v0.1.0 chain: the branch was fast-forward merged as part of |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(conformance): carry the schema the suite judges against
The default spec path was
Path("spec/v0.1.0/memory-cell.schema.json"), relativeto the working directory. Run the suite the way it is meant to be run - against
somebody else's server, from wherever they happen to be standing:
That is the suite failing to find its own fixture, reported in the same red as a
server failing a vector. A third party reading it sees a failure about a file they
have never heard of, naming neither their server nor their mistake. The goal of
this suite is to be runnable by implementations that are not this one, and a
directory-dependent fixture works against exactly that.
spec/v0.1.0/, so the source of truth stays in one place and the copy isgenerated, not duplicated in the tree).
parents, then the packaged copy. Running from a checkout subdirectory now works
too - that case was silently failing as well.
skipwith the flag topass, and the summary counts them separately.
--only schemaexits non-zero,because asking for those vectors by name and getting a skip that exits 0 is the
exact failure mode this suite exists to catch in other people's servers.
--specis unchanged and still wins when given.Two existing tests pinned the old behaviour; both are rewritten to assert the new
one rather than deleted - one of them existed precisely to check that a missing
spec was visible, and that requirement survives as "a skip is counted and named".
Verified: 19 conformance tests pass, the suite run from
conformance/now reports14 schema vectors passing where it previously reported a failure, and the built
wheel contains
amp_conformance/spec/memory-cell.schema.json.