Skip to content

Cover unpaired surrogate escapes in the carrier corpus #65

Description

@kikashy

Goal

Add carrier-conformance cases for JSON string escapes that do not form a valid surrogate pair.

Why

The corpus mentions surrogates nowhere: grep -c surrogate conformance/manifest.json returns 0.

The Go reference runtime refuses them, and deliberately — internal/carrier/decode_test.go carries a twelve-case table for lone high surrogates, lone low surrogates, reversed pairs, one inside an array, one in a member name, and the valid pair that must still be accepted. None of that is in the corpus, so a second implementation has no case telling it this is required.

§2.1 requires rejecting malformed input rather than processing a silent prefix. An unpaired \uD800 is malformed input that many JSON parsers will happily turn into a replacement character, which is exactly the "silent prefix" shape the rule exists to prevent.

Scope

  • Add fixtures under conformance/carrier/ for at least:
    • a lone high surrogate escape in a string value;
    • a lone low surrogate escape;
    • a reversed pair (low followed by high);
    • a valid surrogate pair, which MUST be accepted.
  • Add matching entries to conformance/manifest.json near the other carrier cases, with layer: carrier, the appropriate expectedResult, and the diagnostic code and path the repository's own validator produces.
  • Update the document-corpus count statements — a harness test derives them, so it will tell you which.
  • Do not change the schema, the validator, normative text, or existing cases.

Acceptance criteria

  • Each listed escape has a fixture and a manifest entry.
  • The valid pair is present and expected valid, so the cases distinguish "unpaired" from "surrogate".
  • Expected diagnostics match what the repository's validator actually reports; run the suite to find out rather than guessing.
  • python -m unittest discover -s tests -v passes.
  • Every commit is signed off (git commit -s).

Contributor learning

The carrier layer, why an escape that decodes to a replacement character is a conformance question, and how a corpus case tells another implementation what is required.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions