Skip to content

Convert name selectors and string literals to the string they denote (RFC 9535 2.3.1.2) - #112

Merged
besok merged 1 commit into
besok:mainfrom
gaoflow:unescape-name-selectors
Jul 27, 2026
Merged

Convert name selectors and string literals to the string they denote (RFC 9535 2.3.1.2)#112
besok merged 1 commit into
besok:mainfrom
gaoflow:unescape-name-selectors

Conversation

@gaoflow

@gaoflow gaoflow commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

$["\t"] finds nothing in {"\t": "A"} while a raw tab in the selector finds it, because the parser keeps the source token of a name selector verbatim and leaves the decoding to whoever consumes it — RFC 9535, section 2.3.1.2 says the string must be converted to a member name by removing the quotes and replacing each escape sequence with the character it names. Six places downstream were each doing part of that job (normalize_json_key, the quote trimming in Value::get, prepare_regex, convert_js_path, parse_deletion_path and Pointer::key), which is also why $['\''] today returns the member named \ rather than the one named ', and why $["a"] reports its normalized path as $['"a"']. This decodes once in the parser and removes all six; \uXXXX now also accepts lower-case hex (section 2.3.1.1: the hexadecimal digits "can be either lowercase or uppercase"), surrogate pairs are joined, and normalized paths follow section 2.7 while reference/delete_by_path escape their JSON Pointers per RFC 6901.

Against the suite in rfc9535/ with filtered_cases.json emptied, failures go from 37 to 13, and the 24 that now pass are exactly the entries I removed from that file; the remaining 11 skips and 2 failures are #86, #87, #91 and the descendant-ordering case, all untouched. That is also why these survived to 1.0.6: the run recorded in results.csv as 703; 666; 2 on 2026-07-14 cannot happen in CI, since the CTS is a submodule no job checks out and rfc9535 is a binary crate that cargo nextest run never executes. I left CI alone — arming it would turn the remaining 13 red, and that is your call.

cargo test --all-features goes from 101 to 106 passing, plus cargo fmt --check, cargo clippy --workspace --all-targets --all-features -D warnings and cargo doc clean. Five existing tests asserted the old behaviour and are updated: tab_key and carr_return used documents keyed "\\t" and "\\r" (a backslash followed by a letter) where the CTS uses a real tab and CR, literal_test expected hel\'lo for 'hel\'lo', and name_sel/single_quote asserted the old un-normalized paths. I also checked the fix is not over-eager: re-adding the \\ collapse in prepare_regex breaks two CTS cases that pass on main, and escaping non-ASCII in normalized paths breaks your escaped_up_hex and surr_pairs.

Closes #88, closes #89, closes #90.

RFC 9535, section 2.3.1.2 converts a name-selector string to a member
name by removing the quotes and replacing each escape sequence with the
character it names. The parser kept the raw source token instead and left
the decoding to whoever consumed it, so six places downstream each did
part of the job: normalize_json_key, the quote trimming in Value::get,
prepare_regex, convert_js_path, parse_deletion_path and Pointer::key.
Decode once in the parser and delete all six.

Also accept lower-case hex in \uXXXX, join surrogate pairs, build
Normalized Paths per section 2.7 and JSON Pointers per RFC 6901.

Against the compliance test suite with filtered_cases.json emptied,
failures go from 37 to 13; the 24 that now pass are removed from that
file.
@besok

besok commented Jul 27, 2026

Copy link
Copy Markdown
Owner

thank you for the pr. It looks good

@besok besok left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good.

@besok
besok merged commit 37869de into besok:main Jul 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants