Repository navigation
Commit ac68bba
fix(kyaml): escape U+2028 and U+2029 so keys stay parseable by yq (#250)
Follow-up to #248 (KYAML report output), found while drafting the
upstream yq issue about its KYAML encoder.
## Problem
go-yaml, the parser behind `yq` and standards'
`scripts/kyaml-format.sh`, treats a raw U+2028 (LS) or U+2029 (PS) as a
line break, which is YAML 1.1 behaviour. Our emitter already escaped NEL
(U+0085) but wrote LS and PS raw. A report key holding either one
therefore produced KYAML that `yq` could not read at all:
```
$ yq -p yaml -o json old.kyaml # key "x<LS>y" written raw
Error: ... go-yaml load error in parser (while parsing a flow mapping) ... did not find expected ',' or '}'
```
Values were unaffected, and `crate::yaml` (serde-saphyr, YAML 1.2) read
these files correctly before and after. In #248 I deliberately left
U+2028 raw because YAML 1.2 calls it printable. Testing keys showed that
was the wrong call for interop.
## Fix
`write_quoted` also escapes U+2028 and U+2029 as `
` and `
`. Those escapes are valid in YAML 1.1 and 1.2, so nothing that parsed
the old output loses anything.
## Validation
- `cargo fmt --check`, plus `cargo clippy --all-targets -D warnings`
with and without `--all-features`: clean.
- `cargo test --all-features`: all suites green (lib: 510 passed).
- **New test `line_breaks_in_keys_are_escaped`** covers NEL, LS and PS
in keys: exact output, and a round trip through `crate::yaml`. Run
against the previous escape set, it fails with the raw separator in the
output (`"x\u{2028}y"`).
- `escapes_non_printables_that_yq_writes_raw` is updated: it asserted LS
stays raw, and now asserts it is escaped.
- **End to end with real `yq` v4.54.1:** KYAML from this branch with
NEL, LS or PS in a key reads back intact (`yq -p yaml -o json`).
## One thing this PR can't fix
`kyaml-format.sh --check` still **refuses** these files: `REFUSED — the
KYAML rendering does not parse to the same data`. The gate re-renders
through `yq -o kyaml`, and yq's encoder writes the characters raw again.
That is the upstream yq bug:
- With a `yq` built from my proposed upstream patch, the same three
files pass (`all 3 file(s) are KYAML`).
- The upstream issue is drafted but **not filed yet**; filing it is the
owner's call.
Until yq is fixed, a report key containing NEL, LS or PS can't pass the
gate. Such keys are rare in practice, since report keys are field names
and file paths.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01PFaqv6iBXrt3W5Akd1Fq5k)_
Co-authored-by: Claude <noreply@anthropic.com>1 parent 2285a77 commit ac68bba
2 files changed
Lines changed: 37 additions & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
5 | 13 | | |
6 | 14 | | |
7 | 15 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
24 | 26 | | |
25 | 27 | | |
26 | 28 | | |
| |||
104 | 106 | | |
105 | 107 | | |
106 | 108 | | |
107 | | - | |
108 | | - | |
109 | | - | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
110 | 115 | | |
111 | 116 | | |
112 | 117 | | |
| |||
150 | 155 | | |
151 | 156 | | |
152 | 157 | | |
153 | | - | |
| 158 | + | |
154 | 159 | | |
155 | 160 | | |
156 | 161 | | |
157 | | - | |
| 162 | + | |
158 | 163 | | |
159 | 164 | | |
160 | 165 | | |
161 | 166 | | |
162 | 167 | | |
163 | 168 | | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
164 | 183 | | |
165 | 184 | | |
166 | 185 | | |
| |||
0 commit comments