Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,6 @@
# on all platforms check out with LF — no more CRLF/LF noise in
# `git status` from Windows clients.
* text=auto eol=lf

# Polydoc требует пустую строку в конце промежуточных source parts.
root-docs/**/body-*.md whitespace=-blank-at-eof
42 changes: 37 additions & 5 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,10 +116,41 @@ jobs:
- name: Install test tooling
run: python -m pip install --upgrade pip maturin pytest pytest-cov "tomli>=2.5.0; python_version < '3.11'"
# Lock + provenance check before the native build compiles anything.
# The core is an IMMUTABLE git pin (Cargo.toml rev 447a00bd...);
# the verification step below FAILS CLOSED on rev/version drift —
# it never skips unconditionally.
- name: Generate Cargo.lock (no --locked)
run: cargo generate-lockfile && cargo update -p ktav --precise 0.8.1
- name: Check Rust provenance
run: python scripts/check_rust_provenance.py
run: cargo generate-lockfile
- name: Verify ktav core pin (rev/version, fail closed)
shell: python
run: |
import re, sys
from pathlib import Path

PIN = "447a00bd012e8d3f347c7e1eb59c3f0d9a78332f"
toml = Path("Cargo.toml").read_text(encoding="utf-8")
m = re.search(r'rev\s*=\s*"([0-9a-f]{40})"', toml)
if m is None:
sys.exit("Cargo.toml: ktav dependency has no git rev pin")
if m.group(1) != PIN:
sys.exit(f"Cargo.toml rev {m.group(1)} != immutable pin {PIN}")
lock = Path("Cargo.lock").read_text(encoding="utf-8")
packages = lock.split("[[package]]")
core = [p for p in packages if re.search(r'^name = "ktav"$', p, re.M)]
if len(core) != 1:
sys.exit("Cargo.lock: expected exactly one ktav package")
entry = core[0]
sm = re.search(r'^source = "git\+https://github\.com/ktav-lang/rust\?rev=' + PIN + r'#([0-9a-f]{40})"$', entry, re.M)
if sm is None:
sys.exit("Cargo.lock: ktav is not locked to the pinned git source")
if sm.group(1) != PIN:
sys.exit(f"Cargo.lock rev {sm.group(1)} != immutable pin {PIN}")
vm = re.search(r'version = "([^"]+)"', entry)
if vm is None:
sys.exit("Cargo.lock: ktav entry has no version")
if vm.group(1) != "0.9.0":
sys.exit(f"Cargo.lock: expected ktav core 0.9.0, got {vm.group(1)}")
print(f"ktav core pinned: rev {PIN}, version {vm.group(1)}")
- name: Build extension (release)
run: maturin develop --release --locked
- name: Pytest (both backends; conftest enforces the native scenario)
Expand All @@ -128,7 +159,8 @@ jobs:
# Pure-Python backend, no native extension, on the support matrix
# edges (oldest and newest CPython, both CI-relevant OSes). Installs
# ONLY pytest — no maturin, no Rust toolchain — and runs the
# backend-parametrized conformance runner plus the tests/pure package
# backend-parametrized conformance runner plus the tests/pure and
# tests/edit packages
# with ktav._core guaranteed absent (tests/conftest.py fails the job if
# it is not). The native-only unit suites (tests/parse, tests/serialize,
# tests/roundtrip, tests/api, tests/spec) call the native-default
Expand Down Expand Up @@ -177,7 +209,7 @@ jobs:
- name: Install ONLY pytest + build (no maturin, no Rust)
run: python -m pip install --upgrade pip pytest build
- name: Pytest (conformance both-backends + pure package, no _core)
run: python -m pytest tests/test_conformance.py tests/pure -v
run: python -m pytest tests/test_conformance.py tests/pure tests/edit -v

# Packaged no-native scenario: STAGE only the package files
# (python/ktav/**) plus a staged pyproject.toml whose package-dir
Expand Down
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,27 @@ release: a complete pure-Python parser and generator, available as
native platform wheels. It installs with no Rust toolchain, compiler
or maturin and provides exactly the `ktav.pure` backend.

- `ktav.edit_value(source, path, replacement)` and
`ktav.edit_value_segments(source, segments, replacement)` — lossless
in-place source edits (source-edit 0.9). Only the bytes of the target
value (plus inline wrappers the layout rules force to unfold) change;
comments, key spelling, blank lines, BOM and CR/LF/CRLF survive, and
an equal replacement is a byte-identical no-op. New exception family
`KtavEditError` with `KtavInvalidPathError` (`body` = decimal UTF-8
byte offset into the path), `KtavNotFoundError`, and
`KtavUnsupportedError`; the same ten-field envelope, `error` is one
of `EditInvalidPath` / `EditNotFound` / `EditUnsupported`.
- Spec 0.9 corpus: the conformance runner now executes the
`versions/0.9/tests` fixtures (162 invalid — 72 new `diagnostics09`
EOF/malformed-UTF-8 diagnostics), validates `expected_location`
metadata (line, raw-byte `start`, exact `end` or `minimum_end`), and
feeds every `raw_bytes`-flagged fixture through the shared byte
boundary, so invalid UTF-8 raises the structured `InvalidUtf8`
envelope in both backends.
- `UnclosedCompound` errors now pin the opener: `line` and `line_text`
name the line that opened the innermost unclosed compound, and
`span` runs from that opener to end of input.

### Changed

- Upgraded to Rust core 0.8.1 and spec 0.8.1 (published as the spec
Expand Down
3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ name = "_core"
crate-type = ["cdylib"]

[dependencies]
ktav = ">=0.8.1, <0.9"
# Подготовка 0.9: immutable core pin с source-edit; пакет ещё не опубликован.
ktav = { git = "https://github.com/ktav-lang/rust", rev = "447a00bd012e8d3f347c7e1eb59c3f0d9a78332f" }
pyo3 = { version = "0.28", features = ["abi3-py39", "extension-module", "indexmap", "generate-import-lib"] }
indexmap = "2"
rustc-hash = "2"
Expand Down
37 changes: 37 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,43 @@ Pick by what you are holding: `emit_canonical` when you have an object,
built in between, `format` when you want the document's own comments and
grouping to survive.

> **Unreleased (0.9 preparation):** These editing APIs and updated diagnostics are not included in published 0.8.1 packages. Native backends require the pinned 0.9 core (C ABI 2 for C-FFI); pure backends remain standalone.

### Source edits — `edit_value` / `edit_value_segments`

```python
import ktav

out = ktav.edit_value("port: 8080\nhost: db\n", "port", 9090)
# "port: 9090\nhost: db\n"

out = ktav.edit_value_segments("a: {x: 1, y: 2}\n", ["a", "y"], {"n": [1]})
# "a: {x: 1, y: {n: [1]}}\n"
```

`edit_value` takes a textual path (`key1.key2.0.key4`): inside a
segment `\.` is a literal dot and `\\` a literal backslash; any other
escape raises `KtavInvalidPathError` whose `body` is the decimal UTF-8
byte offset of the problem. `edit_value_segments` takes
already-decoded literal segments with no extra unescaping — a numeric
segment is a string key inside an Object and a zero-based index inside
an Array. Only replacement of an existing value is supported; missing
keys, out-of-bounds indexes and descent through a scalar raise
`KtavNotFoundError`.

The edit is a source splice, not a parse→mutate→serialize of the whole
document: comments, key spelling, order, whitespace, BOM and
CR/LF/CRLF outside the effective change are preserved byte-for-byte,
dotted/inline/block forms are kept, and an equal replacement is a
byte-identical no-op — even for noncanonical spelling. Every
successful patch is re-parsed and semantically verified. When no
lossless patch exists (noncontiguous dotted/reopened targets, internal
comments under structural replacement, a `))` line inside a multiline
replacement) the call raises `KtavUnsupportedError` instead of
guessing. Long or multiline replacements unfold only the touched
inline wrappers; untouched nested branches stay inline. Both the
native and the pure backend implement the identical semantics.

Unlike the JavaScript bindings, `canonical_from_source` buys you no extra
numeric fidelity here — Python distinguishes `int` from `float`, so the
Ktav Integer/Float distinction survives `loads`, and
Expand Down
22 changes: 22 additions & 0 deletions docs/ru/CHANGELOG.ru.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,28 @@
нативными платформенными wheel-ами. Он ставится без Rust toolchain,
компилятора и maturin и предоставляет ровно backend `ktav.pure`.

- `ktav.edit_value(source, path, replacement)` и
`ktav.edit_value_segments(source, segments, replacement)` —
lossless-правка исходника по месту (source-edit 0.9). Меняются только
байты целевого значения (плюс inline-обёртки, которые требует
раскрыть layout-правило); комментарии, написание ключей, пустые
строки, BOM и CR/LF/CRLF сохраняются, а совпадающая замена —
byte-identical no-op. Новое семейство исключений `KtavEditError` с
`KtavInvalidPathError` (`body` = десятичный UTF-8 byte offset в
путь), `KtavNotFoundError` и `KtavUnsupportedError`; тот же
десятипольный конверт, `error` — один из `EditInvalidPath` /
`EditNotFound` / `EditUnsupported`.
- Корпус spec 0.9: conformance runner исполняет fixtures из
`versions/0.9/tests` (162 invalid — 72 новых диагностики
`diagnostics09` EOF/битый UTF-8), проверяет метаданные
`expected_location` (line, raw-byte `start`, точный `end` или
`minimum_end`) и прогоняет все `raw_bytes`-fixture через общий
byte-boundary, так что битый UTF-8 даёт структурированный конверт
`InvalidUtf8` в обоих backend-ах.
- Ошибки `UnclosedCompound` теперь указывают открыватель: `line` и
`line_text` называют строку, открывшую ближайший незакрытый
контейнер, а `span` тянется от него до конца ввода.

### Changed

- Обновление до Rust core 0.8.1 и spec 0.8.1 (опубликованы как каталог
Expand Down
37 changes: 37 additions & 0 deletions docs/ru/README.ru.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,43 @@ print(ktav.format(open("config.ktav").read()))
значение Python, `format` — когда нужно сохранить собственные
комментарии и группировку документа.

> **Ещё не выпущено (подготовка 0.9):** Эти API редактирования и обновлённая диагностика не входят в опубликованные пакеты 0.8.1. Native-backend требует закреплённое ядро 0.9 (C ABI 2 для C-FFI); pure-backend остаётся самостоятельным.

### Правка исходника — `edit_value` / `edit_value_segments`

```python
import ktav

out = ktav.edit_value("port: 8080\nhost: db\n", "port", 9090)
# "port: 9090\nhost: db\n"

out = ktav.edit_value_segments("a: {x: 1, y: 2}\n", ["a", "y"], {"n": [1]})
# "a: {x: 1, y: {n: [1]}}\n"
```

`edit_value` принимает текстовый путь (`key1.key2.0.key4`): внутри
сегмента `\.` — literal точка, `\\` — literal backslash; любой другой
escape бросает `KtavInvalidPathError`, у которого `body` — десятичный
UTF-8 byte offset проблемного места. `edit_value_segments` принимает
уже декодированные literal-сегменты без повторного unescape — числовой
сегмент это строковый ключ в Object и zero-based индекс в Array.
Поддерживается только замена существующего значения: отсутствующий
ключ, выход за границы массива и спуск сквозь скаляр дают
`KtavNotFoundError`.

Правка — это splice исходника, а не parse→mutate→serialize всего
документа: комментарии, написание ключей, порядок, пробелы, BOM и
CR/LF/CRLF вне эффективного изменения сохраняются побайтово,
dotted/inline/block-формы держатся, а совпадающая замена —
byte-identical no-op, даже при неканоничном написании. Каждый
успешный патч перепарсится и семантически проверяется. Когда lossless
патча не существует (noncontiguous dotted/reopen-цели, внутренние
комментарии при структурной замене, строка `))` внутри multiline
замены), вызов бросает `KtavUnsupportedError`, а не угадывает. Длинные
и многострочные замены раскрывают только затронутые inline-обёртки;
нетронутые ветки остаются inline. Native и pure backend реализуют
идентичную семантику.

В отличие от биндингов для JavaScript, `canonical_from_source` не даёт
здесь дополнительной численной точности: Python различает `int` и
`float`, поэтому различие Integer/Float из Ktav переживает `loads`, и
Expand Down
18 changes: 18 additions & 0 deletions docs/zh/CHANGELOG.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,24 @@ Rust 实现见 [`ktav-lang/rust`](https://github.com/ktav-lang/rust)。
它的安装不需要 Rust 工具链、编译器或 maturin,并且只提供
`ktav.pure` 这一个 backend。

- `ktav.edit_value(source, path, replacement)` 与
`ktav.edit_value_segments(source, segments, replacement)` ——
无损的原地源文本编辑(source-edit 0.9)。只有目标值的字节会改变
(加上布局规则要求展开的 inline 包装层);注释、键拼写、空行、
BOM 与 CR/LF/CRLF 全部保留,相同替换是字节级 no-op。新增异常家族
`KtavEditError`,含 `KtavInvalidPathError`(`body` = 路径参数的
UTF-8 字节偏移十进制串)、`KtavNotFoundError` 与
`KtavUnsupportedError`;信封仍是同样的十字段,`error` 取
`EditInvalidPath` / `EditNotFound` / `EditUnsupported` 之一。
- spec 0.9 语料:conformance 运行器现在执行 `versions/0.9/tests`
的 fixture(162 个 invalid —— 其中 72 个是新的 `diagnostics09`
EOF/坏 UTF-8 诊断),校验 `expected_location` 元数据(line、
原始字节 `start`、精确 `end` 或 `minimum_end`),并让所有
`raw_bytes` fixture 走共享的字节边界,因此两个 backend 中坏
UTF-8 都会抛出结构化的 `InvalidUtf8` 信封。
- `UnclosedCompound` 错误现在固定 opener:`line` 与 `line_text`
指向打开最内层未闭合容器的行,`span` 从该 opener 延伸到输入末尾。

### Changed

- 升级到 Rust core 0.8.1 与 spec 0.8.1(发布为规范仓库的
Expand Down
32 changes: 32 additions & 0 deletions docs/zh/README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,38 @@ print(ktav.format(open("config.ktav").read()))
构造出 Python 值时用 `canonical_from_source`,想让文档自身的注释与分组
存活下来时用 `format`。

> **尚未发布(0.9 准备版本):** 这些编辑 API 和更新后的诊断不包含在已发布的 0.8.1 包中。原生后端需要固定的 0.9 核心(C-FFI 使用 C ABI 2);纯实现仍完全独立。

### 源文本编辑 —— `edit_value` / `edit_value_segments`

```python
import ktav

out = ktav.edit_value("port: 8080\nhost: db\n", "port", 9090)
# "port: 9090\nhost: db\n"

out = ktav.edit_value_segments("a: {x: 1, y: 2}\n", ["a", "y"], {"n": [1]})
# "a: {x: 1, y: {n: [1]}}\n"
```

`edit_value` 接受文本路径(`key1.key2.0.key4`):段内 `\.` 表示
literal 点号,`\\` 表示 literal 反斜杠;任何其他转义都会抛出
`KtavInvalidPathError`,其 `body` 是问题位置的 UTF-8 字节偏移十进制串。
`edit_value_segments` 接受已解码的 literal 段,不再额外反转义 ——
数字段在 Object 中是字符串键,在 Array 中是零基索引。仅支持替换
既有值:缺失键、越界索引与穿过标量的下降都会抛出
`KtavNotFoundError`。

编辑是对源文本的 splice,而不是整个文档的
parse→mutate→serialize:有效修改之外的注释、键拼写、顺序、空白、
BOM 与 CR/LF/CRLF 按字节保留,dotted/inline/block 形式得以维持,
相同替换是字节级 no-op —— 即使拼写非规范。每个成功的补丁都会
重新解析并做语义校验。当不存在无损补丁时(noncontiguous 的
dotted/reopen 目标、结构替换下的内部注释、multiline 替换中的
`))` 行),调用抛出 `KtavUnsupportedError` 而不是猜测。长或多行
替换只展开被触及的 inline 包装层;未触及的嵌套分支保持 inline。
native 与 pure backend 实现完全一致的语义。

与 JavaScript 绑定不同,`canonical_from_source` 在这里不会带来额外的
数值精度 —— Python 区分 `int` 与 `float`,因此 Ktav 的 Integer/Float
之分在 `loads` 之后依然保留,并且 `emit_canonical(loads(src))` 产生的
Expand Down
Loading
Loading