Skip to content

bench: measure Python source mutations - #14

Merged
PHPCraftdream merged 5 commits into
mainfrom
perf-source-mutate09
Oct 27, 2026
Merged

PHPCraftdream merged 5 commits into
mainfrom
perf-source-mutate09

Conversation

@PHPCraftdream

@PHPCraftdream PHPCraftdream commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add native/pure mutation baselines and a same-runner pure base/candidate benchmark with alternating samples, exact-source checks, and independent semantic oracles.
  • For pure root insertion, shallow-copy the private expected root Object instead of recursively cloning unchanged siblings. Nested paths retain the existing deep-copy behavior; output is still independently reparsed and semantically verified.
  • Add regressions covering deeply nested untouched siblings, UTF-8/CRLF, and a sentinel proving root insertion avoids recursive cloning. Correct test oracles to preserve literal quote characters in Ktav inline values.

Validation and interpretation

Exact head: a6ac70bcf568fe2527c4ab7018a9df36ae6daf7a. The host CI and benchmark workflows passed 20/20 checks, including paired benchmark 40537622392, native/pure tests, Python 3.9, Ruff, mypy, Rust lint, and docs.

On this runner, the median pure root insert with an unchanged 512-element Array sibling changed from 19.040 to 18.802 ms (1.3% lower). The 512-key root insert changed from 37.573 to 37.927 ms (0.9% higher); unchanged controls varied by up to approximately 1.2%. A meaningful speedup is not established. Five same-runner pairs do not isolate the cost of each parser and verification stage; further profiling is needed before selecting a more substantial optimization.

No local product builds, tests, lint/typecheck, or benchmarks were run by the parent. No dependencies or versions changed; the pre-existing spec checkout change is excluded. This PR remains draft; no merge or release is requested.

🤖 Generated with Claude Code

PHPCraftdream and others added 5 commits October 27, 2026 16:10
Measure native and pure source edits in GitHub CI with exact output and independent semantic oracles.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copy the expected root mapping without recursively cloning unchanged siblings; retain independent verification and benchmark both implementations on the same runner.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Preserve exact-source assertions while expecting literal quote characters from Ktav inline values; fix Ruff import and line formatting.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@PHPCraftdream
PHPCraftdream marked this pull request as ready for review October 27, 2026 16:38
@PHPCraftdream
PHPCraftdream merged commit bcc94dc into main Oct 27, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant