Skip to content

Prepare staged JSON/KYAML migration with pinned readiness evidence - #256

Merged
hyperpolymath merged 7 commits into
mainfrom
arena/6d824023-metamanifold-webui
Oct 23, 2026
Merged

hyperpolymath merged 7 commits into
mainfrom
arena/6d824023-metamanifold-webui

Conversation

@hyperpolymath

@hyperpolymath hyperpolymath commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Scope — first stage only, no premature migration

Start the requested sequence: working kyq → conformance corpus → shared adapters/structural gates → application migration → workflow conversion.

kyq is blocked, not certified. At pinned d448abd, its build/wasm/test/install recipes all exit 1 with “not implemented”. Compiler execution could not be reproduced locally: no OCaml/opam/dune, and the release-asset host is inaccessible. This is evidence about recipe fencing, not an independent compiler result. Upstream work is tracked in metadatastician/kyq#11 . Application and workflow conversion remain paused.

Changes

  • Pin kyq, AffineScript, ijson-jcs and probe tool identities in TOML.
  • Add a clean-source/pinned-version kyq recipe probe. It does not attempt global installation or label recipe success as parser conformance.
  • Add a 34-case raw-byte JSON corpus and executable reference runner. No Python JSON parser/canonicalizer or permissive fallback is introduced. Verify exact stdout and invalid-input exit codes; bound time/input/output and kill descendants; keep input-bearing diagnostics out of logs.
  • Add 21 harness/unit tests, including missing tools, timeout/output limits, binary stdin, child-process cleanup, malformed corpus, and always-success/always-invalid/passthrough/crashing adapter mutants.
  • Add a repository-wide lexical boundary inventory (locations only) and a human-reviewed coverage matrix. Search hits are candidates, not compliance verdicts or automatic exemptions.
  • Correct the existing migration document's inaccurate claim that successful statistics responses already use JCS.
  • Add a read-only preparation workflow which builds the real pinned ijson-jcs CLI, with locked dependencies and Rust 1.85.1, and exercises the corpus. Its summary explicitly leaves kyq and application-wide conformance unverified.

Validation

Local Python 3.11.2:

  • PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s scripts/tests -p 'test_format*.py' -v: 21 tests pass.
  • Actual pinned kyq source probe: expected exit 1 (build/wasm/test each fail); no install run by the probe.
  • Existing governance contract: 41 assertions pass; 9 weakening mutations rejected; 4 pipeline-exit cases, 3 optional-log cases and scanner-boundary regressions pass.
  • New workflow parses with unique mapping keys; embedded Bash syntax passes; git diff --check passes.

No Julia/R/Rust runtime result is claimed locally. At final head 2e57cae, the hosted pinned reference and Sonar gate both pass; broader CI still has running/queued checks. No application, scientific results, persisted IDs, lockfiles or licensing files are changed. No build artifacts/vendor source are committed. New documentation is explicitly unignored.

Follow-ups / limitations

The initial corpus characterizes one-pass compatibility, not complete RFC coverage or closure under all numeric policies. Julia/JS acceptance differences, KYAML comment/profile proofs, isolated install, real WASM IO, fuzz/large-data tests and runtime integration remain explicit obligations in docs/format-migration.md. These are not waived by a green preparation job.

Hosted reference and review update

The initial real Rust reference run 39999978128 built the pinned CLI and passed all 34 cases. The rerun 40000917320 also passed at 0b970ab, whose Sonar security gate passed.

Sonar findings were addressed by removing arbitrary corpus-path/program-argument CLI selection and complexity refactors: subprocess commands now come only from a closed table of complete literal protocols. Executables must be installed on trusted PATH; this is not an adversarial sandbox.

Both Codex findings are addressed: inventory scans embedded workflow YAML operations, and single-threaded Linux subreaper supervision adopts/kills/reaps setsid/double-fork descendants. Regression tests verify the detached PID disappears and unsupported supervision fails closed. The local suite now has 21 passing tests. The final-head proof is now complete for the preparation lane: run 40001674612, job 120579062625, passed all harness/build/corpus steps at 2e57cae; Sonar check 120579331209 also passed. Broader CI is still running/queued, and the PR is not merged. No application/runtime dialect migration or kyq readiness claim.

…orpus

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T18:44:01.522779Z be7b078 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 56 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: d08d7115-f4b9-4bdd-bf0b-66ac70e02593
📥 Commits

Reviewing files that changed from the base of the PR and between 6bb4054 and be7b078.

📒 Files selected for processing: 12
  • .github/workflows/format-readiness.yml
  • .gitignore
  • config/format-tools.toml
  • docs/format-migration.md
  • docs/ijson-jcs.md
  • scripts/formats/conformance.py
  • scripts/formats/descendants.py
  • scripts/formats/inventory.py
  • scripts/formats/probe_kyq.py
  • scripts/formats/process.py
  • scripts/tests/test_format_readiness.py
  • test/fixtures/formats/ijson-jcs.toml
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Fork governance

Lane Status Run
DEED syntax (repo deed) 🟢 conformant run 40005512149
K9 conformance (L0–L2; L3 not verified) green (job: success) same run
Hypatia verdict grid purple (job: success) same run

Hypatia grid purple

🟣 critical · commit be7b07870c6e · run 40005512149

Reporter Status Critical High Medium Minor Since previous revision
Hypatia scan 🟣 critical 51 57 26 0 ↔ unchanged
panic-attack assail 🔴 high 0 7 2 0 ↔ unchanged
pons asinorum 🟡 medium 0 0 4 0 ↔ unchanged
rhodibot (RSR compliance) 🟠 indeterminate 0 0 0 0 — indeterminate
panicbot (panic recovery, error handling) 🟠 indeterminate 0 0 0 0 — indeterminate
echidnabot (formal verification: proofs/) 🟠 indeterminate 0 0 0 0 — indeterminate
seambot (integration health: API routes) 🟠 indeterminate 0 0 0 0 — indeterminate
finishingbot (release readiness) 🟠 indeterminate 0 0 0 0 — indeterminate
accessibilitybot (WCAG on the React UI) 🟠 indeterminate 0 0 0 0 — indeterminate
glambot (presentation quality) 🟠 indeterminate 0 0 0 0 — indeterminate
cipherbot ⚪ not applicable 0 0 0 0 ↔ unchanged
sustainabot ⚪ not applicable 0 0 0 0 ↔ unchanged
Hypatia scan: 134 finding(s)
  • critical cicd_rules/banned_language_file — frontend/src/views/run/parseFilterYaml.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/encode.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/charts.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/export.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/figure.test.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/legend.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/types.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/utils/download.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/utils/text.test.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/utils/timeago.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/utils/text.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useUnsavedGuard.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useApi.test.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useApi.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useAnalysis.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useSSE.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useTabParam.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useNavRefresh.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useJobEvents.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/about.ts — TypeScript file detected -- banned language
  • 114 more; see the structured report in the artifact.
panic-attack assail: 9 finding(s)
  • high DynamicCodeExecution — src/pipeline/dada2/context.jl — eval/Meta.parse in src/pipeline/dada2/context.jl
  • high DynamicCodeExecution — src/analysis/analysis.jl — eval/Meta.parse in src/analysis/analysis.jl
  • high DynamicCodeExecution — src/analysis/differential.jl — eval/Meta.parse in src/analysis/differential.jl
  • high DynamicCodeExecution — src/core/provenance.jl — eval/Meta.parse in src/core/provenance.jl
  • high UnboundedLoop — scripts/formats/descendants.py — Unbounded while True loop in scripts/formats/descendants.py
  • high DynamicCodeExecution — scripts/check-julia-syntax.jl — eval/Meta.parse in scripts/check-julia-syntax.jl
  • high DynamicCodeExecution — test/unit/test_permanova.jl — eval/Meta.parse in test/unit/test_permanova.jl
  • medium UnsafeFFI — src/core/memory_probe.jl — 3 ccall/FFI calls in src/core/memory_probe.jl
  • medium UnsafeFFI — install.jl — 1 ccall/FFI calls in install.jl
pons asinorum: 4 finding(s)
  • medium swallowed-error — ./scripts/formats/process.py:127 — empty or pass-only exception handler (T0, HEURISTIC)
  • medium swallowed-error — ./scripts/formats/descendants.py:86 — empty or pass-only exception handler (T0, HEURISTIC)
  • medium swallowed-error — ./scripts/formats/descendants.py:55 — empty or pass-only exception handler (T0, HEURISTIC)
  • medium swallowed-error — ./scripts/formats/descendants.py:50 — empty or pass-only exception handler (T0, HEURISTIC)
rhodibot (RSR compliance): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
panicbot (panic recovery, error handling): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
echidnabot (formal verification: proofs/): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
seambot (integration health: API routes): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
finishingbot (release readiness): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
accessibilitybot (WCAG on the React UI): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
glambot (presentation quality): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
cipherbot: not_applicable — no cryptography is implemented in this repository
  • ⚪ not applicable: no cryptography is implemented in this repository
sustainabot: not_applicable — fleet adapter slot reserved; no adapter exists yet (gitbot-fleet README)
  • ⚪ not applicable: fleet adapter slot reserved; no adapter exists yet (gitbot-fleet README)

Scope: static findings reported by each row's tool at the named commit. Purple, red, yellow and blue grade the worst finding present; orange is evidence missing, malformed or a scanner that did not run; grey is an explicit scope exclusion with its reason. No row is a security certification. report.json, badge.json and the raw findings are in the verdict-grid artifact on the linked run.

K9 L0–L2 green

🟢 conformant · commit be7b07870c6e · run 40005512149 · validator hyperpolymath/standards@c6d78606f392

Category Status Pass Fail Indeterminate N/A Since previous revision
K9 contract typecheck 🟢 conformant 1 0 0 0 ↔ unchanged
K9 validator self-test 🟢 conformant 37 0 0 0 ↔ unchanged
K9 live corpus (L0–L2) 🟢 conformant 3 0 0 0 ↔ unchanged

Scope: K9 components L0–L2 against contract v1.0.0 at the named standards commit. L3 signature verification and routed coordination-dialect files are not certified. Green is limited to this scope. Orange is missing evidence; yellow is grandfathered debt or warnings; grey is an explicit scope exclusion. report.json, badge.json and the validator logs are in the k9-grid artifact on the linked run.

@hyperpolymath
hyperpolymath enabled auto-merge (squash) October 23, 2026 18:04
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f0a00cc491

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/formats/inventory.py Outdated
Comment on lines +70 to +71
if suffix not in CODE:
continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scan workflow YAML for embedded format operations

Workflow files are emitted as generic documents and then skipped because .yml/.yaml are absent from CODE, so embedded shell operations are never inventoried. For example, the checked repository contains numerous jq calls in .github/workflows/hypatia-scan-reusable.yml, but the inventory reports only that workflow's path rather than those JSON boundaries. Since this output is intended to drive boundary-by-boundary migration review, scan workflow YAML—or at least its run bodies—for the configured operation patterns.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: inventory now scans YAML/KYAML text for operation candidates as well as reporting document rows. Regression checks the exact embedded jq workflow location, and the real inventory now reports the hypatia-scan-reusable.yml jq boundaries. Still explicitly lexical candidate discovery, not structural YAML validation.

Comment thread scripts/formats/process.py Outdated
Comment on lines +93 to +95
try:
os.killpg(child.pid, signal.SIGKILL)
except ProcessLookupError:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Terminate descendants that leave the process group

When a probed tool starts a descendant that calls setsid() or otherwise moves to another process group, killpg(child.pid, ...) reaches only the original group; a detached process therefore survives a timeout even though this runner promises bounded descendant cleanup. This can leave runaway compiler or adapter processes consuming resources on the developer or CI host, so cleanup needs to track the full process tree or use an isolation mechanism that detached descendants cannot escape.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed: added single-threaded Linux subreaper supervision, so setsid/double-fork orphans are adopted, killed, and reaped rather than escaping group-only cleanup. Added a real detached double-fork regression that verifies the descendant PID no longer exists after timeout. Cleanup has a bounded grace period; unsupported hosts or existing-child/threaded contexts fail closed. Prior subreaper state is restored. This supervises trusted tools and is not advertised as an adversarial sandbox.

hyperpolymath and others added 4 commits October 23, 2026 18:08
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2e57cae83b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"Python JSON IO": r"\bjson\.(?:load|loads|dump|dumps)\s*\(",
"Oxygen JSON candidate": r"(?<![\w.])json\s*\(",
"R JSON IO": r"\b(?:jsonlite::)?(?:fromJSON|toJSON|read_json|write_json)\s*\(",
"shell jq/yq candidate": r"(?m)(?<![\w/-])(?:jq|yq)\s+(?:-[A-Za-z]|['\"])",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match long-option and bare jq invocations

The new YAML scan still misses valid jq boundaries unless the first argument is a short option or a quoted filter. In the current workflows, .github/workflows/hypatia-scan-reusable.yml:245, :347, and :469 use jq --arg ..., while .github/workflows/k9-conformance-reusable.yml:291 uses jq length ...; none appears in the generated inventory. This is valid syntax—jq --help documents jq [options] <jq filter> and lists --arg—so the migration census remains incomplete. Fresh evidence in the current tree is that adding YAML to CODE exposes the files but this new argument-shape restriction silently drops these operations.

Useful? React with 👍 / 👎.

@sonarqubecloud

Copy link
Copy Markdown

@hyperpolymath
hyperpolymath merged commit ccbf597 into main Oct 23, 2026
25 checks passed
@hyperpolymath
hyperpolymath deleted the arena/6d824023-metamanifold-webui branch October 23, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant