[moloch--]

Summary

Adds a pinned Rust 1.99 toolchain using this LLVM 23.1.3 fork, with an opt-in seven-pass pipeline, Cargo package/target selection, and native compiler/sysroot packages for macOS ARM64, Linux ARM64/AMD64, and Windows AMD64. The shared Clang transforms, selection reports, and delivery tooling now cover the correctness and verification gaps found during the audit.

Audit fixes

• Preserve string storage and startup semantics: retain explicit sections, early-constructor inputs, and weak/COMDAT storage; keep decoder accesses volatile so optimization cannot restore plaintext. Skip naked functions in bogus-control-flow transformation, with explicit diagnostics.
• Add Clang build-wide selection manifests across parallel translation units, with raw/demangled symbols, effect/skip reasons, strict unmatched/untransformed checks, and failed/interrupted invocation detection.
• Enforce Cargo effects for each selected target and code-producing invocation. Preserve arguments after Cargo's --, isolate compiler-event routing, and use the matching rustdoc. Deliver libtest and rustdoc so unit, integration, and documentation tests work from extracted packages.
• Bind bootstrap, wrapper, and packaged outputs to verified source content, LLVM/compiler inputs, and output hashes. Reject edited source archives and stale or mixed binaries; isolate wrapper caches by actual build inputs.
• Prune excluded Rust source trees before traversal, preventing cyclic Windows directory junctions from hanging stage inventory. Cover the real native junction case and distinguish eligible private strings from deliberately retained Windows COMDAT literals.
• Measure the baseline, seven individual passes, and combined pipeline using final-binary witnesses, runtime/compile samples, memory, and section sizes. Fix section parsing so adjacent objdump rows cannot be merged, including the previously omitted ELF .interp row. Add O0/O2 static string/constant recovery experiments.
• Run native qualification on PRs and development pushes, gate tag publication on extracted-artifact checks, and update usage, delivery, measurement, and roadmap documentation.

Verification

Verified head: f1287d9cf94ea4a1d9426dbfde53296e71bb7293.

Current PR workflows both completed successfully at this head:

• Rust toolchain and extracted artifacts — all four native hosts
• Clang transform smoke tests — Linux x86_64

Coverage includes 16 portable regression suites plus four section-parser tests; LLVM, Cargo launcher, delivery/provenance, and machine-code extraction unit tests; Cargo unit/integration/doctests and output types; strict selection and forwarded arguments; Rust EH/panic, LTO, and seeded parallel-codegen checks; live HTTPS; and final-code/data witnesses. Linux also runs the four-language HTTPS suite. The full local extracted-toolchain gate passed at the same head.

Independently downloaded and verified the four archives from the earlier successful qualification run at the same head:

Native platform	Packaged files verified
macOS ARM64	93
Linux ARM64	89
Linux AMD64	89
Windows AMD64	83
Total	354

Archive checksums, file inventories, source/component provenance, exact revision, clean-source state, and full/HTTPS/benchmark smoke flags matched. All hosts produced nine Clang measurement variants with seven individual-pass witnesses and the same four-million-iteration checksum; O0/O2 recovery experiments checked six inputs each. Windows passed both the cyclic-junction regression and corrected O0/O2 native string-safety checks.

Verification boundaries: the optional Windows Clang global-access LTO runtime subtest skipped because the baseline linker could not consume that bitcode; Rust local ThinLTO checks passed. Platform-specific unit-test skips remain explicit. Hosted packages disable optional LLVM libraries; the separate local macOS build uses Homebrew zstd, so its smoke result alone is not clean-machine portability evidence. Performance and recovery measurements describe the tested fixtures.