dln_c_loader_interpose: all-series patches + the spec-22 acceptance harness — DRAFT, chain-gated - #55
Draft
ronaldtse wants to merge 8 commits into
Draft
dln_c_loader_interpose: all-series patches + the spec-22 acceptance harness — DRAFT, chain-gated#55ronaldtse wants to merge 8 commits into
ronaldtse wants to merge 8 commits into
Conversation
…ce harness Spec 22 class L, ELF (linux gnu/musl): the runtime binary's own dlopen/dlerror definitions preempt process-wide (the interpreter, its C extensions, and any library they pull), so every native-library load -- dln_load, ffi, fiddle, C-extension self-loads -- routes a VFS-resident path through tebako_fs_dlmap2file (library + dependency closure to the exec cache) and loads the real file. Host paths pass through; covered-but-not-held answers ENOENT and the raw call serves it (the dln_load precedent). Materialization failures are named errors (library, mount, verdict) through the dlerror channel. dlsym is not interposed (real handles make it unnecessary; a main-binary dlsym cannot resolve its original on musl). macOS is inert in this TU: dyld applies __interpose tuples only from dylibs, never from the main executable (verified), so the macOS interposition is the driver's self-insertion at boot (crates/tebako-driver). Inert where USE_DLN_DLOPEN is undefined (windows). One body serves 3.2-4.0 (home 3.2); 3.1 carries its diverged context. The ci/spec22 harness is the local reproducible acceptance proof (jailed fiddle + C-ext self-dlopen probe).
The harness README's platform notes still described the pre-discovery design (an __interpose section in the runtime binary). dyld honors tuples only from dylib images (verified empirically, spec 22 'Phase 1 delivery'): the macOS interposition is the driver writing its embedded interpose dylib and re-execing with DYLD_INSERT_LIBRARIES; the ELF delivery is this patch family.
- prefer homebrew bison: stock macOS ships 2.3, ruby regenerates parse.c (needs >= 3.0) when a patch nudges the dependency graph — breaks 3.2.x builds - package layout: accept the sibling form (exe IS $RUNTIME_PKG, image $RUNTIME_PKG.tfs beside it) as well as the directory form - probe natives: canonical header root deps/src/_ruby_$VERSION; the blanket find can land on the deps/stash_* installed-headers copy (include/ruby-X/ruby.h — one level deeper), breaking the dirname arithmetic - derive the .ext/include arch dir instead of hardcoding arm64-darwin24
ruby compiles dln.c with -fvisibility=hidden (XCFLAGS); upstream symbols
get default visibility from the RUBY_SYMBOL_EXPORT pragmas in dln.h, but
the patch's new definitions carried no export marker, so they stayed
GLOBAL HIDDEN in dln.o and LOCAL in the linked exe. -Wl,-export-dynamic
cannot export a hidden symbol, so the exe-preempt design (spec 22 phase
1, class L, ELF delivery) never engaged: nm -D showed no dlopen/dlerror
export (gate exit 66 on the 4.0.6 linux-gnu build).
Annotate both definitions __attribute__((visibility("default"))) (the
3.1 copy and the 3.2 body shared by 3.2-4.0 via manifest refs) and
document the constraint in the patch comment. macOS is unaffected (its
delivery is the driver's self-insertion dylib, not this TU).
The linux/amd64 twin of run.sh: one docker container (ubuntu-20.04 factory CI image, linux/amd64 under qemu-user on Apple Silicon), one host-side driver (run-elf-leg.sh) — toolchain setup, source roll from this checkout, adapter-less gem repo staging, link unit build with the GNU_UNIQUE gate, factory runtime build with the nm -D dlopen/dlerror export gate, and the same four jailed assertions as the macOS leg. Proven green on 4.0.6 (SPEC22-ACCEPTANCE-OK). The container-local deviations are documented in elf/README.md and scripted where automatable: the distro libjemalloc.a neuter (qemu-user hangs miniruby otherwise; setup-toolchain.sh, idempotent) and the libtfs.a reseal recipe for gcc's GNU_UNIQUE template statics in the vendored rnp closure (gated in build-link-unit.sh; reseal.py + swap-members.py + ar-rebuild.py included; the durable fix is upstream arscope work, flagged in the README).
…2 §3 SSOT)
tfs_spawn_prepare now resolves the preload shim's in-VFS path from
TEBAKO_PRELOAD_SHIM (the driver's flow of the env image's layout.yaml
preload_shim grant) and falls back to the literal
/__tfs__/lib/tebako/libtfs_preload.{so,dylib} for images predating the
grant. The env var is authoritative; the literal keeps older runtime
images working through the transition.
Same transform applied to patches/{3.1,3.2,3.3,3.4,4.0} (byte-identical
hook across versions); hunk headers recomputed arithmetically, verified
by line recount. Apply-fixture suite: 110 examples, 0 failures.
…targets Factory run 31699651270 (macos arm64, darwin24) failed the shell-string class-E example with EMPTY output and killed the probe's diagnostic cc: dyld TERMINATES an Apple-restricted binary at exec when the inherited DYLD_INSERT_LIBRARIES names a foreign dylib. darwin23 stripped the variable instead (the behavior spec 22 §3.1 pinned), so under the driver's process-wide export every system()/backtick of a packaged interpreter killed /bin/sh on that host. The hook now drops the inherited variable per spawn whose target is restricted: any shell form (the shell is always a platform binary) and any target resolving into Apple's system binary dirs (/bin, /sbin, /usr/bin, /usr/sbin, /usr/libexec, /System), with an execvp-equivalent parent-side PATH walk for bare names. The nil env pair deletes the inherited entry (rb_execarg_parent_start1's NIL_P branch, st_delete — verified against ruby_4_0 process.c). A restricted target can never honor the insertion on any host, so the scrub loses nothing; a non-restricted host target (a third-party JRE) keeps the delivery — spec 22 §3.1's array form stays green. Patch verified: applies cleanly against ruby_4_0 process.c (patch --dry-run); the PATH-walk + prefix logic passes a 12-case standalone oracle (sh/cc resolve restricted, temurin's hostedtoolcache path and /usr/local stay delivered). All five version patches kept byte-identical.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Spec 22 phase 1 (loader interposition) — the ruby-side pieces:
patches/<series>/dln_c_loader_interpose.patchfor all five series (3.1, 3.2, 3.3, 3.4, 4.0 — 3.2's body is shared by 3.2–4.0 via manifest references): routesdln_open'sdlopenthroughtfs_dlopen_routeso payload C-extensions resolve against the VFS; a miss is a nameddlerror, never a segfault.ci/spec22/— the acceptance harness (run.sh+ fixtures + README): factory-builds a runtime locally from the patched source mirror and runs the jailed probe (fiddle / cext self-dlopen / named dlerror / jail EPERM).Evidence (macOS arm64)
tfs_dlopen_routepresent in every dln.c) — mirror + SHA256SUMS in the run artifacts.SPEC22-ACCEPTANCE-OK, 4 assertions each).Deviations (documented in
ci/spec22/README)dlsymis not interposed (dlopenonly).Status
DRAFT — chain-gated. Do not merge until the whole spec-22 chain is green — tebako #396, tebako-runtime #28, tebako-runtime-ruby #95, plus the ELF acceptance leg — and the owner calls the merge. The previous tebako release is still publishing; nothing lands on
mainbefore the full chain is validated end-to-end.