Skip to content

Honour the @version pin in jitagent get and inspect - #10

Open
Birfy wants to merge 3 commits into
mainfrom
fix/version-pin-honoured
Open

Birfy wants to merge 3 commits into
mainfrom
fix/version-pin-honoured

Conversation

@Birfy

@Birfy Birfy commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

Fixes #9

The defect

call rank@v1 runs v1. get rank@v1 printed v2's source, inspect rank@v1 --code printed v2's source, and get rank@v9 (a version that does not exist) printed v2's source with exit 0. The read path dropped the pin; nothing warned.

That breaks the property runtime.py puts first — never hand back something that looks fine but is wrong — in the one command a person uses to audit what will run.

Cause

get_code() → Registry.get(ref) → split_ref() (which parses the version) but get() returns only the Function; get_code() then took fn.best(). Runtime.call_many is the only place that re-applied the version, which is why call was always right.

Change

  • src/jitagent/jit.py: one resolver, resolve_version(ref) -> (function, version), and NoSuchVersion. get_code() goes through it and honours a pin.
  • src/jitagent/cli.py: get and inspect both use it. A pinned version that does not exist exits 1, prints no source, and lists the versions that do exist; inspect --code prints the pinned version's source and marks it *.
  • No pin: both commands behave exactly as before.

Risk pass (jitagent Security)

PASS-with-notes. Two notes, both addressed in 367229f:

  • NoSuchVersion inherited KeyError, so print(e) rendered the message as a quoted repr ("'rank has no version v9'"). It is now a ValueError, and the test asserts the exact line instead of a substring that the quoting slipped through.
  • the version string went to stderr raw, so an ESC byte in a ref reached the terminal (log spoofing). It is now interpolated with !r; a second test probes rank@v1\x1b[31mINJECTED and asserts no escape byte arrives.

Not widened: the version never becomes a path segment (lookup is HANDLE_RE → hex join, or a name compare); ../, absolute paths, @@, doubled @ and 200k-char refs all return not-found with no traceback; no new dependency, subprocess, write or secret path.

Tests (written first, both red before the fix)

tests/test_cli.py

  • test_get_returns_the_version_you_pinned
  • test_get_with_a_version_that_does_not_exist_fails_loudly
  • test_inspect_code_shows_the_version_you_pinned
  • test_get_does_not_echo_control_bytes_from_a_ref

tests/test_registry.py

  • test_get_code_honours_the_version_pin

Red before: 3 AssertionErrors in test_cli.py plus the get_code one in test_registry.py. Green after.

Commands run

pytest -q                       107 passed
jitagent selftest               6/6 corpus cases behaved as expected
python tools/audit_traps.py     18/18 traps exercised
jitagent get rank@v1            v1 source, exit 0
jitagent get rank@v9            exit 1, "rank has no version 'v9' (it has: v1, v2)", no source
jitagent inspect rank@v1 --code "*v1 ... -- v1 source --"
jitagent call rank@v1           ok v1 (unchanged)

End-to-end on a scratch registry seeded with two real versions of one function, not only in unit tests.

Checks not run

  • tools/audit_tests.py and tools/audit_e2e.py: they need a real model/tokens and are unrelated to a read-path fix.
  • Python 3.11 / 3.13 locally (ran 3.12; CI covers the matrix).
  • No network, no secrets, no writes outside the registry, so no security-relevant surface: the diff touches argument resolution only.

Scope

One defect, one branch (fix/version-pin-honoured), five files (+143/−17). No dependency, format or behaviour change outside the two read commands. Not merged, not force-pushed, main untouched.

Fledge · jitagent · Lead · implemented

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — score 88 / 100

Reviewed at head 8bc75c3 against base 34b3d2f (5 files, +125/−17), and against issue #9. CI on the head commit is green: test (3.11), test (3.12), test (3.13), package. I could not rerun pytest here (no Python interpreter in the reviewer sandbox), so the local counts in the PR body are taken on trust; everything below is from reading the diff and tracing each new test against the base.

1. resolve_version — the ref shapes

resolve_version (src/jitagent/jit.py:267) is the right shape: one place turns a ref into (function, version), and both read commands go through it. Traced against the registry's own parsing (split_ref, registry.py:69; get prefix path, registry.py:199):

ref resolves to verdict
rank (bare name) fn.best() ✅ identical to the old path
fn_7a3c9e (bare handle) handle lookup ✅
fn_7a3c (handle prefix) prefix branch of registry.get, then version applies ✅ but untested (see 3)
rank@ (empty version) split_ref → None → unpinned → newest ✅ lenient, and it matches runtime.py:77 (if want else fn.best()), so get and call agree; worth a test so it stays deliberate
rank@v9 (unknown version) NoSuchVersion listing the versions ✅ this is the defect in #9, and the message is the right one
function with no versions (fn, None) → cmd_get prints has no versions, exit 1 ✅ previously "no function named …", same exit code, better message
a name that contains @ mis-split: get a@b reads as name a + version b ❌ pre-existing hole, now entrenched

On the last row: --name (cli.py:281) accepts any string and Registry.put (registry.py:286) only checks name collisions, so --name "a@b" stores a function that no command can reach — the ref parser splits it before the lookup. call has always had this hole, so it is not a regression, but this PR is the moment the @ separator became a named, load-bearing concept; a one-line guard rejecting @ in --name (plus a test) would close it, or failing that document @ as reserved.

Minor: the ref is split twice per resolve — once inside registry.get (registry.py:205) and again at jit.py:279. Not a bug, just two parses of the same string.

2. get_code swallowing NoSuchVersion — the one place I would change

It is defensible — the old contract was "empty string if there is none", and this keeps get_code("nosuch") working — and the CLI does not rely on it. But it leaves the last silent path in a PR whose whole thesis is never hand back something that looks fine but is wrong: get_code("rank@v9") returns "", and a caller writing get_code(ref) or compile_function(...) recompiles instead of learning the version was wrong. NoSuchVersion subclasses KeyError, so raising it is cheap for callers (except KeyError still works). Given 0.0.x and an [Unreleased] entry, I would raise.

If you keep "", say so where people will look: the CHANGELOG entry is CLI-only, and the docstring at jit.py:290 ("empty string when there is nothing under that ref") is technically true but conflates "no such function" with "wrong version" — the exact distinction NoSuchVersion's own docstring argues for. This is the main reason the score is 88 rather than ~92.

3. The tests: red before, meaningful after

I traced each of the four against 34b3d2f:

  • test_get_returns_the_version_you_pinned — base prints v2, which carries # impl_two, so "impl_two" not in out fails. Red. ✅
  • test_get_with_a_version_that_does_not_exist_fails_loudly — base prints v2 and exits 0, so == 1 fails. Red. ✅ And it checks the half that matters: cap.out == "", i.e. no other version's source on stdout.
  • test_inspect_code_shows_the_version_you_pinned — base header is -- v2 source --, so "v1 source --" in out fails. Red. ✅ Splitting on the header and asserting absence after it is the right assertion; a whole-file equality would have been brittle.
  • test_get_code_honours_the_version_pin — base returns v2 for the @v1 assertion. Red. ✅ The two codes differ by a return value, so equality is meaningful.

The # impl_two marker is legitimate, not accidental: the fixture stores the same requirement twice, so the marker is the only difference, and the tests assert the marker's presence and absence per pin, plus ["v1", "v2"] ordering in the fixture — an overwrite or a "newest wins" regression fails the test, not the marker. One caveat: unlike the credential-gate PR, red-before is not visible in the history (commit 1 is the whole fix; commit 2 is typing plus blank lines), so the claim rests on this reasoning rather than on a red commit in the log.

Gaps worth one test each, all non-blocking: the * marker in inspect without --code (the changed mark = line is otherwise untested); handle + version (fn_7a3c@v1); name@; and nosuch@v1 (message reads no function named 'nosuch@v1', fine but unasserted).

4. Backwards compatibility

No pin → fn.best() (jit.py:281), the same object the old code used, so output for get rank (test_get_prints_the_code), get no-such-name (test_get_on_an_unknown_name_fails_loudly) and inspect rank (test_list_and_inspect_show_the_name) is unchanged — I checked each against the base implementation. call is untouched: neither cmd_call nor runtime.py appears in the diff. Exit codes stay 0/1, errors stay on stderr with nothing on stdout, and the --code header format (-- <vn> source --) is unchanged; only the version name can now differ. The one semantic drift: * in inspect now means "the version you asked for" instead of "newest". For unpinned refs those are the same, and for pinned refs the new meaning is the point — worth a line in inspect --help/README so nobody reads * as "newest" on a pinned query.

5. Things the suite will not catch

  • KeyError quoting. NoSuchVersion subclasses KeyError (jit.py:258), whose __str__ renders repr(args[0]), so print(e) in cmd_get/cmd_inspect emits 'rank has no version v9 (it has: v1, v2)' — wrapped in quotes. The test asserts the substring no version v9, so it passes, and the PR body's end-to-end example shows it unquoted. This matches the existing UnknownHandle(KeyError) style in cmd_call, so I read it as a nit, not a defect; if you want clean stderr, subclass Exception or print e.args[0].
  • Merge order. #7, #8 and #10 all base on 34b3d2f, all insert a ## [Unreleased] block at the same CHANGELOG line, and #8 inserts _call_payload immediately before cmd_call — the exact boundary this PR's last hunk touches in cli.py. Expect trivial conflicts: merge in sequence and rebase the later ones.
  • Docs. README.md:139/143 still show get rank / inspect rank with no @version example, so the feature this PR fixes is invisible in the command table.

Verdict

Correct, minimal, well-scoped; the resolver is the right abstraction, the error path now fails loudly in the CLI, and the tests are honest rather than coincidental. Marked down for the remaining silent get_code path, the @-in-names hole this change entrenches, and the cosmetic message/coverage gaps above.

88 / 100. Merge is the maintainer's call — this is a COMMENT review; I have not approved or merged anything.

Fledge · jitagent · Reviewer · reviewed

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 367229ff — score 90 / 100

Reviewed the full 3-commit diff against 34b3d2f (5 files, +143/−17), not only the delta. CI on this head is green: test (3.11), test (3.12), test (3.13), package; mergeable_state is clean. This supersedes my review at 8bc75c3 (88): the two new commits fix the KeyError quoting I flagged there, and one thing I did not flag — the version string reaching stderr with raw control bytes. I still could not run pytest locally (no Python in the reviewer sandbox); the suite counts in the PR body are unverified by me, not contradicted.

The two new commits

  1. NoSuchVersion(ValueError) (jit.py:258) — correct fix for the right reason: print() renders a KeyError as repr(args[0]), so the CLI was emitting 'rank has no version v9 …' with stray quotes. The test now asserts the exact line (rank has no version 'v9' (it has: v1, v2)) rather than a substring, which is exactly how the quoting survived the first pass. Verified against the code, and the exact-line assertion fails on 8bc75c3 by construction.
  2. {want!r} (jit.py:291) — good catch, and the one I should have raised. jitagent get $'rank@v1\x1b[31mINJECTED' wrote a real ESC to the terminal; a ref is caller-typed text, so this is a log-spoofing primitive. The new test asserts both halves — no raw \x1b, and the escaped \\x1b[31m visible as text — and would be red on 8bc75c3. Note the version list ({have}) is still interpolated raw, but it is registry-derived vN names, not caller input; acceptable, and if you ever accept store-supplied names this is where it would bite.

The rest of the diff is unchanged from my earlier pass and I stand by that assessment: resolve_version resolves bare name, bare handle, handle prefix, rank@ (lenient → newest, matching runtime.py:77), unknown version (loud, with the list), and a version-less function correctly; unpinned output is byte-identical to the old path; call is untouched; exit codes and the stdout/stderr split are preserved.

(a) get_code returning "" for a bad pin — acceptable, with one line of paperwork

The new docstring (jit.py:298) makes the contract explicit, resolve_version is public for callers who need the distinction, and the CLI does not rely on the silent path, so I would not change it in this PR. Two things make it defensible rather than convenient: "" is falsy, so no caller mistakes it for source, and the ambiguity is now documented where a caller reads it. My remaining ask is the CHANGELOG: the entry describes the CLI fix only, so an API user reading the release notes learns nothing about get_code("rank@v9") == "". One sentence there closes it. (A strict= flag later is fine; not now.)

(b) resolve_version in jit vs Registry — right layer, wrong number of copies

The policy ("unpinned means newest") belongs above the store, so jit is a defensible seam; but the mechanism (parse ref, look up version, raise) now exists twice — jit.py:270 and runtime.py:74-81. The divergence is already visible: jitagent call rank@v9 still raises UnknownHandle(KeyError), so it prints a quoted message with no list of versions, while jitagent get rank@v9 prints the unquoted message with the list. Same mistake, two formats. My preference is a Registry.resolve(ref) -> (Function, Version) that raises one NoSuchVersion, used by runtime, get_code and both commands — split_ref and Version already live in registry.py, so nothing has to move far. If you keep the jit seam, at least make Runtime.call_many call it and delete the duplicate; the key-shapes-not-names review made the same argument for one place a rule lives.

Remaining nits (non-blocking, mostly pre-existing)

  1. cmd_inspect still interpolates the handle raw: no such function: {args.handle} (cli.py:204). Same class as the escape bug just fixed, one door over; !r there too.
  2. fn.ref is printed raw in -- no such function/has no versions/inspect header, and --name accepts any string, including @ and ESC. Names are stored unvalidated (registry.py:286 only checks collisions), so a name containing @ is unreachable and a name with a control byte is echoed. One guard at put time — reject @ and control characters in names — fixes both, and it is the natural follow-up to making @ load-bearing.
  3. Coverage still missing: handle + version (fn_7a3c@v1), name@ (empty version), the * marker in inspect without --code, and nosuch@v1.
  4. Merge order: #7, #8 and #10 all base on 34b3d2f, all add ## [Unreleased] at the same CHANGELOG line, and #8 inserts _call_payload right where this PR's cmd_get hunk ends — expect trivial conflicts; merge in sequence.
  5. README.md:139/143 still shows get rank / inspect rank with no @version example.

Verdict

Correct, minimal, and now well-defended: the fix does what the issue says, the read path fails loudly instead of silently printing another version, and the two risk-pass commits hardened the error text and closed a terminal-injection path. Remaining deductions are the duplicate resolver (which keeps call's error inconsistent) and the small coverage/naming gaps above.

90 / 100. Merge is the maintainer's call — this is a COMMENT review; I have not approved or merged anything.

Fledge · jitagent · Reviewer · reviewed

@Birfy

Birfy commented Nov 1, 2026

Copy link
Copy Markdown
Owner Author
Fledge automation · jitagent · job job_ZEvb5EAn4Zks_w (schedule, iterate)

Automated run: jitagent Lead claimed + implemented + tested · jitagent Reviewer reviewed.

Fledge never merges; a human must review and merge.

Birfy added 3 commits November 1, 2026 11:14
`call rank@v1` runs v1. `get rank@v1` printed the newest version's source, and so
did `inspect --code`, because get_code resolved the function through split_ref (which
drops the version) and then took .best(). Reading what will run is the whole point of
being able to pin a version, so the two must agree.

resolve_version() is now the one place a ref becomes (function, version); get_code and
both commands go through it. An unknown version raises instead of silently returning
the newest, and the error lists the versions that do exist.
Reviewer-track finding from the security pass: NoSuchVersion inherited KeyError, so
print(e) rendered the message as a quoted repr — rank has no version v9 — which
reads like a printer bug. It is now a ValueError, and the version the caller typed is
interpolated with !r so an ESC byte in a ref cannot reach the terminal raw.

The CLI test asserted a substring, which is why the quoting survived the first pass; it
now asserts the exact line, and a second test pins the control-byte case.
Birfy added a commit that referenced this pull request Nov 2, 2026
Three review findings, all in the change's own blast radius:

- `put` and `_bank_the_hit` assigned `fn.name` before `save_spec` judged
  it, so a library caller that caught `BadName` kept a Function holding
  the name that was just refused. Both now assign `check_name(name)`, so
  failure is atomic in memory as well as on disk; `save_spec` keeps its
  check as the last line of defence.
- The message assertions were substrings, which is exactly how the
  `KeyError` repr wrapper survived review on #10: `print(e)` would put
  quotes around the whole line and every substring would still match. The
  CLI test now compares the entire unquoted line, and the registry tests
  assert the message starts with the name's repr.
- The claim that a name stored before the guard keeps working was verified
  by reading only. A test now seeds a spec.json with `a@b`, stores a new
  version with no name, and heals it with a valid rename.

Also covers `" "` and `"@A"` (both split to an empty base) and asserts the
edge-space message names the lookup, both flagged as untested shapes.

Refs #11.

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 367229ff — 90/100 (head unchanged since 5727664841)

Head 367229ff97db1ab0ada246151c42f1c5963f885e (3 commits, 5 files, +143/−17). Tree == GitHub tarball at this SHA; no commits since my 5727664841, so the 90 stands.

  • pytest -q in that tree: 107 passed (36.8 s).
  • CI on this SHA: 4/4 success — package, 3.11, 3.12, 3.13 (run 41299296688; all jobs checked).
  • resolve_version still does what the title says: get rank@v1 returns v1's source, a missing version fails loudly with the list (rank has no version 'v9' (it has: v1, v2)), {want!r} keeps control bytes escaped, the unpinned path is byte-identical to the old one, and call is untouched.

Open, unchanged: the CHANGELOG does not tell API callers that get_code("rank@v9") == ""; the ref→version mechanism still exists twice (jit.py vs runtime.py), so call rank@v9 and get rank@v9 still format the same mistake differently; cmd_inspect prints the handle raw; the coverage gaps (handle + version, name@, inspect marker without --code, nosuch@v1) remain; #7/#8/#10 all insert ## [Unreleased] at the same CHANGELOG line, so merge in sequence.

90/100. COMMENT — not approved, not merged.

Fledge · jitagent · Reviewer · reviewed

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 367229ff — 90/100 (head unchanged since 5761485429)

Head 367229ff97db1ab0ada246151c42f1c5963f885e; no commits since that review, so the 90 stands. Re-verified today: pytest -q → 107 passed, CI 4/4 (run 41299296688), mergeable_state: clean. resolve_version still does what the title says: get rank@v1 returns v1's source, a missing version fails loudly with the list, and the unpinned path is byte-identical.

Open, unchanged: the CHANGELOG does not tell API callers that get_code("rank@v9") == ""; the ref→version mechanism exists twice (jit.py vs runtime.py), so call rank@v9 and get rank@v9 format the same mistake differently; cmd_inspect prints the handle raw; the coverage gaps (handle + version, name@, inspect marker without --code, nosuch@v1) remain. #7/#8/#10 all insert ## [Unreleased] at the same CHANGELOG line — merge in sequence.

90/100. COMMENT — not approved, not merged.

Fledge · jitagent · Reviewer · reviewed

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 367229f — 90/100 (head unchanged since 5768467309)

Same head 367229ff97db1ab0ada246151c42f1c5963f885e; no commits since that review, so the 90 stands. Re-verified today: pytest -q → 107 passed; CI 4/4 (run 41299296688), mergeable_state: clean, base 34b3d2f.

resolve_version still does what the title says: get rank@v1 returns v1's source, a missing version fails loudly with the list, and the unpinned path is byte-identical.

Open, unchanged: the CHANGELOG does not tell API callers that get_code("rank@v9") == ""; the ref→version mechanism exists twice (jit.py vs runtime.py), so call rank@v9 and get rank@v9 format the same mistake differently; cmd_inspect prints the handle raw; the coverage gaps (handle + version, name@, inspect marker without --code, nosuch@v1) remain. #7/#8/#10 all insert ## [Unreleased] at the same CHANGELOG line — merge in sequence.

90/100. COMMENT — not approved, not merged.

Fledge · jitagent · Reviewer · reviewed

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 367229f — 90/100

Head unchanged since the last review; re-ran everything at this SHA just now.

Verified by running (CPython 3.12.16, PYTHONPATH=src)

  • pytest -q → 107 passed; selftest 6/6; audit_traps 18/18; CI run 41299296688 success; mergeable_state: clean; base 34b3d2f.
  • Built a registry with v1 and v2: get count@v1 → v1's source; get count → v2's; get count@v9 → rc 1, count has no version 'v9' (it has: v1, v2); call count@v1 still runs the pinned code; get_code at the API level returns "" for a missing pin by contract (jit.py:300-306).

Non-blocking

  1. The CHANGELOG does not tell API callers that get_code("name@v9") == ""; it is in the docstring only.
  2. Ref→version parsing exists twice (jit.resolve_version at jit.py:270 vs runtime), so call rank@v9 and get rank@v9 format the same mistake differently.
  3. cmd_inspect (cli.py:194) prints the handle raw.
  4. Coverage gaps: handle + version together, bare name@, inspect --code marker without a pin, nosuch@v1.
  5. Merge note: #7/#8/#10/#21 all insert ## [Unreleased] at the same CHANGELOG line — merge in sequence.

90/100. COMMENT — not approved, not merged.

Fledge · jitagent · Reviewer · reviewed

@Birfy Birfy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 367229f — 90/100

Head unchanged since review 5785359393 at the same SHA; re-ran everything at this head just now.

Verified by running (CPython 3.12.16, PYTHONPATH=src)

  • pytest -q → 107 passed; selftest 6/6; audit_traps 18/18; CI 4/4 success; mergeable_state: clean; base 34b3d2f.
  • Registry with v1 and v2: get count@v1 → v1's source; get count → v2's; get count@v9 → rc 1, count has no version 'v9' (it has: v1, v2); call count@v1 runs and reports v1.

Non-blocking (unchanged)

  1. The CHANGELOG does not tell API callers that get_code("name@v9") == ""; that contract is only in the docstring (jit.py:300-306).
  2. Ref→version parsing exists twice (jit.py:270 vs runtime), so call rank@v9 and get rank@v9 format the same mistake differently.
  3. cmd_inspect (cli.py:194) prints the handle raw; coverage gaps remain (handle+version, bare name@, nosuch@v1).
  4. Merge note: #7/#8/#10/#21 share the same CHANGELOG insertion line.

90/100. COMMENT — not approved, not merged.

Fledge · jitagent · Reviewer · reviewed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

jitagent get and inspect --code ignore the @version pin and print the newest source

1 participant