Add first step towards a validator - #373
Open
r-ash wants to merge 4 commits into
Open
Conversation
New leapfrog-validate CLI/library: build-params/run/diff independently build leapfrogr at a git ref via an isolated worktree, run the model, and compare a single indicator (total_population) between two refs with the hybrid atol + rtol*|ref| tolerance formula from the validation-system PRD. Proves the shape of the wider leapfrog-validation system end-to-end before the rest (five indicators, Spectrum comparison, PR reporting) gets layered on top. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same commands/options/behavior, just typer's type-hint-driven style (Annotated arguments, a callback for the shared --cache-dir option instead of a click group context).
…ktree Both Rscript subprocess calls run with cwd set to the cached per-ref git worktree (needed for R CMD INSTALL, not for these), so a relative -o path was resolving there instead of the caller's actual directory -- a file written with -o some.h5 would silently land in ~/.cache/leapfrog-validate/worktrees/<sha>/some.h5 instead of next to where the command was run, and a later `run` pointed at that same relative path would then correctly report it missing. Resolve pjnz/output/params to absolute paths before handing them to the subprocess. Adds a regression test for each command exercising a relative -o path against a worktree with a different cwd. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
R scripts: parse args with docopt::docopt() (matching leapfrogr/scripts/create_model_output.R's existing convention) instead of raw commandArgs(); call leapfrog::/leapfrog::: qualified rather than library(leapfrog). Ruff: the ignore list was copied wholesale from codegen/pyproject.toml. Cleared it and re-ran against the real codebase -- only docstring rules and subprocess S603/S607 actually fired. Added real docstrings to every flagged public class/function in src/ (16 sites), exempted test functions from docstring rules per-file (matching leapfrog-py/tests' existing undocumented-test-function convention), and kept S603/S607 ignored with a one-line justification -- shelling out to git/R/uv with argument lists is this tool's whole job, not untrusted input reaching a shell. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR will