Repository navigation
Add pks list-references to list the references between packs - #71
Merged
Merged
Conversation
`pks check` reports only the references that break a rule. Nothing shows how packs use each other's constants where the rules allow it, as through a declared dependency or a public constant, and tools that read package_todo.yml files, such as query_packwerk, see only recorded violations (rubyatscale/query_packwerk#8). `pks list-references` prints, as JSON, every reference from one pack to a constant that another pack defines, with the violations `check` finds in it. Each violation gives its type, whether it is strict, and whether a package_todo.yml records it, so an allowed reference lists none. It exits 0 either way. `--defining-pack` and `--referencing-pack` limit the list to references to or from the packs they name, and can be repeated. An unknown pack is an error, as in `rm`. References within a pack, and to constants that no pack defines, are left out. A constant defined in several files of a pack, which only the experimental parser resolves to more than one definition, yields a copy of the reference for each file. The copies are listed as one reference with every violation any of them makes, as `check` reports them. Taking one copy's violations would drop a privacy violation whenever that copy's file was public, and which copy comes first varies between runs. pks keeps no source positions in ERB and reports line 0 there, so an ERB reference has a null line and column, and an ERB file's references to one constant are listed once. schema/list-references-output.json describes the output, and the tests check each output against it.
The schema now says what a consumer needs to rely on the output: - References are sorted by file, line, column, constant and defining pack, and a constant that two packs define gives a reference to each at the same place. - `violations` lists recorded violations as well, which `check` leaves out, in checker order. An empty list means the reference breaks no enforced rule, which includes a rule that isn't enforced. - `strict` is the strictness of the pack that enforces the rule: the referencing pack for dependency and layer, the defining pack for the rest. - `column` counts bytes, and it is null exactly when `line` is. ERB's null is `check -o json`'s 0. The sort and the checker loop run in parallel, as `check`'s checkers do. On a synthetic app with 1M references, the command took 5.4 s instead of 7.2 s, and its output was byte for byte the same. Copies of a reference give the same output in any order, so an unstable sort is enough. `par_chunk_by` needs rayon 1.10, and since Cargo.lock isn't tracked, Cargo.toml's requirement is what guarantees it. Dropping `line`, `column`, `constant_name` or `defining_pack_name` from `identity()` left every test passing, since no fixture had two references that differ only in one of them. New tests cover each, along with line 1 and column 0, the order of files, a violation recorded but not strict, and the visibility, layer and folder_privacy types. The several-definitions test caught a regression to one copy's violations only when that copy came first. Now `packs/foo` ignores dependencies on the private copy's folder, so each copy makes a violation the other doesn't, and taking either copy alone fails every run. The tests drop `RUST_LOG`, which makes pks log to stderr, since they expect stderr to be empty.
3 tasks done
dduugg
added a commit
that referenced
this pull request
Oct 12, 2026
#71 needed `rayon`'s requirement raised from 1.7.0 to 1.10.0 by hand, for `par_chunk_by`. Nothing else raises a requirement, so a new API is found missing only when the build breaks. This takes the bundler and github-actions setup from rubyatscale/packs-specification: monthly, all updates in one group per ecosystem, with a 7-day cooldown. Cargo.lock isn't tracked, and Dependabot's default strategy leaves a requirement alone when it already allows the new release, as `1.7.0` allows 1.12. `versioning-strategy: increase` raises the requirement to the newest release instead, so Cargo.toml keeps up with what CI builds. dist generates release.yml and pins its actions in dist-workspace.toml, so the actions update leaves that file out.
dduugg
added a commit
that referenced
this pull request
Oct 12, 2026
…7.0 (#72) * Print `--print-files` progress to stderr `-p` printed "Started processing" and "Finished processing" lines to stdout, so the JSON from `check -o json` and `list-references` came after them and didn't parse. They are progress, not output, so they now go to stderr, where pks's warnings already go. * Read `<%==` tags as Ruby in ERB Rails' `<%== %>` outputs without escaping, but the regex that pulls Ruby out of ERB took only one `=` after `<%`, so `<%== foo(Bar) %>` became `= foo(Bar)`. That doesn't parse. lib-ruby-parser recovers only part of the file, and the tag's references and every reference after it were missed: with `<%= SomeConcern %>`, `<%== Bar.bar %>` and `<%= Baz %>`, pks found only `::SomeConcern`. In three large Rails apps, these were the only files that failed to parse. Packwerk reads ERB with a real parser and handles the tag. * Warn about files that don't parse When lib-ruby-parser couldn't parse a file, pks used whatever partial tree it recovered, or none, and said nothing, so `check` could pass while missing the file's references. Packwerk reports a syntax error as an offense. pks now prints a warning to stderr with the file and the first error, and its line in Ruby. ERB gets no line, since its lines are those of the Ruby pks makes from it. The error rides on `ProcessedFile` as `syntax_error`, which isn't serialized, so the cache format that pks shares with packwerk doesn't change. A file pks warns about isn't written to the cache, so the warning comes back on every run until the file is fixed, instead of only when the file is first parsed. Only the app's own files, its included files, warn. `expose-monkey-patches` also parses Ruby's and the gems' sources, which can use syntax pks can't parse and aren't the app's to fix; they stay quiet and cached. An empty file has no AST but no error either, so it doesn't warn. Three large Rails apps give no warnings under either parser, now that `<%==` is read correctly. The parser can't read some newer syntax, such as Ruby 3.2's anonymous argument forwarding, and those files warn too. * Read only the referencing packs' files in `list-references` `--referencing-pack` filtered references after pks had parsed every included file, so asking about one pack cost as much as asking about all of them. Only a pack's own files can make references from it, so pks now passes just those to `get_all_references`, as `check <paths>` passes its paths. The experimental parser still reads every file for definitions inside `get_all_references`, so it gains less. For two packs of a large Rails app, the output is the same as the full list limited to those packs, under both parsers, and the run took 2.6 s instead of 8.0 s with the default parser. A test checks that a file in another pack isn't read: one that doesn't parse would make pks warn. * Share test helpers, and keep `RUST_LOG` out of tests that check stderr pks's logger reads `RUST_LOG` even without `--debug`, so a developer with `RUST_LOG=debug` set saw every test that expects an empty stderr fail: 7 in `rm_test.rs`, all of `list_references_test.rs` and 5 in `check_test.rs`. CI doesn't set it, so only local runs were affected. `rm_test.rs` and `list_references_test.rs` each had the same `pks()` helper, and `list_references_test.rs` and `check_test.rs` each had a schema validator. Both move to `tests/common`, where `pks()` removes `RUST_LOG`; the new syntax error tests use it too. The `check_test.rs` tests run against the fixtures in place rather than through `Fixture`, so those five remove it themselves. With `RUST_LOG=debug`, the suite now fails only on `test_gitignore_negation_patterns`, which depends on the global gitignore rather than on logging. * Share list-references' reference selection with `rm` `rm`'s search for references into the pack it removes filtered, sorted and deduplicated the references the way `list-references` does, with its own copy of each step. `between_packs` now holds the filter and the sort, and `rm` asks it for the references to one pack from the others, then keeps one copy of each, as `list-references` merges them. It and `identity` move to `reference_extractor`, which both commands already use, rather than one command reaching into the other's module. The order `rm` prints references in is unchanged, since `identity` only adds the defining pack, which is the same for all of them. Nothing tested that `rm` lists a reference once when several of the pack's files define its constant, so a new test does, under the experimental parser. It passes before and after this change and fails without the deduplication. * Have Dependabot update Rust dependencies and actions monthly #71 needed `rayon`'s requirement raised from 1.7.0 to 1.10.0 by hand, for `par_chunk_by`. Nothing else raises a requirement, so a new API is found missing only when the build breaks. This takes the bundler and github-actions setup from rubyatscale/packs-specification: monthly, all updates in one group per ecosystem, with a 7-day cooldown. Cargo.lock isn't tracked, and Dependabot's default strategy leaves a requirement alone when it already allows the new release, as `1.7.0` allows 1.12. `versioning-strategy: increase` raises the requirement to the newest release instead, so Cargo.toml keeps up with what CI builds. dist generates release.yml and pins its actions in dist-workspace.toml, so the actions update leaves that file out. * Bump version to 0.7.0 `pks list-references` is a new command, and pre-1.0 a new feature wants a minor bump. Once this merges, auto-release.yml tags and releases v0.7.0, since Cargo.toml's version has no tag yet. Retitle `## Unreleased` to `## 0.7.0`, the heading dist takes the release notes from.
This was referenced Oct 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pks list-referencesprints, as JSON, every reference from one pack to a constant that another pack defines, with the violationspks checkfinds in it. Each violation gives its type, whether it's strict, and whether apackage_todo.ymlrecords it. A reference that the packs' rules allow, such as one covered by a declared dependency or to a public constant, is listed with an emptyviolationslist.checkreports only the references that break a rule, so today nothing shows how packs use each other where the rules allow it. Tools that readpackage_todo.ymlfiles, such as query_packwerk, see only the recorded violations. This is the pks half of rubyatscale/query_packwerk#8. AReferencescollection in query_packwerk would be built from this output.--defining-packand--referencing-packlimit the list to references to or from the packs they name. Each can be repeated. An unknown pack is an error, and a trailing slash is accepted, as inrm.schema/list-references-output.jsondescribes the output, asschema/check-output.jsondoes forcheck -o json.This PR doesn't bump the version. The entry goes under
## Unreleased.Design notes
checkreports. Taking one copy's violations would drop a privacy violation whenever that copy's file was public, and which copy comes first varies between runs.lineandcolumn, and an ERB file's references to one constant are listed once.check -o jsonreports these as"line": 0, which its own schema forbids (minimum: 1). That's left for a follow-up, along with keeping positions through the ERB-to-Ruby conversion.BufWriter, since a large app produces tens of MB of JSON.check's checkers do.par_chunk_byneeds rayon 1.10, soCargo.toml's requirement goes up from 1.7;Cargo.lockisn't tracked, so nothing else guarantees it.get_checkersbecomespub(crate)so the new module can run the checkers itself.check_allandpks rm's reference search are unchanged.Who this affects
pks list-referencesis new. Nothing else changes.Test plan
tests/list_references_test.rschecks every output against the schema. It covers: the full output forsimple_app; strict and recorded flags in every combination (uses_strict_mode_partially_recorded,contains_package_todo); every violation type; both filters, combined, and--defining-packrepeated and with a trailing slash; an unknown pack for each filter; leaving out a reference within a pack; several uses of a constant in one file, and one at line 1, column 0; the order of files; ERB, with two constants and two files; and, under the experimental parser, a constant with a public and a private definition, and one that two packs define.pks check. Its unrecorded violations matchcheck -o json's violations exactly, line and column included. Every reference matches the schema, and none repeats.cargo test,cargo fmt --check, andcargo clippy --all-targets --all-featureswith-Dwarnings. Locally,test_gitignore_negation_patternsfails onmaintoo when the global gitignore has*.log, and it passes without one.