|
| 1 | +# plotnik |
| 2 | + |
| 3 | +Query language for tree-sitter AST with named subqueries, recursion, and type inference. See [docs/REFERENCE.md](docs/REFERENCE.md) for spec. |
| 4 | + |
| 5 | +Lexer (logos) + parser (rowan) are resilient: collect errors, don't fail-fast. |
| 6 | + |
| 7 | +## Project Structure |
| 8 | + |
| 9 | +``` |
| 10 | +crates/ |
| 11 | + plotnik-lib/ # Core library |
| 12 | + src/ |
| 13 | + ast/ # Syntax infrastructure |
| 14 | + lexer.rs # Token definitions (logos) |
| 15 | + syntax_kind.rs # SyntaxKind enum |
| 16 | + nodes.rs # Typed AST wrappers over CST |
| 17 | + parser/ |
| 18 | + core.rs # Parser infrastructure |
| 19 | + grammar.rs # Grammar rules |
| 20 | + error.rs # Parse errors |
| 21 | + tests/ # Parser tests (snapshots) |
| 22 | + query/ # Query processing |
| 23 | + mod.rs # Query struct, new(), pipeline |
| 24 | + dump.rs # dump_* debug output methods |
| 25 | + errors.rs # Error access methods |
| 26 | + alt_kind.rs # Alternation validation |
| 27 | + named_defs.rs # Name resolution, symbol table |
| 28 | + ref_cycles.rs # Escape analysis (recursion validation) |
| 29 | + shape_cardinalities.rs # Shape inference |
| 30 | + lib.rs # Re-exports Query |
| 31 | + plotnik-cli/ # CLI tool |
| 32 | + src/commands/ # Subcommands (debug, docs, langs) |
| 33 | + plotnik-langs/ # Tree-sitter language bindings |
| 34 | +docs/ |
| 35 | + REFERENCE.md # Language specification |
| 36 | +``` |
| 37 | + |
| 38 | +## Pipeline |
| 39 | + |
| 40 | +```rust |
| 41 | +ast::parse() // Parse → CST |
| 42 | +alt_kind::validate() // Validate alternation kinds |
| 43 | +named_defs::resolve() // Resolve names → SymbolTable |
| 44 | +ref_cycles::validate() // Validate recursion termination |
| 45 | +shape_cardinalities::infer() // Infer shape cardinalities |
| 46 | +shape_cardinalities::validate() // Validate field constraints |
| 47 | +``` |
| 48 | + |
| 49 | +Module = "what", function = "action". |
| 50 | + |
| 51 | +## CLI |
| 52 | + |
| 53 | +Run: `cargo run -p plotnik-cli -- <command>` |
| 54 | + |
| 55 | +- `debug` — Inspect queries/sources |
| 56 | +- `docs [topic]` — Print docs (reference, examples) |
| 57 | +- `langs` — List supported languages |
| 58 | + |
| 59 | +### debug options |
| 60 | + |
| 61 | +Inputs: `-q/--query <Q>`, `--query-file <F>`, `--source <S>`, `-s/--source-file <F>`, `-l/--lang <L>` |
| 62 | + |
| 63 | +Output: `--show-query`, `--show-source`, `--only-symbols`, `--cst`, `--raw`, `--spans`, `--cardinalities` |
| 64 | + |
| 65 | +```sh |
| 66 | +cargo run -p plotnik-cli -- debug -q '(identifier) @id' --show-query |
| 67 | +cargo run -p plotnik-cli -- debug -q '(identifier) @id' --only-symbols |
| 68 | +cargo run -p plotnik-cli -- debug -s app.ts --show-source |
| 69 | +cargo run -p plotnik-cli -- debug -s app.ts --show-source --raw |
| 70 | +cargo run -p plotnik-cli -- debug -q '(function_declaration) @fn' -s app.ts -l typescript --show-query |
| 71 | +``` |
| 72 | + |
| 73 | +## Syntax |
| 74 | + |
| 75 | +Grammar: `(type)`, `[a b]` (alt), `{a b}` (seq), `_` (wildcard), `@name`, `::Type`, `field:`, `*+?`, `"lit"`/`'lit'`, `(a/b)` (supertype), `(ERROR)`, `Name = expr` (def), `[A: ... B: ...]` (tagged alt) |
| 76 | + |
| 77 | +SyntaxKind: `Root`, `Tree`, `Ref`, `Str`, `Field`, `Capture`, `Type`, `Quantifier`, `Seq`, `Alt`, `Branch`, `Wildcard`, `Anchor`, `NegatedField`, `Def` |
| 78 | + |
| 79 | +Expr = `Tree | Ref | Str | Alt | Seq | Capture | Quantifier | Field | NegatedField | Wildcard | Anchor`. Quantifier/Capture wrap their target. |
| 80 | + |
| 81 | +## Errors |
| 82 | + |
| 83 | +Stages: `Parse` → `Validate` → `Resolve` → `Escape`. Use `Query::errors_for_stage()`. |
| 84 | + |
| 85 | +## Constraints |
| 86 | + |
| 87 | +- Defs must be named except last (entry point) |
| 88 | +- Fields: `field: expr` — no sequences as direct values |
| 89 | +- Alternations: same-name captures need same type; use `@x :: T` for merged structs; tagged alts for discriminated unions |
| 90 | +- `.` anchor = strict adjacency; without = scanning |
| 91 | +- Names: `Upper` = user-defined, `lower` = tree-sitter nodes |
| 92 | +- Captures: snake_case only, no dots |
| 93 | + |
| 94 | +## Data Model |
| 95 | + |
| 96 | +- Nesting in query ≠ nesting in output: `(a (b @b))` → `{b: Node}` |
| 97 | +- New scopes only from captured `{...}@s` or `[...]@c` |
| 98 | +- `?`/`*`/`+` = optional/list/non-empty list |
| 99 | + |
| 100 | +## AST Layer (`ast/nodes.rs`) |
| 101 | + |
| 102 | +Types: `Root`, `Def`, `Tree`, `Ref`, `Str`, `Alt`, `Branch`, `Seq`, `Capture`, `Type`, `Quantifier`, `Field`, `NegatedField`, `Wildcard`, `Anchor`, `Expr` |
| 103 | + |
| 104 | +Use `Option<T>` for casts, not `TryFrom`. Use `QueryPrinter` from `query/printer.rs` for output. |
| 105 | + |
| 106 | +## Testing |
| 107 | + |
| 108 | +Uses `insta` for snapshot testing. Critical workflow: |
| 109 | + |
| 110 | +1. Use `indoc!` macro for multi-line query input |
| 111 | +2. Always write empty string `@""` for new snapshots |
| 112 | +3. Run `cargo insta accept` to populate snapshots (or `cargo insta review` to inspect) |
| 113 | + |
| 114 | +```rust |
| 115 | +#[test] |
| 116 | +fn my_test() { |
| 117 | + let input = indoc! {r#" |
| 118 | + (function_declaration |
| 119 | + name: (identifier) @name) |
| 120 | + "#}; |
| 121 | + |
| 122 | + let query = Query::new(input); |
| 123 | + assert!(query.is_valid()); |
| 124 | + insta::assert_snapshot!(query.dump_ast(), @""); // <-- empty string, always |
| 125 | +} |
| 126 | +``` |
| 127 | + |
| 128 | +Then run: |
| 129 | + |
| 130 | +```sh |
| 131 | +cargo test --workspace |
| 132 | +cargo insta accept |
| 133 | +``` |
| 134 | + |
| 135 | +Never write snapshot content manually. Let insta generate it. |
| 136 | + |
| 137 | +**Test patterns:** |
| 138 | + |
| 139 | +- Valid parsing: `assert!(query.is_valid())` + snapshot `dump_*()` output |
| 140 | +- Error recovery: `assert!(!query.is_valid())` + snapshot `dump_errors()` only |
| 141 | + |
| 142 | +## Coverage |
| 143 | + |
| 144 | +Uses `cargo-llvm-cov`, already installed. |
| 145 | + |
| 146 | +Find uncovered lines per file: |
| 147 | + |
| 148 | +```sh |
| 149 | +cargo llvm-cov --package plotnik-lib --text --show-missing-lines 2>/dev/null | grep '\.rs: [0-9]\+\(, [0-9]\+\)\*\?' |
| 150 | +``` |
| 151 | + |
| 152 | +## Invariants |
| 153 | + |
| 154 | +Two-tier resilience strategy: |
| 155 | + |
| 156 | +1. Parser: resilient, collects errors, continues parsing |
| 157 | +2. Our code: strict invariants, maximal coverage in tests, panic on violations |
| 158 | + |
| 159 | +Invariant checks live in dedicated modules named `invariants.rs`. |
| 160 | +They are excluded from test coverage because they're unreachable. |
| 161 | +They usually wrap a specific assert. |
| 162 | +It was done due to limitation of inline coverage exclusion in Rust. |
| 163 | +But it seems to be useful to extract such invariant check helpers anyways: |
| 164 | +- if it just performs assertion and doesn't return value, it starts with `assert_` |
| 165 | +- if it returns value, it's name consists of' `ensure_` and some statement about return value |
| 166 | +Find any of such files for more examples. |
| 167 | + |
| 168 | +## Not implemented |
| 169 | + |
| 170 | +- Semantic validation: casing rules |
| 171 | + |
| 172 | +## Deferred |
| 173 | + |
| 174 | +- Predicates (`#match?` etc.) — runtime filters, not structural |
| 175 | + |
| 176 | +## Rules |
| 177 | + |
| 178 | +- Update AGENTS.md when changes add useful context |
| 179 | +- Check diagnostics after changes |
| 180 | +- Follow rnix-parser/taplo patterns |
| 181 | +- Span-based tokens, no text in intermediate structures |
| 182 | +- Don't put AI slop comments in the code |
| 183 | +- IMPORTANT: Avoid nesting logic, prefer early exit code flow in functions (return) and loops (continue/break) |
0 commit comments