Repository navigation
docs: architecture map with a numbered diagram - #3
Conversation
Add docs/architecture.md: the components, the trust and data boundaries, the path one request or run takes through the system as numbered steps that match the arrows on docs/architecture.svg, and a table of what is proven and where the evidence lives. The README gains an Architecture section that shows the diagram and links the page. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Widen node boxes, balance the four regions and row spacing, and align SQL above Caller. Route schemas around the perimeter, separate the tables and differential paths, and move labels and numbered badges clear of borders, text and unrelated edges. Preserve every component, edge, fact and step number. No labels were cut; only the caller note gains a line break. The generator is unchanged. Verified through four render/gate/visual rounds, a final GATE PASS, and exact checks for edge crossings and badge placement. Co-Authored-By: Codex astra <noreply@openai.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 01efe41a6f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 2. **Lexer → parser.** A hand-written recursive-descent parser (`sql::parse_select`) builds a `sql::SelectQuery` AST. Errors carry a position: `parse error at position N: …`. | ||
| 3. **Parser → binder.** `sql::bind_select` resolves names against a `catalog::Catalog`, types every expression (`Int64` or `String`), normalises `RIGHT` joins to `LEFT`, tracks correlation depth for subqueries, and produces a typed `plan::LogicalPlan` (`Scan`, `Join`, `Filter`, `Project`, `Aggregate`, `Window`, `Distinct`, `Sort`, `Limit`, `Explain`). | ||
| 4. **Binder → memo.** `Memo::insert` places the plan into equivalence groups bottom-up with deterministic group ids. | ||
| 5. **Memo ↔ rewrite rules.** `explore_memo_to_fixpoint(default_memo_rules())` applies the thirteen memo rules (semi and anti join introduction for `EXISTS`, `NOT EXISTS`, `IN` and `NOT IN`, including their correlated forms; `LeftJoinToInnerRule`; `FilterIntoJoinRule`; `FilterThroughAggregateRule`; `JoinCommuteRule`; `JoinAssociateRule`) until no group gains a new expression. Four standalone rules (`ConstantFoldComparisonRule`, `MergeAdjacentFiltersRule`, `AlwaysFalseFilterRule`, `DropAlwaysTrueFilterRule`) run outside the memo through `rewrite_to_fixpoint`. Every rule file carries a proof section in the shape of [`rewrite-rule-template.md`](./rewrite-rule-template.md). |
There was a problem hiding this comment.
default_memo_rules() actually returns the four simplification rules listed here as well as the thirteen optimizer-only rules, for 17 memo rules total. Describing those four as running only outside the memo gives readers an incorrect model of which transformations memo exploration performs; clarify that they implement both Rule and MemoRule and run in both paths.
Useful? React with 👍 / 👎.
|
|
||
| There is no single `run(sql)` entry point. Callers (tests, benchmarks, the fuzzer) compose the stages; `execute_vectorized` performs steps 7 to 9 itself, and the optimizer stage (4 to 6) is optional. | ||
|
|
||
| 1. **SQL text → lexer.** The lexer turns the text into tokens and enforces the nesting depth (`kMaxNestingDepth = 64`) and boolean operator (`kMaxBooleanOperators = 200`) bounds. |
There was a problem hiding this comment.
The lexer only tokenizes input; kMaxNestingDepth, kMaxBooleanOperators, NestingGuard, and count_boolean_operator are all implemented by Parser in src/sql/parser.cpp. This stage description, also reflected by the Lexer annotation in the diagram, directs readers to the wrong component when investigating or changing these safeguards.
Useful? React with 👍 / 👎.
| | Vectorized engine | `BatchView`, `SelectionVector`, typed kernels | `src/execution/vectorized.cpp` | | ||
| | Catalog and storage | `catalog::Catalog` (schemas), `execution::Catalog` (schemas plus batches), `ColumnarBatch`, `Int64Column`, `StringColumn` with copy-on-write values and an optional validity vector | `include/catalog/catalog.hpp`, `include/storage/columnar_batch.hpp` | | ||
| | Tests | Eight CTest executables, `differential_verifier.hpp` | `tests/` | | ||
| | Benchmarks | `sql_bench` (13 workloads) and `yard_export` | `benchmarks/`, [`benchmarks.md`](./benchmarks.md), [`yard/`](./yard/) | |
There was a problem hiding this comment.
make_workloads() in benchmarks/sql_bench.cpp currently constructs 14 workloads, not 13. The component inventory is stale and should report 14 so the architecture map matches the benchmark executable it directs readers to.
Useful? React with 👍 / 👎.
1 participant
Adds a reference-architecture drawing and a walkthrough for this repository.
docs/architecture.svg: boxed regions, component icons and numbered arrows, rendered from a JSON spec and checked by a geometry gate (text fits its box with slack for wider fonts; nothing touches or crosses anything else).docs/architecture.md: the numbered path arrow by arrow, the component table, the trust and data boundaries, what is proven and where the evidence lives, and the identifiers the code relies on. Every claim comes from this repository's own README, ADRs and code, and the page says what is not claimed.Two commits: the map, then the geometry pass.