[prrao87]

Why

Before this change, SQL could reach the graph only through joins written by hand, so it never used the CSR, and SqlEngine read every Lance dataset into memory before running a query. This PR follows DuckPGQ's design, a graph pattern embedded in a SQL FROM clause, without implementing the SQL/PGQ dialect. The Cypher planner plans the pattern, so its hops run on the CSR and the rest of the query is ordinary DataFusion SQL over lazily scanned Lance data.

sql_engine = SqlEngine(datasets, config, use_csr=True)
sql_engine.execute("""
    SELECT g.person, p.age AS friend_age
    FROM GRAPH_TABLE (
        MATCH (a:Person)-[:KNOWS]->(b:Person) WHERE a.age < $max_age
        COLUMNS (a.name AS person, b.id AS friend_id)
    ) g
    JOIN person p ON p.id = g.friend_id
    WHERE p.age > $min_age
""", {"max_age": 30, "min_age": 25}, timeout=5)

What changed

• GRAPH_TABLE clause. GRAPH_TABLE (MATCH ... WHERE ... COLUMNS (...)) is rewritten to a graph_table('<Cypher>') table function. The function plans the Cypher and returns the plan as a view, which DataFusion inlines into the SQL plan. CSR hops therefore appear as CsrExpand nodes, and the existing CSR optimizer rules (filter pushdown, column pruning, degree counting) apply with no changes. The pattern bodies are Cypher. There is no CREATE PROPERTY GRAPH: the graph is the engine's GraphConfig, and a graph name before MATCH is accepted and ignored.
• SqlEngine runs on a CypherEngine snapshot. In Rust and Python, SqlEngine(datasets, config=None, use_csr=False) builds one, and SqlEngine.from_engine(engine) shares an existing engine's datasets, CSR and statistics, so both engines move together on checkout_latest(). Without a config it is plain SQL, and GRAPH_TABLE raises ValueError.
• Lance pushdown. Lance datasets are scanned in place at their pinned version, with column and filter pushdown, as Cypher queries scan them. SqlQuery still reads datasets into memory on each call. A namespace-opened Lance dataset can't be reopened by URI, so a graph-less SqlEngine still reads it into memory; with a config it raises ValueError, as CypherEngine does.
• Parameters and plan cache. execute and explain take one params dict for $name in the SQL and in every pattern. Patterns are planned as Cypher templates, so a whole SQL plan is cached per snapshot under the Cypher cache's rules: up to 128 entries, keyed by the text plus parameter names and types, with values bound before optimization. List and map values, and patterns that can't be templates, are planned on every call. A missing value raises ValueError at planning.
• Timeout. execute(..., timeout=...) uses the same cooperative deadline as CypherEngine.execute.
• Read-only. DDL, DML (INSERT, COPY ... TO) and statements such as SET are rejected, so SQL can't write files.
• SqlDialect moves to its own module to break an import cycle. It is still exported from lance_csr and lance_csr.sql, so imports don't change.
• Crate docs. Neither Rust crate calls its API experimental any more. lance-csr-catalog now explains what it is for (the table catalog the planner uses, and DirNamespace), with a doctest example. The lance-csr crate docs no longer say it translates Cypher into SQL.

Behaviour change for existing SqlEngine users: in-memory tables are now split into one partition per core, and Lance datasets are no longer read eagerly. Results are unchanged.

Joins written in plain SQL outside GRAPH_TABLE still run as ordinary joins. Lowering those to the CSR would need a separate join-to-CSR optimizer rule.

Validation

• cargo test --workspace: 693 passed, including new tests that check CSR and join-path results match Cypher, Lance filter pushdown, rejected writes, template reuse across values and types, uncached list parameters, and the cache reset on checkout_latest().
• cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --all --check: clean.
• make test: 1694 passed, including the new SqlEngine tests. The timeout test stops a nested-loop join at 0.2 s; without a deadline, the same query runs for more than 3 s.
• make lint (ruff and pyright): clean.
• Docs: page snippets run, zensical build --strict and typos pass.
• cargo doc still reports 6 broken intra-doc links. All are in code that was already on main, and the count is the same before and after this PR.
• Not benchmarked: this adds a new path and doesn't change how Cypher queries are planned or run. CI is pending.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying lance-csr with    Cloudflare Pages

Latest commit:	cfbc86d
Status:	✅  Deploy successful!
Preview URL:	https://f0d50e16.lance-csr.pages.dev
Branch Preview URL:	https://feat-sql-graph-table.lance-csr.pages.dev

View logs