Skip to content

fix(binder): map COPY FROM subquery columns by position - #1374

Merged
adsharma merged 2 commits into
LadybugDB:mainfrom
Tyagiquamar:fix/1365-copy-subquery
Oct 24, 2026
Merged

adsharma merged 2 commits into
LadybugDB:mainfrom
Tyagiquamar:fix/1365-copy-subquery

Conversation

@Tyagiquamar

Copy link
Copy Markdown
Contributor

Fixes #1365

Problem

COPY <table> FROM (subquery) raised a spurious NULL primary key error whenever the subquery RETURN list reused a variable:

CREATE NODE TABLE q3(ID INT64 PRIMARY KEY, a INT64, b INT64);
COPY q3 FROM (UNWIND range(20, 22) AS i RETURN i, i, i);
-- Copy exception: Found NULL, which violates the non-null constraint of the primary key column.

Binder::bindQueryScanSource renamed the subquery result columns in place with setAlias. RETURN i, i, i binds all three positions to the same Expression object, so that object ended up holding only the last expected name (b). matchColumnExpression then looked the primary key column up by name, found nothing, and fell back to the column default, which is NULL for a key. The same defect inserts NULL instead of the value for any reused non-key column, which is why PR #1350's regression test had to write i + 0.

BoundStatementResult already documents why the name cannot be recovered from the expression: one column object can carry several names.

Fix

Scan sources already align their columns with the expected schema by position. The file, object, table function and parameter binders name their bound columns from expectedColumnNames index by index and validate the count, and bindQueryScanSource validates the count too. So the copy binder now resolves each table property against expectedColumnNames by position and no longer mutates the bound subquery expressions.

Two cases keep matching by name, exactly as before:

  • columns past the expected names, which are the IGNORE_ERRORS copy warning columns;
  • a source bound without expected names, i.e. a table whose only property is SERIAL (dropped by skipPropertyInSchema), where the names the reader produced are all there is to match against.

For every source that goes through the expected-schema renaming, positional and name resolution select the same column, so the change is only observable for a query source whose RETURN reuses an expression.

Test

test/test_files/issue/issue1365.test covers node copy with RETURN i, i, i and RETURN i, 'Grace', i, the two workarounds from the issue, an aliased RETURN list, COPY r2 (a, ID) FROM (subquery) where the property order differs from the source order, and a relationship copy with a reused variable.

Validation

No local build was run, this machine has no configured Ladybug C++ build, so the evidence comes from the CI Workflow dispatched on my fork. The workflow has cancel-in-progress: true per ref, so the two steps were dispatched on separate refs.

  • Pre-fix, test file only at 75033d8: https://github.com/Tyagiquamar/ladybug/actions/runs/40061763673. minimal test (linux) fails with exactly one of 3181 tests, e2e_test_issue~issue1365.Issue1365CopyFromSubqueryDuplicateVariable, reporting Copy exception: Found NULL, which violates the non-null constraint of the primary key column. for both node cases, and rows 1| / 2| instead of 1|1 / 2|2 for COPY r2 (a, ID).
  • Post-fix at 890feb0: https://github.com/Tyagiquamar/ladybug/actions/runs/40064897262. sanity checks, clang-format and minimal test (linux) pass with 100% tests passed, 0 tests failed out of 3181. The minimal linux extension test job has no cloud-storage secrets on a fork, so it says nothing about this change; upstream CI will run it.

An intermediate revision of this fix asserted that the source column count equals the expected column count. CI caught that it is not true for IGNORE_ERRORS copies, where warning columns are appended, and for a table whose only column is SERIAL, where there are no expected names at all. That is why the two cases above are handled explicitly.

Formatting was checked with clang-format 19 locally; the repo pins 18, and the clang-format job above is the authoritative check.

Tyagiquamar added 2 commits October 24, 2026 09:41
Adds issue1365.test, which copies node and relationship tables from
subqueries whose RETURN list reuses one variable. Expected values follow
the positional mapping of result columns onto the table schema.

Signed-off-by: Tyagiquamar <347892697+Tyagiquamar@users.noreply.github.com>
Binder::bindQueryScanSource renamed the subquery result columns in place with
setAlias. A RETURN list that binds the same variable twice resolves both
positions to one shared Expression object, so the object kept only the last
expected name and the copy binder, which looked columns up by that name, fell
back to the column default. A reused primary key column then raised a NULL PK
violation and a reused non-key column was inserted as NULL.

Scan sources name their leading columns from the expected schema by position
(file, object and parameter readers do this while validating the count, and
bindQueryScanSource validates the count too), so bind each table property
against that position and stop mutating the bound subquery expressions. Columns
beyond the expected names are copy warnings and a source bound without expected
names keeps its own, so both still resolve by name.

Signed-off-by: Tyagiquamar <347892697+Tyagiquamar@users.noreply.github.com>
@adsharma
adsharma merged commit 134dcda into LadybugDB:main Oct 24, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: COPY FROM subquery reusing same variable fails with spurious NULL PK (binder alias collision)

2 participants