Skip to content

Support ADO.NET schema data types and metadata probing - #363

Closed
alaricd wants to merge 27 commits into
Giorgi:mainfrom
pengdows:fix/timespan-parameter-without-target-type
Closed

alaricd wants to merge 27 commits into
Giorgi:mainfrom
pengdows:fix/timespan-parameter-without-target-type

Conversation

@alaricd

@alaricd alaricd commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add standard DataSourceInformation metadata needed by ADO.NET schema probing.
  • Add DataTypes metadata for DuckDB native types.
  • Align metadata column ordering, types, normalized versions, DBNull values, and restriction validation with the common ADO.NET contract.
  • Correct index restriction metadata.
  • Add focused schema coverage.

Validation

  • dotnet test DuckDB.NET.Test/Test.csproj --no-restore --filter FullyQualifiedName~SchemaTests -m:1
  • 23 tests passed.

Giorgi and others added 27 commits September 29, 2026 02:53
The test project does not enable nullable reference types, so the
reference-type annotations in the managed appender tests produced 18
CS8632 warnings. Without a nullable context they have no effect, so
removing them changes nothing; the int? value types stay.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
- DuckDBConnectionStringBuilder.TryGetValue now matches the base
  signature ([NotNullWhen(true)] out object? value); the base method
  assigns null when the key is missing (CS8601).
- DuckDBAppenderRow.AppendSpan checks for a span with no memory using
  Unsafe.IsNullRef instead of comparing it to null (CA2265). IsEmpty,
  as the analyzer suggests, would be wrong: a null byte[] or a default
  span writes NULL, but an empty array is an empty BLOB. A new test
  covers both cases.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
The test ordered its rows by an INTEGER column filled with
AppendValue(1). An integer literal binds to the byte? overload, so the
low-level appender wrote one byte into each INTEGER slot and left the
rest uninitialized; ORDER BY then sorted on garbage and the test failed
at random. A (int) cast does not help because the value is still a
constant; (int?) or a variable would.

The test now has only the BLOB column and orders by rowid.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
Rewrite the agent guidance against the current code: build, test and
native library commands, the DuckDB update and release process, the
architecture of connections, commands, readers and writers, the reader
invariants guarded by regression tests, coding style, and rules for
writing tests. Fixes outdated claims, such as streaming being the
default result mode.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
DuckDB closes a streaming result when a fetch reaches its end, and every
later fetch then fails with "Attempting to execute an unsuccessful or
closed pending query result". Since f94d52b the reader throws the
result's error after a NULL chunk, so a Read() after the end of a
streaming result threw, and so did the first Read() of an empty result,
whose end the constructor's fetch had already reached. The Arrow stream
had the same problem. 1.5.5 returned false in both cases.

DuckDBDataReader and DuckDBArrowArrayStream now remember a clean end of
stream and stop fetching. An error while producing a chunk still throws,
and the reader resets the flag when NextResult moves to a new result.

New tests cover Read() after the end, an empty result, NextResult after
reading past the end, and the Arrow stream, each in streaming and
materialized mode. Fixes Giorgi#358.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
artifacts.duckdb.org/<branch>/ serves the nightly shared libraries of a
DuckDB branch, and 'latest' is DuckDB's main, which is now 2.1
development. The fix for the wrong BIGNUM casts (duckdb/duckdb#25939)
was merged into v2.0-cyanoptera only, so the nightly kept failing the
same eight VarintTest cases against main.

Download from the v2.0-cyanoptera path until DuckDB 2.0 ships. Linux,
Windows and macOS archives are all published there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
NumericVectorDataWriter.AppendBigInteger always wrote a DuckDBHugeInt,
whatever the column type. On a UHUGEINT column a value of 2^127 or more
threw ArgumentOutOfRangeException, and a negative value was written as
its two's-complement bytes, so -1 was stored as a huge positive number.
UHUGEINT[] list and array columns were affected too.

Write a DuckDBUHugeInt when the column is UHUGEINT. Its constructor
checks the range 0 to 2^128 - 1, so out-of-range values now throw.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
Four tests need a streaming query to fail only after some rows were
read. With default settings that no longer happens on DuckDB 2.0: it
buffers past the failing row, discards the buffered rows when the error
occurs, and reports the error from ExecuteReader, before any row is
returned. On DuckDB 1.5 the same tests sometimes failed with
OperationCanceledException, because an error on a worker thread can be
reported as an interrupt.

The tests now use a private in-memory connection with threads=1 and a
100KB streaming buffer. One thread produces the chunks in order and can
get only a few chunks ahead of the reader, so the rows before the
failing one are read first on both versions. One thread alone was not
enough on 2.0. streaming_buffer_size is a per-connection setting, so it
is applied with SET; the connection string rejects it.

Verified on DuckDB 1.5.6 and on v2.0.0-alpha44255: the four tests pass
6 runs out of 6 on each.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
The mapped appender tests used only int, string, float, DateTime and
byte[] properties, so most branches of PropertyMapping.AppendToRow never
ran. A new test maps bool, sbyte, short, long, byte, ushort, uint, ulong,
double, decimal, DateTimeOffset, TimeSpan, Guid, BigInteger, DateOnly and
TimeOnly, plus a nullable int and a null string, and reads the values
back.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
Coverlet 6.0.0 counted the copy constructor the compiler generates for a
record without a primary constructor as uncovered code, so the record
line of ScalarFunctionOptions showed as missed
(coverlet-coverage/coverlet#1633). 10.1.0 no longer reports it. The
coverage report is still written to DuckDB.NET.Test/coverage.net8.0.info.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
The descriptor for a DuckDB configuration option reads its value from
the connection string as text and converts it to the option's type, and
writes values back as invariant text. Only the UBIGINT read and write
and one boolean write were tested.

New tests cover reading BOOLEAN, BIGINT, UBIGINT, DOUBLE and text
options, text that does not fit the type, empty text, writing under a
culture with a decimal comma, writing null, and the descriptor's
metadata (ComponentType, IsReadOnly, CanResetValue).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
Many typed configuration properties of DuckDBConnectionStringBuilder had
a getter or setter that no test used. A reflection-driven test now sets
each typed property to sample values, reads them back, and clears them,
so a property added later is covered too.

Further tests cover the spellings DefaultOrder accepts, malformed sort
order values, undefined enum values, and empty number and boolean
options.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
…UDFs

The mapped appender rejected properties of type DuckDBDateOnly and
DuckDBTimeOnly with "not supported for mapping", although the low-level
appender and the mapping code handle both. DuckDBDateOnly matters
because it can hold the infinite dates that DateOnly cannot.

Both types are now in the CLR-to-DuckDB type map, as DATE and TIME. The
map also decides the types of user-defined functions, so scalar
functions and table function columns and parameters accept them too.

Table function parameters needed a fix: DuckDBValue.GetValue<T> always
produced a DateOnly or TimeOnly and reinterpreted it as T without a
check, so GetValue<DuckDBDateOnly>() returned garbage. It now produces
the requested type for DATE and TIME values.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
DuckDBClientFactory overrode CreateCommandBuilder and CreateDataAdapter
to throw NotImplementedException. DbProviderFactory.CanCreateCommandBuilder
and CanCreateDataAdapter work by calling those methods and checking for
null, so the two properties threw too. Generic data code asks them
precisely to avoid an exception.

The overrides are removed. The base methods return null, as ADO.NET
specifies for a provider without these features, and both properties
now report false.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
Looking up a parameter by name through DbParameterCollection threw
ArgumentOutOfRangeException ("Index was out of range") when the name did
not exist, because GetParameter(string) used the index -1. The typed
indexer, the setter and RemoveAt already threw IndexOutOfRangeException
with "Parameter '<name>' not found"; the getter now does too.

New tests use the collection as DbParameterCollection, the way generic
data code does: AddRange(Array), Insert, Contains, IndexOf, both
indexers, CopyTo, enumeration and Remove.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
Add ducklabs-logo.png to the sponsors folder and point the README's
sponsor image at it instead of duckdb-labs-logo.png.

The image URL reads from main, so the logo appears only after this
change is merged there.
DuckDB's HUGEINT is a 128-bit two's complement integer, from -2^127 to
2^127 - 1. DuckDBHugeInt.HugeIntMinValue was -(2^127 - 1), so the
constructor rejected -2^127: the value could be read from DuckDB but not
written through a parameter or an appender.

The minimum is now -2^127. The constructor needed no other change: for
-2^127 the upper half is exactly -2^63 and the lower half is 0.

SimpleNegativeHugeIntTest builds its SQL from the constant and needed
parentheses, because a cast binds tighter than the minus sign and 2^127
itself is not a HUGEINT. A new test writes both ends of the range as
parameters and through the appender, and checks that one step outside
is rejected.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
A BigInteger parameter was always converted to a signed DuckDBHugeInt,
so a value of 2^127 or more threw ArgumentOutOfRangeException even when
DuckDB reported the parameter's type as UHUGEINT, for example when it
was inserted into or compared with a UHUGEINT column.

Such a parameter is now built as a DuckDBUHugeInt, which accepts 0 to
2^128 - 1 and rejects anything else.

Parameters for which DuckDB reports no type, such as "$1::UHUGEINT",
still go through the signed conversion.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
When DuckDB reports no type for a parameter, as in "SELECT $1" or
"$1::UHUGEINT", a BigInteger was always converted to a signed
DuckDBHugeInt. A value outside the HUGEINT range threw
ArgumentOutOfRangeException before the query ran, even when the query
cast it to UHUGEINT.

Such a parameter is now bound as HUGEINT if the value fits, as UHUGEINT
for values up to 2^128 - 1, and as BIGNUM beyond that. Values that fit
HUGEINT bind exactly as before.

This is a behaviour change for larger values: they used to throw and are
now sent to DuckDB, which applies any cast in the query.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
The validation required the single DuckDB type a .NET type maps to, so a
BigInteger property was accepted only for HUGEINT. It now checks the column
against the types the property can be appended to: HUGEINT and UHUGEINT for
BigInteger, the mapped type for everything else. The range is checked per
value when it is appended.

The mismatch error names every accepted type and prints the underlying type
of a nullable property.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
CreateAppender<T, TMap> created the low-level appender and passed it to the
DuckDBMappedAppender constructor. When the constructor threw (no mappings,
wrong column count, type mismatch), nothing disposed the appender, and its
native objects stayed alive until the finalizer ran.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzLKjBGAwccxYB6ny5bpmY
DuckDB's interval micros is signed (duckdb_interval.micros is int64_t) and days can be negative, but
DuckDBInterval converted both as unsigned: reading any negative interval as a TimeSpan threw
ArgumentOutOfRangeException ("INTERVAL '-90 minutes'", even "-2 days"), a negative month was silently
dropped instead of refused, and binding or appending a negative TimeSpan threw OverflowException.

Micros stays a ulong for compatibility and is read and written as its two's complement. A non-zero
month count, either sign, is refused, and the range check is done in Int128.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TimeSpan has no DbType, so where DuckDB can't infer a parameter's type (a bare "?") it was sent as
TimeSpan.ToString(): it came back as a string, and DuckDB can't parse that text as an interval once
there are days ("1.01:01:01"). It is now bound as an INTERVAL there. Where DuckDB knows the type
(an INTERVAL, TIME or VARCHAR target) nothing changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Where DuckDB can't infer the type and the value ends up as a TIME (INSERT ... SELECT ?, a UNION with a
TIME, ?::TIME::VARCHAR), the new interval default fails, since DuckDB has no INTERVAL -> TIME cast. As in
Npgsql, a caller who wants a time declares DbType.Time: the interval arm stays out of the way, and the
DbType.Time value creator accepts a TimeSpan through TimeOnly.FromTimeSpan, which refuses a negative value
or one of a day or more (it threw InvalidCastException before).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alaricd

alaricd commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Replaced by clean schema-only draft PR #364, based directly on the latest upstream main.

@alaricd alaricd closed this Oct 14, 2026

This branch is waiting to be deployed

1 waiting deployment
external — 43e8e257 Waiting Oct 14, 2026 by alaricd via Authorize #645
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants