You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Shredded variant readers throw InvalidCastException for large_binary and binary_view storage #448
Describe the bug, including details regarding any error messages, version, and platform.
VariantArray accepts binary, large_binary or binary_view for its metadata and value fields, at the top level and inside shredded element groups (VariantExtensionDefinition.FindBinaryFieldIndex, ShredSchema.IsElementGroupStruct). ShredSchema.FromArrowType also maps large_utf8 and large_binarytyped_value columns.
The shredding-aware readers in Apache.Arrow.Operations.Shredding (ShreddedVariant, ShreddedObject, ShreddedArray) cast those columns to BinaryArray / StringArray, so any other representation throws InvalidCastException. This affects:
GetLogicalVariantValue on any such column, shredded or not. For an unshredded large_binary column the core VariantArray.GetVariantValue works, but GetLogicalVariantValue throws.
ShreddedVariant.GetString() / GetBinaryBytes() on large_utf8 / large_binary typed columns.
Residual reads in ShreddedObject / ShreddedArray, TryGetResidualReader, etc.
Expected: the readers accept every storage type that VariantArray and ShredSchema accept.
Found while reviewing #447, whose new array-level entry points go through these readers.
Activity
fix: Read large_binary and binary_view storage in shredded variant re…