Skip to content

Shredded variant readers throw InvalidCastException for large_binary and binary_view storage #448

Closed

Description

@CurtHagenlocher

Describe the bug, including details regarding any error messages, version, and platform.

VariantArray accepts binary, large_binary or binary_view for its metadata and value fields, at the top level and inside shredded element groups (VariantExtensionDefinition.FindBinaryFieldIndex, ShredSchema.IsElementGroupStruct). ShredSchema.FromArrowType also maps large_utf8 and large_binary typed_value columns.

The shredding-aware readers in Apache.Arrow.Operations.Shredding (ShreddedVariant, ShreddedObject, ShreddedArray) cast those columns to BinaryArray / StringArray, so any other representation throws InvalidCastException. This affects:

  • GetLogicalVariantValue on any such column, shredded or not. For an unshredded large_binary column the core VariantArray.GetVariantValue works, but GetLogicalVariantValue throws.
  • ShreddedVariant.GetString() / GetBinaryBytes() on large_utf8 / large_binary typed columns.
  • Residual reads in ShreddedObject / ShreddedArray, TryGetResidualReader, etc.

Expected: the readers accept every storage type that VariantArray and ShredSchema accept.

Found while reviewing #447, whose new array-level entry points go through these readers.

Component(s)

C#

Activity

  1. added a commit that references this issue on Sep 27, 2026
    d952f30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions