[BUG] Binary fields return 500 or silently wrong results when a pushdown references them

[cnoramut]

Query Information

PPL Command/Query:

source=type_probe | stats count() by bin
source=type_probe | sort bin
source=type_probe | where isnotnull(bin) | fields bin

Expected Result: all three succeed. Four buckets, four ordered rows, and four matching documents, since every document has bin populated.

Actual Result: the first two return HTTP 500 from a shard-level failure. The third returns HTTP 200 with zero rows, which is worse, because nothing signals that the answer is wrong.

IllegalArgumentException: Can't load fielddata on [bin] because fielddata is
unsupported on fields of type [binary]. Use doc values instead.

The response reports "code": "UNKNOWN", "type": "RuntimeException", and "context": {"stage": "executing"}, so it names neither the field nor a cause a user could act on.

Ten cells fail, across three pushdown paths.

Command	Result	Path
stats count() by bin	500	aggregation
top 2 bin	500	aggregation
rare 2 bin	500	aggregation
dedup bin	500	aggregation
timechart span=1m count() by bin	500	aggregation
chart count() over m by bin	500	aggregation
sort bin	500	sort
xyseries m bin IN ('x','y') c	500	aggregation, through the agg filter argument
where bin = 'zzz'	500	filter
where isnotnull(bin)	200, zero rows	filter

eventstats count() by bin, fields bin, cast(bin as string), and concat(bin, 'x') all succeed. upper(bin) returns a correct 400 naming the type. So binary is handled everywhere except where a pushdown touches it.

Dataset Information

Dataset/Schema Type

• OpenTelemetry (OTEL)
• Simple Schema for Observability (SS4O)
• Open Cybersecurity Schema Framework (OCSF)
• Custom (details below)

Index Mapping

Only bin is under test. @timestamp is required by timechart span=1m, and m and c by chart and xyseries.

{
  "mappings": {
    "properties": {
      "@timestamp": { "type": "date" },
      "m":          { "type": "keyword" },
      "c":          { "type": "double" },
      "bin":        { "type": "binary" }
    }
  }
}

Sample Data

{"@timestamp":"2026-09-09T10:00:00Z","m":"alpha","c":1.5,"bin":"U29tZUJpbmFyeQ=="}
{"@timestamp":"2026-09-09T10:00:30Z","m":"alpha","c":2.5,"bin":"QW5vdGhlckJpbg=="}
{"@timestamp":"2026-09-09T10:01:00Z","m":"beta","c":3.5,"bin":"VGhpcmRCaW4="}
{"@timestamp":"2026-09-09T10:02:00Z","m":"beta","c":4.5,"bin":"Rm91cnRoQmlu"}

Bug Description

Issue Summary

A binary field has no fielddata and no doc values, so OpenSearch cannot bucket, sort, or filter on it. Three Calcite pushdown paths build a request that references the field anyway, and the query then fails at the shard. One of those paths fails silently and returns a wrong answer rather than an error.

This is a pushdown bug, not a type-support gap. With plugins.calcite.pushdown.enabled=false all ten queries above succeed and return correct results. So the engine already handles binary correctly and only the pushed-down form breaks.

Steps to Reproduce

# 1. Index with a binary field
curl -s -XPUT "localhost:9200/type_probe" -H 'Content-Type: application/json' -d '{
  "mappings": { "properties": {
    "@timestamp": { "type": "date" },
    "m": { "type": "keyword" }, "c": { "type": "double" },
    "bin": { "type": "binary" } } } }'

curl -s -XPOST "localhost:9200/type_probe/_bulk?refresh=true" \
  -H 'Content-Type: application/x-ndjson' --data-binary '
{"index":{}}
{"@timestamp":"2026-09-09T10:00:00Z","m":"alpha","c":1.5,"bin":"U29tZUJpbmFyeQ=="}
{"index":{}}
{"@timestamp":"2026-09-09T10:00:30Z","m":"alpha","c":2.5,"bin":"QW5vdGhlckJpbg=="}
{"index":{}}
{"@timestamp":"2026-09-09T10:01:00Z","m":"beta","c":3.5,"bin":"VGhpcmRCaW4="}
{"index":{}}
{"@timestamp":"2026-09-09T10:02:00Z","m":"beta","c":4.5,"bin":"Rm91cnRoQmlu"}
'

# 2. Defaults are calcite on, pushdown on. Confirm with
curl -s "localhost:9200/_cluster/settings?flat_settings=true&include_defaults=true" | grep calcite

# 3. Each of these returns 500
curl -s -XPOST "localhost:9200/_plugins/_ppl" -H 'Content-Type: application/json' \
  -d '{"query":"source=type_probe | stats count() by bin"}' -w '\nHTTP %{http_code}\n'
curl -s -XPOST "localhost:9200/_plugins/_ppl" -H 'Content-Type: application/json' \
  -d '{"query":"source=type_probe | sort bin"}' -w '\nHTTP %{http_code}\n'

# 4. This one returns HTTP 200 with zero rows, though all four documents have bin
curl -s -XPOST "localhost:9200/_plugins/_ppl" -H 'Content-Type: application/json' \
  -d '{"query":"source=type_probe | where isnotnull(bin) | fields bin"}' -w '\nHTTP %{http_code}\n'

# 5. Turn pushdown off and every one of them succeeds with correct results
curl -s -XPUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' \
  -d '{"transient":{"plugins.calcite.pushdown.enabled":false}}'

Why existing guards do not catch it

Atomic and aggregatable are different properties, and only the first is checked. geo_point becomes GEOMETRY, is not atomic, and is correctly rejected by the guard added in #5751. binary maps to ExprCoreType.UNKNOWN in OpenSearchDataType, reads as atomic, and passes straight through.

The two types on identical commands, measured on this fixture.

Command	geo_point	binary
timechart	400, Cannot chart by [geo] because it holds multiple values	500
chart	400, same guard	500
xyseries	400, y-name-field must be a scalar type, got: GEOMETRY	500
upper	400, signature check	400, signature check
stats top rare dedup sort	500	500

Impact

Any saved query or dashboard panel that groups, sorts, or filters on a binary field fails with a 500 that names no field and carries code: UNKNOWN, so a user cannot tell which field to change. The where isnotnull case is more serious, because it returns 200 with silently incomplete results.

Root Cause

Three pushdown sites accept a field reference without asking whether OpenSearch can read that field from the index.

CalciteLogicalIndexScan.pushDownAggregate already declines when a bucket is ARRAY, but has no equivalent for binary.

// opensearch/src/main/java/org/opensearch/sql/opensearch/storage/scan/CalciteLogicalIndexScan.java:407
List<String> bucketNames = outputFields.subList(0, aggregate.getGroupSet().cardinality());
if (bucketNames.stream()
    .map(b -> fieldTypes.get(b))
    .filter(Objects::nonNull)
    .anyMatch(expr -> expr.getOriginalType() == ExprCoreType.ARRAY)) {

AbstractCalciteIndexScan.pushDownSort builds a fieldSort with no type check.

// opensearch/src/main/java/org/opensearch/sql/opensearch/storage/scan/AbstractCalciteIndexScan.java:395
// Keyword field is optimized for sorting in OpenSearch
ExprType fieldType = osIndex.getFieldTypes().get(fieldName);
String field = OpenSearchTextType.toKeywordSubField(fieldName, fieldType);
sortBuilder = SortBuilders.fieldSort(field).missing(missing);

CalciteLogicalIndexScan.pushDownFilter pushes the predicate without inspecting the types it references, which is where the silently wrong zero rows come from.

// opensearch/src/main/java/org/opensearch/sql/opensearch/storage/scan/CalciteLogicalIndexScan.java:171
Map<String, ExprType> fieldTypes = this.osIndex.getAllFieldTypes();
QueryExpression queryExpression =
    PredicateAnalyzer.analyzeExpression(
        filter.getCondition(), schema, fieldTypes, rowType, getCluster());

pushDownCollapse in the same class already carries the right shape of guard, so the precedent exists in the file.

// opensearch/src/main/java/org/opensearch/sql/opensearch/storage/scan/CalciteLogicalIndexScan.java:241
if (!ExprCoreType.numberTypes().contains(originalExprType)
    && !originalExprType.legacyTypeName().equals("KEYWORD")
    && !originalExprType.legacyTypeName().equals("TEXT")) {

xyseries reaches the aggregate site rather than the filter site. Its bucket is m and bin appears only inside the aggregate call's filter argument, x_c=MAX($0) FILTER $2, so a guard on bucket names alone does not see it.

Proposed fix

Decline the pushdown rather than reject the query, since the in-memory path already produces correct results.

Measured on a 50000 document index with one shard, five groups of known uneven size plus a marker document indexed last whose value sorts first. Declining the pushdown returned all six groups with exact counts, 20000, 15000, 10000, 4000, 999, 1, identical to the same query on an equivalent keyword field. sort returned the marker first, so the full document set is seen rather than a truncated prefix. plugins.query.size_limit caps output rows and not documents scanned, so aggregate values stay correct above the limit.

The cost is aggregation latency. On that index stats count() by took 0.03s pushed down against 0.11s declined, roughly 3.7x, with no measurable difference on sort. Not measured above 50000 documents.

A 400 would be the wrong remedy here, because it would reject queries the engine answers correctly. That differs from #5751, where charting by an object field has no correct answer at any layer and rejecting was right.

1. In pushDownAggregate, return null when a bucket field is binary, as a sibling of the existing ARRAY refusal.
2. In pushDownSort, return null when the sort field is binary.
3. In pushDownFilter, return null when the condition references a binary field.
4. In pushDownAggregate, also inspect the Project it receives, so xyseries filter arguments are covered.

Key the check on MappingType.Binary rather than on ExprCoreType.UNKNOWN. text, match_only_text, geo_point, and alias all share UNKNOWN, and text pushdown currently works through its .keyword subfield.

Unaffected. Every other type keeps its current pushdown, eventstats, fields, cast, and concat on binary already work and stay working, and upper(bin) keeps its 400 because that rejection comes from the function signature and not from a pushdown.

Environment Information

OpenSearch Version: 3.9.0-SNAPSHOT, built from main at 64088734f.

Additional Details

Engine settings were all at their build defaults, plugins.calcite.enabled=true, plugins.calcite.pushdown.enabled=true, plugins.calcite.fallback.allowed=false.

Type matrix proposed in #5754. 9 of 23 500s in that run came from this one type.

Two related observations that are not part of the fix above.

• Equality on a binary field fails independently of pushdown. where bin = 'zzz' also fails with pushdown disabled, with Cannot cast "java.lang.String" to "org.apache.calcite.avatica.util.ByteString". That is an engine-level comparison bug of the kind tracked in [BUG] Janino compile failures return the query plan in the error response #5753. Declining the pushdown for that cell exposes it rather than fixing it, so the error message changes without the cell going green.
• The aggregate and sort paths have no bucket-type guard at all. stats, top, rare, dedup, and sort also return 500 on geo_point for the same structural reason. That broader gap overlaps [BUG] Dedup crashes with ClassCastException #5614 and is not covered here.