Carry the partial_result override off ThreadContext so it survives the security handoff

[ahkcs]

Related

• Issue: [AUTOCUT] Integration Test Failed for sql-3.9.0 #5739 — partial-result warnings gate lost across the security thread handoff
• Previous fix: Fix partial-result warnings gate lost across the security thread handoff (#5739) #5743 (fixes [AUTOCUT] Integration Test Failed for sql-3.9.0 #5739) — moved the warningsSupported gate off Log4j ThreadContext onto the object graph

This PR is a direct follow-up: the same transport→worker ThreadContext handoff bug, for the sibling partial_result override that #5743 left behind.

Description

The per-request partial_result override (the partial_result boolean in the PPL request body, e.g. from the Dashboards toggle) was stored in Log4j ThreadContext via QueryContext.setPartialResultOverride. It is set on the transport thread (TransportPPLQueryAction) and read on the worker/planning thread by CalciteLogicalIndexScan.tryPartialResultAggregate → QueryContext.isPartialResultEnabled.

The security plugin's transport interceptor does not preserve Log4j ThreadContext across the transport→worker handoff. So on a security-enabled cluster the override is dropped before the worker thread reads it, and isPartialResultEnabled silently falls back to the cluster setting. A request that sends partial_result: false to opt out therefore still returns a partial result with a PARTIAL_RESULT warning whenever the cluster setting plugins.query.partial_result.on_mapping_conflict.enabled is on.

Only the override is affected — the cluster-setting path is read from osIndex.getSettings() on the worker thread, so it works. The bug is invisible without the security plugin (a plain ./gradlew run cluster preserves the ThreadContext), which is why it wasn't caught earlier.

This is the same transport→worker handoff bug fixed in #5743 (issue #5739) for warningsSupported. That change moved warningsSupported onto the object graph but left the partial_result override on ThreadContext, reasoning the settings fallback made it resilient — but the fallback masks the drop rather than surviving it.

Fix

Carry the override the same way warningsSupported is carried, off ThreadContext and onto the object graph:

PPLQueryRequest.partialResult (set from the request on the transport thread) → AbstractPlan.partialResultOverride → QueryPlan.execute() (runs on the worker thread) → CalcitePlanContext.setPartialResultOverride(...). It is included in CalcitePlanContext's thread-local snapshot (so it survives the planner's own complex-worker handoff) and reset per query. tryPartialResultAggregate now reads the override from CalcitePlanContext and falls back to the cluster setting. The now-dead QueryContext.setPartialResultOverride/isPartialResultEnabled (and the transport-side clear) are removed.

Testing

Added PartialResultSecurityIT.perRequestPartialResultFalseOverridesClusterSettingUnderSecurity — with the cluster setting on, an authenticated user sends partial_result: false and must get the complete result (no warning). Run under integTestWithSecurity:

test	before this change	after
perRequestPartialResultFalseOverridesClusterSettingUnderSecurity	❌ FAIL — expected:<3> but was:<2> (override dropped → partial)	✅ PASS
partialResultWarningSurvivesSecurityHandoff	✅	✅
completeResultCarriesNoWarningWithSecurity	✅	✅

QueryPlanTest and PartialResultAggregatePushdownTest also pass; :opensearch-sql-plugin:compileJava + :integ-test:compileTestJava clean.

Check List

• New functionality includes testing.
• Commits are signed per the DCO.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ No major issues detected

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Reset legacyPreferredFlag in cleanup method The legacyPreferredFlag ThreadLocal is not being cleared in clearTimewrapSignals(), which could cause state leakage between queries. Since this method clears all other ThreadLocals, legacyPreferredFlag should also be reset to its default value to prevent unintended behavior in subsequent queries. core/src/main/java/org/opensearch/sql/calcite/CalcitePlanContext.java [276-284] public static void clearTimewrapSignals() { stripNullColumns.set(false); timewrapUnitName.set(null); timewrapSeries.set(null); executionPool.set(null); pendingWarnings.remove(); warningsSupported.set(false); partialResultOverride.remove(); + legacyPreferredFlag.set(true); } Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies that legacyPreferredFlag is not being cleared in clearTimewrapSignals() while other ThreadLocals are. This could lead to state leakage between queries on pooled threads. However, the impact depends on whether legacyPreferredFlag is intentionally persistent across queries or if it's set per-query elsewhere in the codebase.	Medium

[RyanL1997]

I'm just thinking of a mechanism to catch the changes like this which will potentially impact the security path. With the current CI setup I think there is still a gap for us to test with security enabled.