[WIP] Stack/pipelined shuffle pr8 rtm - #57692

Open

jerrypeng wants to merge 16 commits into

apache:masterfrom

jerrypeng:stack/pipelined-shuffle-pr8-rtm

Commits on Jul 30, 2026

[SPARK-XXXXX][CORE] Register a pipelined shuffle with the streaming output tracker
jerrypeng
committed
[SPARK-XXXXX][CORE] Unregister a pipelined shuffle from the streaming output tracker on cleanup
jerrypeng
committed
[SPARK-XXXXX][CORE] Fully separate pipelined shuffles from the MapOutputTracker
jerrypeng
committed
[SPARK-XXXXX][CORE] Harden the MapOutputTracker decoupling: fail loud (before any state mutation) with no streaming tracker, and abort the group on a base-path pipelined FetchFailed
jerrypeng
committed
[SPARK-XXXXX][CORE] Fix scalastyle line length in the fail-loud routing test
jerrypeng
committed
[SPARK-XXXXX][CORE] Address review: fix comment line lengths and a stale availability comment
jerrypeng
committed
[SPARK-XXXXX][CORE] Address post-merge review comments from #57361
jerrypeng
committed
[SPARK-XXXXX][CORE] Select the output tracker by shuffle-dependency type via a common interface
jerrypeng
committed
[SPARK-XXXXX][CORE] Add leak-coverage tests for pipelined-shuffle cleanup
jerrypeng
committed
[SPARK-XXXXX][CORE] Balance shuffleDriverComponents.removeShuffle for a pipelined shuffle on cleanup
jerrypeng
committed

Commits on Aug 1, 2026