activation: Add XPU backend to kernels-community/activation - #1071
Open
jiqing-feng wants to merge 4 commits into
Open
activation: Add XPU backend to kernels-community/activation#1071jiqing-feng wants to merge 4 commits into
kernels-community/activation#1071jiqing-feng wants to merge 4 commits into
Conversation
jiqing-feng
force-pushed
the
activation_xpu
branch
2 times, most recently
from
August 10, 2026 03:43
0854a71 to
8b437ac
Compare
Implement the eleven activation operators in SYCL and register them under the XPU dispatch key, so existing callers work unchanged on Intel GPUs. Previously the kernel declared only cuda, rocm, metal and cpu, and could not be loaded on XPU at all. Gated operators map one work-group to one token; element-wise operators use a flat grid-stride loop. Both use 128-bit vectorized accesses with a runtime alignment check and a scalar fallback, which is what keeps the element-wise operators on par with eager PyTorch, whose TensorIterator already vectorizes. Parameterize tests/kernels/test_activation.py over the available accelerator rather than hard-coding CUDA, so the existing suite runs unmodified on XPU. It passes in full on Intel Arc Pro B60 with torch 2.13.0+xpu, including the gated operators that are compared against eager PyTorch with zero tolerance. Add two cases to the same file for shapes and alignments that force the scalar path, and for empty inputs. Requires huggingface/kernels#752, which makes kernel-builder request IEEE correctly rounded fp32 division for SYCL kernels; without it silu_and_mul and mul_and_silu differ from eager PyTorch by one ULP.
jiqing-feng
marked this pull request as ready for review
August 14, 2026 06:53
Contributor
Author
|
huggingface/kernels#752 merged. This PR is ready to be reviewed. |
# Conflicts: # activation/tests/kernels/test_activation.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a SYCL implementation of the
activationkernels so the component can beused on Intel GPUs. All eleven operators exposed by the CUDA backend are
implemented and registered under the
XPUdispatch key, so existing callerswork unchanged on XPU devices.
Previously
activationdeclared onlycuda,rocm,metalandcpu. On XPUthe kernel could not be loaded at all, forcing every consumer to carry a
separate eager fallback path.
Operators:
silu_and_mul,mul_and_silu,gelu_and_mul,gelu_tanh_and_mul,fatrelu_and_mul,gelu_new,gelu_fast,gelu_quick,silu,gelu,gelu_tanh. Supported dtypes:float32,float16,bfloat16.New sources live in
activation_xpu/, following the layout of the existing CPUbackend.
tests/kernels/test_activation.pyis parameterized over the availableaccelerator instead of hard-coding CUDA, so the existing suite runs unmodified
on XPU.
Depends on huggingface/kernels#752, which makes
kernel-builderrequest IEEEcorrectly rounded fp32 division for SYCL kernels.
Correctness
Validated on Intel Arc Pro B60 with
torch 2.13.0+xpuand oneAPI 2026.0.The existing
tests/kernels/test_activation.pypasses in full on XPU (420cases), including the gated operators that are compared against eager PyTorch
bit-for-bit with zero tolerance. Two cases were added to that file, covering
shapes and alignments that force a vectorized backend onto its scalar path, and
empty inputs. Both are device-parameterized like the rest of the suite.
Performance
Measured on one Intel Arc Pro B60. Speedup is eager time divided by kernel time;
values above 1.00 favour the kernel. Gated operators use a
16384 x 16384input, element-wise operators
16384 x 8192; the4096columns are the samemeasurement at
4096 x 8192and4096 x 4096.silu_and_mulmul_and_silugelu_and_mulgelu_tanh_and_mulfatrelu_and_mulgelu_newgelu_fastgelu_quicksilugelugelu_tanhThree regimes are visible:
[n, d]intermediates and cut memory trafficfrom
6ndto3nd.fatrelu_and_mulgains more because its eager formadditionally materializes a boolean mask.
eight or more element-wise launches in eager mode, so fusing them gives the
largest speedups.
corresponding PyTorch operator and are therefore on par with it. They exist
for interface completeness, so callers need no device-dependent branch.
Memory
Peak memory allocated while producing the result, via
torch.xpu.max_memory_allocated. The output tensor is included and the inputexcluded, so both paths are charged for what they produce. The ratio is
independent of dtype and shape; absolute figures are
float16at16384 x 16384(gated) and16384 x 8192(element-wise).silu_and_mul,mul_and_silu,gelu_and_mul,gelu_tanh_and_mul,fatrelu_and_mulgelu_quickgelu_newgelu_fastsilu,gelu,gelu_tanhThe fused kernels never allocate beyond the output tensor. Eager
gelu_fastpeaks at four times that because three intermediates are live simultaneously.
Build
Both XPU variants offered by
kernel-builderbuild cleanly and pass themanylinux_2_28and Pythonabi3compatibility checks:Limitations