Skip to content

Register activation kernel layers on XPU - #47858

Draft
jiqing-feng wants to merge 1 commit into
huggingface:mainfrom
jiqing-feng:activation-xpu-layers
Draft

Register activation kernel layers on XPU#47858
jiqing-feng wants to merge 1 commit into
huggingface:mainfrom
jiqing-feng:activation-xpu-layers

Conversation

@jiqing-feng

@jiqing-feng jiqing-feng commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

CI

Summary

kernels-community/activation now ships an XPU backend, so map the six layers
it provides to XPU in addition to CUDA. They were previously registered for
CUDA only, which left XPU on the eager fallback.

Affected entries in _KERNEL_MAPPING: FastGELU, QuickGELU, NewGELU,
SiLU, GeLU, GeluTanh. Each reuses the CUDA repository, layer name and
version, under Mode.INFERENCE | Mode.TORCH_COMPILE.

FastGELU, NewGELU and QuickGELU have no native PyTorch operator and
expand into several element-wise launches in eager mode, so the fused kernel is
several times faster. SiLU, GeLU and GeluTanh read and write the same
amount of data as the corresponding PyTorch operator and are on par with it;
they are mapped for parity with CUDA.

Validation

Checked on Intel Arc Pro B60 with torch 2.13.0+xpu. All six layers run in
eager mode and under torch.compile, with compiled and eager results
identical:

Layer speedup over eager max diff vs PyTorch reference
FastGELU 6.20x 1.95e-03
NewGELU 5.40x 1.95e-03
QuickGELU 3.44x 1.95e-03
SiLU 0.98x 0.00e+00
GeLU 0.98x 0.00e+00
GeluTanh 1.18x 0.00e+00

Speedups are float16 at 16384 x 8192. The differences for the three
composite activations are inherent to their approximations, which do not match
the PyTorch operator they are compared against, and are the same on CUDA.

Dependencies

Requires huggingface/kernels-community#1071, which adds the XPU backend to
activation, and huggingface/kernels#752, which that backend depends on. Both
must be merged and an activation build carrying the XPU variant published on
the Hub before this mapping resolves.

kernels-community/activation now ships an XPU backend, so map the six layers
it provides to XPU as well. They were previously registered for CUDA only,
which left XPU on the eager fallback.

FastGELU, NewGELU and QuickGELU have no native PyTorch operator and expand
into several element-wise launches in eager mode, so the fused kernel is
several times faster. SiLU, GeLU and GeluTanh are on par with eager and are
mapped for parity with CUDA.
@github-actions

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 31353982233:2
Result: success | Jobs: 16 | Tests: 166,813 | Failures: 0 | Duration: 15h 59m

@Rocketknight1

Copy link
Copy Markdown
Member

cc @IlyasMoutawwakil

@IlyasMoutawwakil

Copy link
Copy Markdown
Member

@jiqing-feng tell me when it's ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants