Repository navigation
Conversation
…ine libs with single-context vision attention (IRON#138) Vendors the CI-built gemma4e_flash, gemma4e_dflash and gemma4e_npu libraries (XRT and HRX, Linux .so and Windows .dll/.lib) from the PR Check run https://github.com/AMD-ROCm-Internal/FastFlowLM_IRON/actions/runs/41388386154 (AMD-ROCm-Internal/FastFlowLM_IRON#138, all checks green). Models: gemma4e-flash:e2b/e4b, gemma4-it:e2b/e4b, nemotron3.5-bolt:e4b, nemotron3.5-bolt-dflash:e4b. Same change as #919 (IRON#137): the vision encoder reconfigures the array in-stream inside the vision_mm hw_context instead of switching between the vision_mm and vision_attn hw_contexts twice per layer. No xclbin changes. Linux + XRT only inside the libs; FLM_VISION_SINGLE_CTX=0 selects the old path. Local XRT, 30 random image requests per model, old vs new path: identical text on all six models; median server TTFT gemma4e-flash:e2b 0.482 -> 0.407 s (-15.6%), e4b 0.620 -> 0.552 s (-11.0%); gemma4-it e2b -3.7%, e4b -4.7%; nemotron3.5-bolt -3.9%; nemotron3.5-bolt-dflash -3.5%. The Linux XRT libs vendored here are bit-identical to the ones tested locally. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Consolidated into #919 (will vendor all four Gemma4e engines from the CI libs of IRON#137). Closing. |
tawei-amd
pushed a commit
that referenced
this pull request
Nov 3, 2026
…ndor all four Gemma4e engine libs with single-context vision attention (IRON#137) CI-built libs from IRON PR Check run 41475496041 (head 38bb0332d): gemma4e_dflash_flash, gemma4e_flash, gemma4e_dflash, gemma4e_npu (XRT + HRX, Linux + Windows). Linux XRT libs are byte-identical to the locally tested builds. Consolidates #920. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Vendors the CI-built
gemma4e_flash,gemma4e_dflashandgemma4e_npuengine libraries (XRT + HRX, Linux.soand Windows.dll/.lib; 18 files, nothing else) from AMD-ROCm-Internal/FastFlowLM_IRON#138 (PR Check run 41388386154, all checks green). Models:gemma4e-flash:e2b/e4b,gemma4-it:e2b/e4b,nemotron3.5-bolt:e4b,nemotron3.5-bolt-dflash:e4b.Same change as #919 for the other Gemma4e engines: the vision encoder reconfigures the array in-stream inside the
vision_mmhw_context instead of switching between thevision_mmandvision_attnhw_contexts twice per layer (~3 ms each). No xclbin changes. Linux + XRT only inside the libs;FLM_VISION_SINGLE_CTX=0selects the old path.Results (local, Strix Halo, XRT; 30 random image requests per model, 1-3 images each, old vs new path)
The Linux XRT libs vendored here are bit-identical to the ones tested locally. Pressure runs (concurrent clients + client aborts) on gemma4-it:e2b and nemotron3.5-bolt:e4b: 0 failures; gemma4e-flash:e2b is intrinsically nondeterministic on some prompts on the old path too (13/160 vs 11/160 reply-set misses under the same stress). Details in the IRON PR.
Order
Draft until IRON#137 and IRON#138 are merged (and #919 for the dflash-flash lib). If review changes the IRON code the libs will be re-vendored from the new run.
🤖 Generated with Claude Code