Repository navigation
PERF: Use plain device memory on every GPU - #36
Merged
Merged
Conversation
Managed memory was the default for a single rank on a unified-memory APU (AMD MI300A), to get past a ~62.8 GiB limit on plain allocations. Measured on Hunter (4x MI300A, ROCm 7.0.2, Cray MPICH 9.1 with GPU support): plain hipMalloc reaches 120 GiB with XNACK on or off, and an all-to-all between the four GPUs of a node, 64 MiB per peer, takes 0.9 ms on plain buffers, 39 ms on managed and 35 ms on host memory. Plain device memory is now the default everywhere; managed_memory=True or MUTOPOPT_MANAGED=1 still selects managed memory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
muTopOpt picked managed memory automatically for a single rank on a unified-memory APU (AMD MI300A), to get past a ~62.8 GiB limit on plain allocations. New measurements on Hunter (HLRS; 4x MI300A per node, ROCm 7.0.2, Cray MPICH 9.1 with GPU support enabled) show that plain memory is the better choice there too:
hipMallocreached 120 GiB of the 128 GB package (the test's cap) with XNACK on and off.Plain device memory is now the default on every GPU.
managed_memory=TrueorMUTOPOPT_MANAGED=1still selects managed memory.Changes
_device_memory_policy(on_device, requested=None): plain unless managed is requested. It no longer takes the GPU kind or the rank count._device_has_unified_memoryis removed, since nothing uses it any more.device memory: plain device memory...ordevice memory: managed memory..., so job scripts that grep for it keep working.homogenization.py, themanaged_memorydocstring and the README section are rewritten around the new measurements.test/test_device_memory.pyis updated for the new rule.CHANGELOG.md: a PERF entry under v1.5.0 (not yet released).Testing
I couldn't run pytest locally:
import muGridhangs in my environment, before any test code runs. I checked the two policy functions on their own, extracted from the module, against the cases the updated tests cover. CI should run the real suite.Companion docs change in muGrid: muSpectre/muGrid (
doc/plain-device-memory-apu).🤖 Generated with Claude Code