Repository navigation
Report hugepage-backed allocatable memory as hypervisor capacity - #103
Conversation
2167d0a to
e7d59ae
Compare
Memory capacity was derived from each NUMA cell's total physical memory reported by libvirt. That value fluctuates between otherwise-identical hosts by a few hundred MB because linux reserves/frees varying amounts during boot, making capacity reporting non-deterministic. All KVM nova flavors use hw:mem_page_size=large, so the memory that can actually be allocated to VMs is the hugepage-backed memory, which is explicitly configured and therefore stable. Report that as the memory capacity instead. The libvirt capabilities XML already exposes per-cell hugepage counts and the Go structs already parse them, so no CRD or schema change is needed. A new HugepageMemoryToResource helper sums all non-base page sizes. If a cell reports no hugepages, fall back to its total physical memory so we never report a spurious zero capacity. Status.Capabilities.HostMemory is left as the physical total for information; it will now intentionally exceed Status.Capacity[memory]. Signed-off-by: Philipp Matthes <p.matthes@sap.com>
e7d59ae to
95083dc
Compare
Merging this branch will increase overall coverage
Coverage detailsCompared to commit c320213 (run #39742049513)
Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code. Changed unit test files:
|
fwiesel
left a comment
fwiesel
left a comment
There was a problem hiding this comment.
I'd say it is better than before, but we should be aware that this query the host capabilities.
Currently, in practice, all huge-pages are only for VMs, but that doesn't have to be.
Technically, we could (and probably will) limit the resources for VMs.
For faster network operations for example, we need hugepages just for the networking stack. That is not schedulable memory for the VMs.
Ideally, we would query the cgroupv2 for the machine.slice directly. But then I am not 100% sure in which slice these hugepages for networking will land either.
So, that may be wasted work then.
|
Thanks for the added context! |
Problem
In
internal/libvirt/libvirt.go,addAllocationCapacityderives the hypervisor's memory capacity (Status.Capacity[memory]and per-cellCell.Capacity[memory]) from each NUMA cell's total physical memory reported by libvirt capabilities.That physical value fluctuates between otherwise-identical hypervisors by a few hundred MB, because linux reserves/frees varying amounts during boot:
This makes capacity reporting non-deterministic across the fleet.
Fix
All KVM OpenStack Nova flavors use the extra spec
hw:mem_page_size=large, i.e. every VM runs on hugepages. The memory that can actually be allocated to VMs is therefore the hugepage-backed memory, which is explicitly configured (likeHugePages_Totalin/proc/meminfo) and does not fluctuate between hosts.We now report that stable value as the memory capacity.
The libvirt capabilities XML already exposes per-cell hugepage counts (
<pages size='2048'>N</pages>), and the Go structs already parse them (CapabilitiesHostTopologyCell.Pages) — so no CRD or XML-schema change is needed. We only change which value we write into the existingResourceMemorycapacity.Changes
HugepageMemoryToResource(pages)inutils.go— sums all non-base (hugepage) page sizes to bytes. The base page is the smallest reported size and is excluded; both 2 MiB and 1 GiB hugepages count.addAllocationCapacityuses the helper instead ofcell.Memory.addEffectiveCapacityand all CPU/allocation logic are unchanged (effective capacity is generic over the capacity map).Notes for reviewers
Status.Capabilities.HostMemoryis intentionally left as the physical total for information; it will now exceedStatus.Capacity[memory]— the gap is host-reserved + non-hugepage memory.Tests
TestHugepageMemoryToResource(2 MiB, 1 GiB, mixed, empty, base-only, base-not-first, bad-unit).TestAddAllocationCapacity_FallsBackToPhysicalMemoryWithoutHugepages.Pages.Verification:
go build ./...,go vet ./internal/libvirt/...,go test ./internal/libvirt/...all pass.