Currently the Pool manager thread checks if the average number of free elements in the pool is greater than the object cap, and if so, prunes the number of objects in the pool presently down to the object cap.
This conflates two different metrics:
- the average number of free objects since last manager thread check
- the number of free elements at the present instant
Neither of these are good metrics.
The number of free elements at any instant is fairly meaningless - just becuse the pool has 5 more free elements than the cap now does not mean they're unlikely to be needed in a short period. Pruning until the number of free objects right now matches the object cap (line 105) is not ideal.
Likewise, given that the number of free instances fluctuates both up and down, the average number of available objects across all GetNew calls is always greater than the minimum number of available objects seen by any specific GetNew call. If we want the pool to never run dry, we should only prune when the minimum number of available objects always stayed above some limit (0, 1, or some cap).
To remedy this, rather than tracking the running average (via counter/sum, lines 66, 67, 104) instead track the minimum number fo free objects (min_free = std::min(min_free, objects.size()) at line 66) and prune the difference between min_free and object_cap (when min_free > object_cap) at line 105.
This would minimize unused memory, while alllowing a configurable minimum excess to account for potential spikes in demand.
However there is another type of object cap that makes sense: a hard limit on the number of instances at any given time.
Currently, GetNew always returns a new instance. If a free one is available it will be used, but otherwise new ones will be allocated without limit. To enforce a hard cap, Pool would need to track the number of created instances, refusing to create more (e.g. by returning nullptr from GetNew) when the pool is dry and that number has been reached.
This would want to be a new feature disabled by default, as it would require callers to handle possible nullptr returns from GetNew.
Currently the Pool manager thread checks if the average number of free elements in the pool is greater than the object cap, and if so, prunes the number of objects in the pool presently down to the object cap.
This conflates two different metrics:
Neither of these are good metrics.
The number of free elements at any instant is fairly meaningless - just becuse the pool has 5 more free elements than the cap now does not mean they're unlikely to be needed in a short period. Pruning until the number of free objects right now matches the object cap (line 105) is not ideal.
Likewise, given that the number of free instances fluctuates both up and down, the average number of available objects across all GetNew calls is always greater than the minimum number of available objects seen by any specific GetNew call. If we want the pool to never run dry, we should only prune when the minimum number of available objects always stayed above some limit (0, 1, or some cap).
To remedy this, rather than tracking the running average (via counter/sum, lines 66, 67, 104) instead track the minimum number fo free objects (
min_free = std::min(min_free, objects.size())at line 66) and prune the difference betweenmin_freeandobject_cap(whenmin_free > object_cap) at line 105.This would minimize unused memory, while alllowing a configurable minimum excess to account for potential spikes in demand.
However there is another type of object cap that makes sense: a hard limit on the number of instances at any given time.
Currently, GetNew always returns a new instance. If a free one is available it will be used, but otherwise new ones will be allocated without limit. To enforce a hard cap, Pool would need to track the number of created instances, refusing to create more (e.g. by returning
nullptrfrom GetNew) when the pool is dry and that number has been reached.This would want to be a new feature disabled by default, as it would require callers to handle possible nullptr returns from GetNew.