Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions docs/occupancy-2026-10.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,10 @@ on 2026-10-14, and checks it against the typed reading.
* `julia_allocated_bytes`: the `Base.gc_num()` delta over the stage (cost);
* per task (one spawned search): the largest sampled VmRSS of the child
process it started (`peak_rss_kib`), the kernel's own VmHWM for it
(`hwm_kib`), and the sample window its children were live in
(`first_sample`, `last_sample`) β€” what its grade is read from;
(`hwm_kib`), and the sample window its children were live in β€” `first_sample`,
the first sample taken after its first child started, and `last_sample`, the
last taken before its last was waited for, 0 for never β€” which its grade is
read from;
* `child_peak_rss_kib`: the largest sum of the stage's children's VmRSS at
one sampling instant (the stage's occupancy).

Expand Down Expand Up @@ -74,8 +76,14 @@ whole window (`MemoryProbe`'s `task_grades`, `stage_grade`,
`predicted_peak_rss_kib`, `cost_bound_rss_kib`, `reclaimed_kib`) and attaches the
result to a recorded stage as `grades`; a record with no sample windows is left
ungraded rather than graded on an assumed order. Tasks are read in sampled order
(first sample, ties by record order); a task's drift is its peak when the next
task still held it, and 0 when it had exited.
(first sample, ties by record order). Each handover keeps only the tasks still
live at it, so a task's `drift_kib` is the change in the live peak sum across its
boundary β€” negative when the boundary also releases an earlier task β€” and
`release_kib` is everything freed between its start and the next boundary, which
can exceed that task's own peak. The folded drift is the sum of the peaks still
live when the stage ends; the pinned instance's own `drift` is that non-negative
live sum, the signed per-task delta being the extension its header records as
H-2.

For the three orders above, the algebra's predicted peak reproduces the measured
stage peak exactly β€” 9204, 9192 and 18380 KiB β€” and the slack against the cost
Expand Down
12 changes: 9 additions & 3 deletions src/core/memory_probe.jl
Original file line number Diff line number Diff line change
Expand Up @@ -326,8 +326,14 @@ end
# hwm peak drift with drift ≀ peak
#
# * `peak` β€” the high-water mark this interaction reached, in KiB;
# * `drift` β€” the net change in memory held by the next interaction's start;
# * `release` = peak βˆ’ drift β€” what the frontier reclaims.
# * `drift` β€” the change in the live peak sum across the interaction's
# boundary, so a boundary that also releases an earlier task is negative;
# * `release` = peak βˆ’ drift β€” everything freed between the interaction's start
# and the next boundary, which can exceed the interaction's own peak.
#
# A task's drift subtracts earlier tasks that exited before the next start, so it
# can be negative β€” the signed-drift extension `Instances/HWM`'s header records
# as H-2. The folded drift is the sum of peaks still live at the stage's end.
#
# A task's drift subtracts earlier tasks that exited before the next start, so
# it can be negative. The folded drift is the sum of peaks still live there.
Expand Down Expand Up @@ -368,7 +374,7 @@ struct Grade
end
end

"`release(grade)` β€” what the frontier reclaims, `peak βˆ’ drift`."
"`release(grade)` β€” `peak βˆ’ drift`: everything freed by the interaction's boundary."
release(g::Grade) = g.peak - g.drift

"`seq(a, b)` β€” sequence `b` after `a`: the suffix starts from `a`'s drift."
Expand Down
21 changes: 21 additions & 0 deletions test/unit/test_occupancy_grade.jl
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,27 @@ _occ_stage(tasks; measured) = OrderedDict{String,Any}(
@test !MG._overlaps((1, 2), (3, 4))
end

@testset "the reviewer's three-task example: expiry, not the sum of drifts" begin
# Windows [1,3], [2,10], [4,5] and peaks 100, 1, 100 KiB β€” CodeRabbit's
# example on #117. A overlaps B and exits before C starts, so the largest
# simultaneous sum is 101 (A+B at samples 2-3, then B+C at 4-5), not the
# 201 a rule that kept every earlier drift would read, and not the 201 KiB
# cost. A's release at C's boundary makes B's delta negative, which is how
# the fold stops charging for it.
tasks = [_occ_task(100; label = "A", first_sample = 1, last_sample = 3),
_occ_task(1; label = "B", first_sample = 2, last_sample = 10),
_occ_task(100; label = "C", first_sample = 4, last_sample = 5)]
occ = _occ_stage(tasks; measured = 101)
@test [g.drift for g in MG.task_grades(occ)] == [100, -99, -1]
@test MG.predicted_peak_rss_kib(occ) == 101
@test MG.cost_bound_rss_kib(occ) == 201
@test MG.reclaimed_kib(occ) == 100 # the 100 KiB that never coexist
@test MG.disagreement(occ) === nothing
# The bound the expiry sharpens: 201 KiB resident at once is now the
# disagreement, where before the fix it was the prediction.
@test MG.disagreement(_occ_stage(tasks; measured = 201)) isa String
end

@testset "registration keeps the first child's sample boundary" begin
# No background sampler: watch! takes the first sample deterministically.
probe = MG.Probe(0.01, ReentrantLock(), MG.TaskTrace[], MG.TaskTrace("(stage)"),
Expand Down
Loading