Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions internal/p2p/load_router_queue_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,14 @@ func TestLoadRouterSharedQueueOccupancyAndDrops(t *testing.T) {
// queue the router builds, at a size small enough to reach its ceiling.
const capacity = 8
q := newSimplePriorityQueue(ctx, capacity)
// Account for the queue's heap and both channel buffers. Depending on
// scheduling, draining can make every one of these retained envelopes
// observable before the queue gets another chance to trim its heap.
// The queue holds envelopes in three places: waiting in the input buffer,
// in the heap, and popped into the output buffer. The heap is trimmed to
// maxSize on every push. The run loop holds one envelope in none of these
// between taking it from input and pushing it, which frees an input slot
// for the producer, but that push trims the heap back to maxSize, so the
// envelope replaces one already held instead of adding to the count. A
// flood that nobody drains fills all three, and draining can then hand
// over every one of them before a later push trims the heap again.
ceiling := q.maxSize + cap(q.input) + cap(q.output)

// Nothing drains while this runs, which is the case that matters: a
Expand Down
25 changes: 12 additions & 13 deletions internal/p2p/rqueue.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,26 +73,25 @@ func (q *simpleQueue) run(ctx context.Context) {
timestamp: time.Now().UTC(),
})

select {
case signal <- struct{}{}:
default:
if len(pq) > q.maxSize {
sort.Sort(pq)
for _, dropped := range pq[q.maxSize:] {
dropped.envelope.NotifyDelivery()
}
pq = pq[:q.maxSize]
}
}

case <-ticker.C:
// Trim on every push, so the heap never holds more than
// maxSize. Trimming only when the signal was already pending
// let every push that refilled it skip the trim, so with
// nobody draining, the heap ran past maxSize until the next
// tick.
if len(pq) > q.maxSize {
sort.Sort(pq)
for _, dropped := range pq[q.maxSize:] {
dropped.envelope.NotifyDelivery()
}
pq = pq[:q.maxSize]
Comment on lines 81 to 86

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Avoid whole-queue sorting for each overflowing push

The unconditional trim fixes the bound, but every overflowing insertion now invokes sort.Sort over the entire backlog to evict exactly one envelope. The shipped vote channel has RecvBufferCapacity 4096, making maxSize 16,777,216; a saturated insertion therefore sorts 16,777,217 entries. This adds whole-backlog sorting to the pushes that previously skipped trimming, increasing work precisely when the queue is backpressured. Consider an eviction structure that can locate and remove the lowest-ranked envelope without sorting the entire queue, while preserving the size/time ordering and NotifyDelivery behavior. This is non-blocking: sorting already existed, and O(n log n) is its worst-case bound, not a measured cost for each push; realistic saturation measurements should guide the optimization.

source: glm-5.3-flash (phase1-reviewer: general)

}

select {
case signal <- struct{}{}:
default:
}

case <-ticker.C:
if len(pq) > 0 {
select {
case signal <- struct{}{}:
Expand Down
Loading