Skip to content

A put compacts history on the protocol path: KVShard.Put/PutDyna/PutPerm call Compress every 5,000 writes #92

Description

@PaulSnow

Invariant: 1.2 (no protocol-path cost that degrades toward linear in the age of the store) and 1.6 ("the protocol path never waits for maintenance"). Section 2.7 says all maintenance copies run off the protocol path, and the adapter moved its own calls there (#57/#58). The store still has a second door.

Measured. Accumulate soak 20260916T185711Z (8 shards, 8 execution shards, 500 tps, 1 s blocks), 30 s CPU profile of bvn2-val4 at 19 minutes of load: KVShard.PutDyna 3.69 s of 30 s, of which KV2.Compress → SegmentStore.CompactHistory → writeMergedRun → mergeIndexes 3.43 s (11.4% of the process) — synchronous, on the block-committing goroutine, inside bcdb.(*Database).commit → writeThrough → putRouted. The adapter's own maintenance goroutine (maintain.func1) cost 0.22 s in the same window. At 15 minutes on a serial run the same path was 0.31 s; the cost grows with history, and it is what bends every run's block-time tail at ~20 minutes: cumulative blocks over 0.82 s went 5% at minute 5 to 15% at minute 19 while writes per commit fell.

Code. kv_shard.go:288-321: PutDyna, PutPerm and Put each do if writes > 5000 { return k.Shards[index].Compress() }. KV2.Compress (kv_2.go:597) runs CompactHistory and clears the counters. Nothing else schedules this; it dates from the sharding commit (684f176) and predates the spec. KV2.PutPerm returns dWrites, the dyna counter (copy-paste), so permanent puts trigger dynamic-history compaction on the dyna count.

Fix. A put never triggers maintenance. Remove the three triggers; Compress stays for the adapter's cadence, which already runs it off the protocol path. A test drives a shard past the threshold with history to compact and asserts CompactHistory never runs from a put — and that it does run when Compress is called, so the assertion is not vacuous. Spec 2.2/2.7 say so.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions