Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,15 @@ signals a backwards-compatible change.
retired `0x17 0x00`, whose payload also carried `inserted`, still decodes,
with the counter dropped, and is never encoded. The `sketchlib-golden-bytes`
submodule gains the `bloom_bits_*` fixtures.
- **Breaking: `SpaceSaving` no longer keeps a `total`.** `total()` is removed;
no Space-Saving readout read it, and after a merge it cannot be read off the
counts. A caller that needs the stream's total weight keeps a separate exact
count. The decode check that `total` covered the counted weight is gone.
- **ASAPv1: Space-Saving encodes as `0x18 0x01`, CountMinSketchWithSpaceSaving
as `0x1e 0x01` and CountSketchWithSpaceSaving as `0x1f 0x01`.** Each payload
drops the summary's `total`. The retired `0x18 0x00` / `0x1e 0x00` /
`0x1f 0x00` still decode, with `total` dropped, and are never encoded. The
`sketchlib-golden-bytes` submodule gains the `*_nototal_*` fixtures.
- **Count Sketch estimates no longer allocate.** The per-row estimates a
median is taken over are collected on the stack instead of in a fresh
`Vec`: up to 20 rows for `Vector2D` (the fast and regular paths),
Expand Down
8 changes: 5 additions & 3 deletions docs/api/api_count_min_sketch_with_space_saving.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,12 +71,14 @@ fn to_asapv1(&self) -> Result<Vec<u8>, RmpEncodeError>
fn from_asapv1(bytes: &[u8]) -> Result<Self, RmpDecodeError>
```

The **ASAPv1** envelope, kind `0x1e 0x00` (spec §3.24), for
The **ASAPv1** envelope, kind `0x1e 0x01` (spec §3.24), for
`CountMinSketchWithSpaceSaving<Vector2D<T>, Mode, H>` with `T` `i32`, `i64` or `f64`, `Mode`
`FastPath` or `RegularPath`, and `H: HashProfile`. The metadata is Count-Min's
plus `capacity` and `key_type`; the payload is the matrix followed by the
summary's triples and running scalars. A matrix holding `p != 1` does not
serialize. There is no delta kind.
summary's triples and its ceiling `discarded_max`. The retired kind `0x1e 0x00`,
whose summary also carried a `total`, still decodes with that field dropped and
is never encoded. A matrix holding `p != 1` does not serialize. There is no
delta kind.

## Examples

Expand Down
5 changes: 3 additions & 2 deletions docs/api/api_count_sketch_with_space_saving.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,11 @@ fn to_asapv1(&self) -> Result<Vec<u8>, RmpEncodeError>
fn from_asapv1(bytes: &[u8]) -> Result<Self, RmpDecodeError>
```

The **ASAPv1** envelope, kind `0x1f 0x00` (spec §3.25), for
The **ASAPv1** envelope, kind `0x1f 0x01` (spec §3.25), for
`CountSketchWithSpaceSaving<Vector2D<T>, Mode, H>` with `T` `i32`, `i64` or `f64`, `Mode`
`FastPath` or `RegularPath`, and `H: HashProfile`. Same layout as
[CountMinSketchWithSpaceSaving](./api_count_min_sketch_with_space_saving.md). There is no delta kind.
[CountMinSketchWithSpaceSaving](./api_count_min_sketch_with_space_saving.md); the retired `0x1f 0x00` still
decodes, its `total` dropped, and is never encoded. There is no delta kind.

## Examples

Expand Down
16 changes: 10 additions & 6 deletions docs/api/api_space_saving.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ fn is_guaranteed(&self, value: &DataInput) -> bool
fn top_k(&self, k: usize) -> Vec<(OwnedDataInput, u64, u64)>
fn entries(&self) -> Vec<(OwnedDataInput, u64, u64)>
fn min_count(&self) -> u64
fn total(&self) -> u64
fn len(&self) -> usize
fn capacity(&self) -> usize
```
Expand Down Expand Up @@ -111,8 +110,11 @@ a key both sides evicted cannot be recovered. `merge_equivalence_battery` does
not apply for that reason.

The `min_count` a one-sided key picks up is weight the stream never carried, so
after a merge the counts sum to more than `total` and `estimate / total` is no
longer a frequency.
after a merge the counts can sum to more than the weight both streams recorded.

The summary keeps no total weight: no readout above needs it, and after a merge
it cannot be read off the counts. A caller that needs the stream's total weight
keeps a separate exact count beside the summary.

## Serialization

Expand All @@ -121,17 +123,19 @@ fn to_asapv1(&self) -> Result<Vec<u8>, RmpEncodeError>
fn from_asapv1(bytes: &[u8]) -> Result<Self, RmpDecodeError>
```

These produce/consume the **ASAPv1** wire envelope (kind `0x18 0x00`) — see the
These produce/consume the **ASAPv1** wire envelope (kind `0x18 0x01`) — see the
[ASAPv1 wire format spec](../asapv1_wire_format.md). The impl is bounded on
`H: HashProfile`, so a summary built with an unprofiled hasher cannot serialize
at all.

A summary travels as `capacity`, `total`, the unmonitored ceiling, and one
A summary travels as `capacity`, the unmonitored ceiling, and one
`(key, count, error)` triple per monitored key. The bucket and counter lists and
the key index all follow from the triples and are rebuilt on load, so no arena
index reaches the wire and no crafted state can point one out of bounds or into
a loop. `capacity` and the key type are ASAPv1 metadata; the payload is
`[keys, counts, errors, total, discarded_max]`.
`[keys, counts, errors, discarded_max]`. The retired kind `0x18 0x00`, whose
payload also carried a `total`, still decodes with that field dropped and is
never encoded.

Keys are `OwnedDataInput`s, so the key type is a runtime property: the metadata's
`key_type` names the **exact** variant (`"i32"` stays `"i32"`, never widened to
Expand Down
Loading
Loading