Skip to content

fix(transform): normalize flood() output to remove overlapping events - #143

Merged
ErikBjare merged 1 commit into
ActivityWatch:masterfrom
TimeToBuildBob:fix/flood-non-overlap
Jul 22, 2026
Merged

fix(transform): normalize flood() output to remove overlapping events#143
ErikBjare merged 1 commit into
ActivityWatch:masterfrom
TimeToBuildBob:fix/flood-non-overlap

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Problem

flood() can return overlapping positive-duration events when the pairwise
flooding loop mutates events that have already been processed. Downstream
duration aggregation counts each event independently, so overlapping events
cause hourly totals to exceed wall-clock time — reported as 60–71 minute hours
in the ActivityWatch Summary timeline.

Minimal reproducer from ActivityWatch/activitywatch#1369:

events = [
    Event(timestamp=start, duration=0, data={"title": "first"}),
    Event(timestamp=start, duration=1, data={"title": "first"}),
    Event(timestamp=start, duration=1, data={"title": "second"}),
]
result = flood(events)
# result contains two 1-second events at the same timestamp → 2s counted in 1s of wall-clock

The root cause: when gap < -negative_gap_trim_thres and the two events have
different data, the existing code only logs a warning and leaves both events
unchanged. Because the pairwise loop has already advanced past the earlier pair,
those overlaps survive the final zero-duration filter.

Fix

Replace the final zero-duration filter with a normalization pass that guarantees
a non-overlapping stream at the output boundary:

  • Same-data overlap: merge into one event (union of both durations).
  • Different-data overlap: later event wins; clip the earlier event to end at
    the later event's start. Drop the earlier event if clipping reduces it to zero
    duration.

This is the same policy the reporter's local fix used and is consistent with the
earlier-event-trimming direction in #105.

The normalization pass is a purely additive safety net — it does not change the
pairwise flooding logic, so existing behaviour for non-overlapping inputs is
preserved.

Tests

  • Updated test_flood_negative_gap_differing_data to assert the new (correct)
    behaviour: the earlier event is clipped away, leaving only the later event.
  • Added test_flood_zero_duration_chain_does_not_leave_overlaps (the exact
    reproducer from the bug report).
  • Added test_flood_normalization_preserves_non_overlapping_tail — ensures the
    pass does not disturb events that follow the overlap.
  • Added test_flood_normalization_merges_same_data_after_zero_duration_event
    covers the equal-data merge path.

All 174 tests pass; Ruff clean; targeted mypy clean.

Related

@greptile-apps

greptile-apps Bot commented Jul 22, 2026

Copy link
Copy Markdown

Greptile Summary

This PR normalizes flood() output to prevent overlapping events. The main changes are:

  • Merge overlapping events with identical data.
  • Clip earlier events when overlapping data differs.
  • Add tests for overlap chains, merging, and non-overlapping tails.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.
  • Inputs are copied and sorted before normalization.
  • The stack remains ordered, and each emitted boundary is non-overlapping.

Important Files Changed

Filename Overview
aw_transform/flood.py Adds a final normalization pass that merges equal-data overlaps and applies the documented later-event-wins policy to differing data.
tests/test_flood.py Updates the overlap expectation and adds tests for the reported chain case and both normalization branches.

Reviews (1): Last reviewed commit: "fix(transform): normalize flood output o..." | Re-trigger Greptile

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI-green and mergeable (Greptile 5/5) — waiting only on a maintainer click.

This PR is ready to merge, but the bot has pull-only access to this repo and can't self-merge — surfacing it here so it isn't lost. The monitoring loop will stop re-flagging it now that this note is posted.

@ErikBjare
ErikBjare merged commit 62276a7 into ActivityWatch:master Jul 22, 2026
5 checks passed
ErikBjare pushed a commit that referenced this pull request Jul 22, 2026
…cy tests (#145)

Port two improvements from #105 that weren't included in #143:

1. Sort key change: sort by (timestamp, duration) instead of just timestamp.
   When multiple events share the same timestamp, shorter events now sort first.
   This matches aw-server-rust's sort_by_timestamp behavior and gives
   deterministic, spec-aligned ordering.

2. Two new tests from #105:
   - test_flood_idempotent: verifies repeated flood() calls produce the same
     result (important correctness property)
   - test_flood_unsafe_gap: verifies overlapping differing-data events don't
     double-count time (validates the normalization pass from #143)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants