WIP: preserve YCBT sleep fidelity and explore overnight fragment stitching

[DBozhinovski]

Status: WIP / request for design feedback

This is deliberately a draft PR, not a merge-ready recommendation.

The branch contains a tested implementation of one possible solution, but local hardware evidence exposed an ambiguity in what a YCBT sleep-record boundary means. Before narrowing or finalizing the code, I would like feedback on:

• whether separate YCBT records should be stitched at all;
• whether this behavior is specific to the R10M or shared by other YCBT devices;
• what an unreported interval between records means semantically;
• how any inferred interval should be represented in PulseLoop and Health Connect.

Problem

PulseLoop currently treats each complete YCBT sleep record as an independent sleep session. The Today tile then surfaces the longest session for the waking day.

On a real R10M FCF4 (wearableModelID=r10m, YCBT, firmware 2.32) connected to a Pixel 7, one complete CRC-accepted 1,640-byte sleep-history block contained three records for the same apparent overnight period:

Record	Header bounds	Classified duration
A	02:07:46–03:08:32	60.7 min
B	05:01:08–05:51:46	50.6 min
C	06:33:32–08:30:17	116.7 min

Each record had valid header bounds and contiguous timestamped deep/light/REM segments. There were no explicit awake segments covering the gaps.

Observed downstream behavior:

• PulseLoop displayed only record C: 1h56m.
• The three records contain about 3h48m of classified sleep.
• Their overall span is 02:07–08:30, about 6h22m.
• The wearer independently estimated roughly six hours of sleep.

This suggests the ring may be emitting fragments of one overnight session, but the wire data does not prove whether the two gaps mean:

1. sleep that the ring failed to classify;
2. awake or off-finger time inside one night;
3. deliberate boundaries between separate sleep sessions.

Additional correctness findings

The investigation also exposed less ambiguous pipeline problems:

• YCBT header start/end timestamps and timestamped segments were flattened into first timestamp + summed stage minutes.
• Truncated/malformed concatenated records could consume the next record or become authoritative.
• Independent minute flooring could lose substantial stage duration and short explicit awake transitions.
• Complete-session reconciliation could leave stage blocks outside their parent Room session.
• A corrected Room session was not guaranteed a strictly higher Health Connect version.
• Sleep Health Connect IDs depended on main/nap rank, so a correction could change identity and leave stale records.

Experimental implementation in this branch

The branch currently:

• preserves and validates YCBT header bounds and timestamped segments;
• safely recovers from malformed/truncated concatenated records;
• preserves explicit awake stages and fills uncovered time within accepted bounds as UNKNOWN;
• quantizes the complete timeline cumulatively at Room's minute resolution;
• keeps persisted blocks inside their parent session and reconciles corrections idempotently;
• uses stable v2 Health Connect IDs derived from stable Room session identity;
• guarantees increasing correction versions;
• includes a one-time app-scoped Health Connect sleep identity migration;
• experimentally stitches complete overnight YCBT fragments when all of these hold:
  • same waking day;
  • both start in the overnight window (>=19:00 or <12:00 local);
  • non-overlapping, chronological records;
  • gap <=3 hours;
  • combined span <=16 hours;
  • neither record is partial/non-authoritative.

The stitched gaps are represented as UNKNOWN, while explicit AWAKE remains awake. Daytime naps, different waking days, >3-hour gaps, partial records, and overlapping/duplicate records stay separate.

The generic 60-minute nap/session segmentation behavior for other ring families is unchanged.

Tradeoffs

Stitch as one session with UNKNOWN gaps

Pros

• Produces a single 02:07–08:30 overnight session for the observed R10M data.
• Preserves all measured stages while being honest that the gaps are unclassified.
• Avoids the Today/HA result being determined solely by the longest fragment.

Cons

• Health Connect and downstream HA aggregation generally count UNKNOWN as sleep, so this can turn missing/off-finger/awake time into reported sleep duration.
• The current three-hour threshold is evidence-based for one capture, not a protocol guarantee.
• It may merge genuinely separate early-morning sessions.
• Including unknown time in total duration can dilute deep-sleep ratios and change sleep scores.

Keep records separate

Pros

• Makes no inference beyond the ring's explicit record boundaries.
• Preserves split sleep and nap semantics if those boundaries are intentional.

Cons

• PulseLoop's single main-session surfaces under-report nights like this one.
• Summing only the three records still gives 3h48m rather than the user's approximately six-hour experience.
• Health Connect receives multiple sessions and HA may show only the latest/longest depending on its query.

Device-specific versus YCBT-wide behavior

The safe parser/persistence/identity fixes are general. The stitching behavior is proven only on one R10M firmware. Applying it to every YCBT profile may be too broad; gating it behind a model/profile quirk is safer, but requires deciding which devices share this wire semantic.

Health Connect migration

The proposed v2 identity migration deletes only PulseLoop-owned SleepSessionRecords and rebuilds them from Room. Room and other apps' records are not touched. However, deletion and reinsertion are not atomic, and migration ordering matters: it should not export old fragmented Room rows under new IDs immediately before a corrected sync collapses them.

Questions for maintainers / device owners

1. What does one YCBT sleep record represent? A complete independent session, a classification fragment, or something device/firmware-specific?
2. Has anyone captured multiple same-night sleep records from another R10M, TK5, SmartHealth-Colmi/Yawell, or other YCBT device?
3. Should gaps with no explicit stage count as:
   • sleep of unknown stage;
   • time in bed but not sleep;
   • awake;
   • no data that must not affect sleep duration?
4. Should the product display both classified sleep and overall overnight span/time in bed instead of forcing one scalar?
5. If stitching is appropriate, what boundary should define one night: a maximum gap, overnight clock window, same waking day, motion/HR evidence, or another wire field?
6. Is a three-hour maximum gap defensible, or should the threshold differ?
7. How should true split sleep and early-morning naps remain distinguishable?
8. Should stitching be:
   • R10M-only;
   • enabled through a YCBT model/profile quirk;
   • the default for every YCBT device;
   • omitted entirely?
9. Should sleep scoring use the full stitched duration or only classified non-awake minutes?
10. For Health Connect, should UNKNOWN contribute to the exported sleep session, or should unclassified gaps be omitted while retaining broader Room/time-in-bed bounds?
11. Should the Health Connect identity migration wait until a successful post-upgrade ring reconciliation before deleting/rebuilding old records?
12. What additional captures would make this decision robust (other models, firmware versions, known awake/off-finger intervals, naps, travel/DST)?

Possible directions

• A. No stitching: keep records separate and change UI/HA projection only.
• B. Aggregate classified duration: keep separate sessions but show their summed classified sleep for the waking day.
• C. Stitched time-in-bed: preserve one overnight span but expose classified sleep and unknown gaps separately.
• D. Model-gated stitching: enable the current experiment only for confirmed R10M variants.
• E. Evidence-assisted stitching: use adjacent HR/wear/movement evidence rather than a gap threshold alone.

My current preference is C + D: keep the general correctness fixes, gate overnight stitching behind an explicit R10M/profile quirk, and distinguish classified sleep from inferred/unknown time rather than silently presenting the entire span as measured sleep.

Verification

• Full JVM unit suite: 1,249 tests, 0 failures.
• :app:assembleDebug: passed.
• git diff --check: clean.
• The exact overnight-stitch regression fails when stitching is disabled and passes with the experimental implementation.
• Local Pixel 7/R10M test reproduced the three raw records and the 1h56 dashboard result.
• The experimental stitched build was installed locally, but final post-fix Room/Health Connect readback is intentionally not claimed yet.

Scope / non-goals

• This PR does not address the separate Health Connect step-source overlap/doubling issue.
• No production release or migration has been performed.
• Please do not merge while the stitching semantics and Health Connect migration ordering remain unresolved.