Skip to content

Find, ring and watch an accessory over direct BLE - #139

Merged
parawanderer merged 66 commits into
parawanderer:mainfrom
ubrt:feature/beep-nearby-tag
Sep 12, 2026
Merged

parawanderer merged 66 commits into
parawanderer:mainfrom
ubrt:feature/beep-nearby-tag

Conversation

@ubrt

@ubrt ubrt commented Aug 22, 2026 •

Copy link
Copy Markdown
Contributor

Implements #17, and carries the rest of the BLE work with it.

Both branches live here at your suggestion, since debugging the Bluetooth side as a user is much easier when the signal strength and the other BLE details are on screen at the same time.

Ringing an accessory

  • DeviceInfoActivity: a "Play Sound Nearby" menu entry, one shot.
  • MapsActivity: the Ring button on the tag card is a continuous ping toggle. Scan, trigger, pause, repeat until tapped again.

Both show live progress: Scanning, Connecting, Sending, result.

Ringing is a repeating burst rather than a continuous tone. Tapping to sounding takes roughly five seconds, the chirp runs a few seconds, then there is a gap, so while walking you get a burst every several seconds rather than something to home in on. The gap is a single constant, CONTINUOUS_PING_PAUSE_MS, currently 4 s.

Rationale for the FindMy.py / Java split is in my issue comment.

Seeing a tag that is in the room

A passive scan runs while a screen is open. Under its own "Over Bluetooth" heading, the device screen shows when the tag was last heard, its signal strength, and the battery level the accessory broadcasts.

The battery reading is persisted per tag, with the time it was heard. Nothing is copied from Apple's own battery field: that value is stale or unset for exactly the users this is for, and would arrive presented as something the phone had heard.

Where the phone was when it heard a tag is written as a location report of its own, marked local in a provenance column so it can never be confused with a decrypted one. The map draws it like any other position, which is what answers "you left it somewhere in this building".

Listening while the app is closed

An opt-in foreground service, off by default, with a permanent notification. Without it the radio only runs while a screen is open, which keeps the app a display feature. With it the app records, which is why it is opt-in and why the setting says so in its own text.

With the service running, a tag that goes quiet while the phone moves on raises a left-behind alert. A targeted verification scan runs before alerting, so silence only has to be worth checking rather than having to prove anything on its own.

The alert is per tag and off by default, since most tags a person owns are put down on purpose. The wait before it fires and the sound it makes are configurable, and it plays on the alarm stream on repeat until the notification is swiped: a chime at notification volume from a pocket is the thing that gets missed.

Finding the tag: the key derivation

The addresses worth scanning for come from the stored key alignment, extrapolated forward at one index every fifteen minutes.

The candidate margin is 48 hours, derived rather than chosen. One secondary key covers 192 primary indices, so a sighting matched through one places the tag no more precisely than that.

A window too wide to derive is cut to its newest thousand indices. A tag advertising right now has been running, so its index tracks the clock and sits at the top of the window; the bottom belongs to a tag that was switched off for months and has nothing to match anyway.

A primary-key match corrects the alignment in either direction. update_alignment only moves forward, which is right for a fetch and wrong for a BLE match: that comes from a wide, symmetric window and can legitimately land below where alignment believes the tag is, which is proof it has run ahead. Correcting that means writing the serialized state directly, with update_alignment's backward-time guard re-implemented so the bypass does not lose it. This is the piece I would rather push upstream than keep as a bypass.

Only a primary match writes an alignment. A secondary key is reported at the first index the search reaches, so its index is a lower bound rather than a position. A secondary match therefore only raises the floor, upward, which can undershoot the truth but never overshoot it.

Each candidate address is paired with the index it came from, and a sighting hands that back as a hint, so confirming an alignment costs three key derivations at one index rather than the whole window again: 0.02 s against 1.02 s for the same answer.

Derived addresses are kept across launches, in a file per tag. An address is a pure function of the accessory's keys and an index, so a stored one can never be wrong, only incomplete, and a rebuild asks Python only for the part it is missing. The index is kept only for primary keys, which sit at one index forever; a secondary key's index depends on where the deriving range began, so it is stored as unknown rather than as a number that would later be believed.

WideningSearch looks progressively further back for a tag nobody has heard: five hundred indices at a time, at most one tag a minute, never in the first two minutes after a scan starts, and only for tags not heard in the last ten. It stops when the tag turns up or when it reaches a hundred days. See the caveat below.

What it costs

Key derivation, desktop 3.3 s per 1000 indices, linear from 500 to 9600
Key derivation, idle phone 1.8 to 2.9 s per 1000
Key derivation, phone during app startup 15 to 33 s per 1000
Python interpreter warm-up 11 to 12 s, once per process
Addresses produced about one per index: keys_between de-duplicates, and secondary keys collapse
Stored index, in memory 136 bytes per entry as a HashMap<String, Match>

The warm-up is the reason nothing expensive runs in the first two minutes after a scan starts, and the reason the derived addresses are kept: on a cold start with a populated store, a tag costs one index of derivation rather than several hundred.

Limits worth knowing

WideningSearch is not backed by an observation. It exists on the theory that a tag whose index has drifted far below the extrapolation would otherwise never be found. I have no measurement showing that happens to a tag that stayed powered: the index follows the tag's own clock rather than network contact, and the bounded slice already allows about ten days of slack. It is tested and it costs nothing for a tag that is heard, but I would not defend it as necessary.

This branch therefore also carries the measurement that would settle it. When a fetch moves a tag's alignment, that is a real observation of where the tag was, and extrapolating the previous alignment forward to the same moment gives the drift directly. It is appended to a file in the app's external files directory rather than only logcat, since the reading is worth something as a series over weeks and logcat on a busy phone holds minutes.

Owner-nearby does not work. With the owner present, a 15 second scan sees the owner-present short form (12 02, one observed at −49 dBm) and nothing derived from the key schedule matches it. It fails at detection, not at connect or write, which is consistent with stek29's note that authorised playback is a separate, L2CAP based mechanism. Worth knowing when testing: a tag sitting near an iPhone looks missing, and that has been mistaken for a bug more than once.

Not tested: Google Find My Device accessories, third-party brands beyond the one below, and Android versions other than 17.

Attribution

The protocol constants and fallback order in ble/BleGattSoundTrigger.java derive from AirGuard (Apache-2.0). A NOTICE names what is derived and from where. AirGuard ships no NOTICE of its own and its LICENSE has no copyright line filled in, so the file attributes the project rather than restating a notice its authors never wrote.

Testing

Pixel 10 Pro, Android 17, release build, over several days of ordinary use.

Ringing runs through to DONE (SUCCESS) on real AirTags and on a third-party FineTrack Mini Smart Finder. The passive scan, the alignment correction, the local position write and the background service have each been observed working on a device.

Suites: 348 JVM, 269 Python bridge, 346 strings in each of the ten locale files, flake8 and pyright clean.

The instrumented suite has not been run against this state. It passed at 568 with 5 ignored and 0 failures on aosp-atd, and the two migrations added here are additive, but I have not put a number behind that and will not imply one.

Pin

The FindMy.py pin points at parawanderer/FindMy.py@ddc7f234, so the blocker this PR opened with is resolved.

@parawanderer
parawanderer self-requested a review August 23, 2026 08:32
@parawanderer parawanderer self-assigned this Aug 23, 2026
@parawanderer parawanderer added enhancement New feature or request @app Issues regarding the OpenTagViewer Android app labels Aug 23, 2026
@parawanderer parawanderer added this to the App version 1.1.0 milestone Aug 23, 2026
@ubrt
ubrt temporarily deployed to Android Build August 23, 2026 10:10 — with GitHub Actions Inactive
@parawanderer
parawanderer force-pushed the feature/beep-nearby-tag branch from 2bd1379 to 6637e02 Compare August 23, 2026 11:11
@ubrt

ubrt commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

Marking this as work in progress — please hold off merging.

Testing with two real third-party Find My accessories surfaced a reliability gap: an accessory's stored key alignment can drift ahead of its true index, and once that happens the 12h search margin alone doesn't recover it (one of the two needed a multi-day margin to be found at all, which isn't a fix, just a wider blind spot).

Root cause: update_alignment only refuses to move backwards, so anything that feeds it a matched key's index — a network fetch, or the sighting-feedback this branch just added in recordAccessorySeen — can push alignment past the true index and never come back, if that index came from a secondary key match. A secondary key covers ~96 primary indices, so its first-match index in a search window is only a lower bound, not the true one. Only a primary-key match identifies the index uniquely enough to trust.

Working on a fix that restricts alignment corrections (both the BLE feedback path here and the existing network-fetch path) to primary-key matches only, and allows correcting downward when one disagrees with the stored value. Will follow up once it's tested end to end — a real AirTag test is planned for next week, which should also cover ground this branch hasn't (only third-party accessories tested so far).

@ubrt
ubrt marked this pull request as draft August 23, 2026 11:17
@parawanderer

Copy link
Copy Markdown
Owner

Agreed, holding. Confirming the network half from the pinned source so you don't have to re-derive it — findmy/reports/reports.py:456:

for i in sorted(key_to_ind[key], reverse=True):
    accessory.update_alignment(report.timestamp, i)

key_to_ind is populated from both cur_keys_primary and cur_keys_secondary, so a report that decrypts against a secondary key feeds its lower-bound index straight in. Same mechanism, already shipping — this branch adds a second door to a room the app is in already. Which also means a drifted tag gets harder to find over the network, not just over BLE, and presents as "stopped updating" with nothing in the logs.

Note I pushed to your branch while you were writing this (rebase onto main, pin moved to the merged 254a7624, plus a 12h margin and the sighting feedback). The feedback half is exactly what you're describing as unsafe — I took the docstring's "safe to hand over regardless" at face value and you've disproved it on hardware. Happy to drop it and leave the field unused until your fix lands, or leave it in place for you to build on; your call, since you're the one testing it.

@parawanderer has an AirTag and an owner iPad that can be powered off for the separated case, so the AirTag gap is coverable here too if that's useful alongside your test next week.


(Reply drafted via Claude Code, reviewed by me.)

@ubrt

ubrt commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

Pushed the primary-key-only alignment fix mentioned above.

Also been running a passive companion build locally on top of this branch's BLE work: live battery and signal strength for owned tags on the map and device info screen while in range. Mainly useful as a simple way to tell whether you're actually receiving anything from the tag at all - watching the reading update live, per advertisement, is what made it obvious something was wrong when a tag stopped being found despite sitting right next to the phone. Never pinned down what originally caused that drift, but the fix closes a real gap in how a match got trusted, and the tag has stayed found since. Not part of this PR, still iterating; may open it separately later.

image

If you want, it would really help if you tested with your own AirTag too :)

@ubrt
ubrt temporarily deployed to Android Build August 23, 2026 14:39 — with GitHub Actions Inactive
@ubrt
ubrt temporarily deployed to Android Build August 23, 2026 14:39 — with GitHub Actions Inactive
@parawanderer

parawanderer commented Aug 23, 2026 •

Copy link
Copy Markdown
Owner

Quick note I wanted to drop here before I go all in on my own review as I am finishing up the other open items I still had for 1.1.0: so the app now optionally connects to iCloud. When it does fetch your tags from iCloud, if you enable the debug option in Settings, you'll see that on the "My Devices" device page of every AirTag it has the battery status as reported by iCloud on it. Now I think currently I only get the list from iCloud every 6 hours (I will probably tweak it because iPads update more frequently), but I'm curious if that matches the status you're seeing the tags report by bluetooth? 🤔

@ubrt

ubrt commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

Good question. Two things worth knowing about what the Bluetooth side reports before comparing:

The advertisement only carries a 2-bit battery field, so it is one of four coarse levels (full / medium / low / critically low), not a percentage. It's the same encoding FindMy.py's BATTERY_LEVEL map reads. So the comparison can only ever be "same bucket or not".

On my two third-party tags there is nothing to compare against, which is itself the interesting result: both were imported via the iCloud account connection (not the zip import), and the iCloud record still reports battery level 0 ("not yet reported") for both. So for these tags the BLE reading is the only battery signal that exists at all. Presumably only Apple's own devices ever write that field for third-party accessories, or these vendors never report it. Your AirTags may well behave differently there, so your comparison would cover the case mine can't.

More generally: for anyone without an Apple device around (this app's core audience, arguably), BLE is the only battery source there is, whatever iCloud's refresh interval. The design question that actually falls out of this is persistence: right now my build only shows the BLE value while it's live and lets it age out. Should a sighting's battery level be persisted and shown as "last known (from BLE, at time X)" until a fresher value arrives from whichever source? Curious what you think.

The passive-watch build lives on my fork if you want to poke at it: https://github.com/ubrt/OpenTagViewer/tree/feature/nearby-tag-status (built on top of this PR's branch; not proposing it here, still iterating).

@ubrt
ubrt force-pushed the feature/beep-nearby-tag branch from 046f3db to ebffe4e Compare August 23, 2026 19:55
@parawanderer

parawanderer commented Aug 23, 2026 •

Copy link
Copy Markdown
Owner

Coming off my sprint to implement the whole iCloud stuff here just to answer your question @ubrt (I will check your work tomorrow or as soon as I am able to with my devices). But basically I would say that: yes you should save that battery status information and probably the time it was recorded too. It's useful for the owner of the tag to have, imo. I would combine that storage with the persistence of the BLE scans as an alternative source of location history - on top of the apple FindMy network reports (to the extent that you fill the fields of the LocationReports with BLE info).

Anyways, I highly recommend you rebase on main right now to pull all the iCloud changes since I think those are more or less complete now and have a bunch of account state management bug fixes.

@ubrt
ubrt temporarily deployed to Android Build August 23, 2026 21:24 — with GitHub Actions Inactive
@ubrt
ubrt temporarily deployed to Android Build August 23, 2026 21:24 — with GitHub Actions Inactive
ubrt and others added 6 commits August 24, 2026 13:00
Computes the accessory's current expected BLE MAC address(es) through the
pinned FindMy.py fork's rolling-key derivation (main.py:currentMacAddresses,
backed by the new RollingKeyPairSource.current_mac_addresses), scans for a
match, and writes the DULT/FindMy/AirTag GATT play-sound characteristic -
the same thing Find My itself does when a tag is close enough to reach,
without going through Apple's network. New menu entry on DeviceInfoActivity.

The GATT protocol details in ble/BleGattSoundTrigger.java - the service and
characteristic UUIDs, the start opcodes, and the order the three protocols
are tried in - are derived from AirGuard (Apache-2.0), verified against its
AppleFindMy.kt and GoogleFindMyNetwork.kt. This repository is MIT, so the
Apache-2.0 terms are recorded for the derived portion in a new NOTICE file
rather than only in a Javadoc header. AirGuard ships no NOTICE of its own,
so there is none to propagate.

The three protocols are a fallback chain rather than belt-and-braces: not
every accessory exposes the same characteristic, so relying on one alone
misses devices. Cheap to keep - discoverServices() fetches the whole service
table in one round trip and the three checks are local.

Temporarily pins a personal FindMy.py fork (ubrt/FindMy.py) across all four
places this repository pins it - see the comments at each - until the
current_mac_addresses() addition has been offered upstream and lands in
parawanderer's fork in turn.

Verified: full JVM suite, the Python bridge suite, flake8, pyright, and on
real hardware - see the branch's PR description for which accessories and
in what state.
MapsActivity's "Ring" button was already there in the layout, wired to a
no-op onClickRing - this fills it in rather than adding new UI. Toggling it
starts AccessorySoundTrigger.playSoundContinuously (scan, trigger, pause,
repeat via Rx repeatWhen) for that card's tag until tapped again or another
tag's ring is started; the icon/label swap to a red stop glyph while running.

BeaconInformation gained ownedBeaconAccessoryJson (mirroring the existing
ownedBeaconPlistRaw), populated in BeaconDataParser, since MapsActivity's
per-card data previously had no path to the accessory JSON the ble/ package
needs - DeviceInfoActivity's one-shot trigger reads it from OwnedBeacon
directly, but the map screen's BeaconInformation DTO didn't carry it.

Updates TagCardLayoutTest, which had a test pinning the ring button as
GONE from before this - "so that stops being true on purpose rather than
by accident". That guard now flips: the button is shown at rest, plus new
layout tests for the default label and for TagCardHelper's toggle/label
behavior, run on the same aosp-atd managed device (no Maps involved).

Requests BLE permission on tap, same as DeviceInfoActivity's one-shot
trigger - missed on the first pass here, and invisible on a debug install
that had already granted it via that other screen first. A fresh install
(found by testing an actual release build) surfaced it: the ring button
did nothing but log MISSING_PERMISSION on a loop, forever, with no dialog
ever shown.

Verified: full JVM suite (124 tests), instrumented layout suite (17 tests,
including the new ring-button ones), installed and running - including the
permission-request fix, verified end to end on a real release build after
the bug surfaced there (two full scan/connect/trigger cycles, one of them
after a retry).
AccessorySoundTrigger.playSound/playSoundContinuously now emit
BleSoundTriggerUpdate items (SCANNING/CONNECTING/TRIGGERING, then one
terminal DONE) instead of a single terminal result. Without this, both
callers went silent for however long the scan and GATT handshake took,
which reads as "nothing is happening" - especially the first time. Both
screens now show the current phase (a replacing toast in DeviceInfoActivity,
the ring button's own label on the map) instead of just a final result.
The map's ring button also swaps its icon for a spinner for as long as an
attempt is actually in flight (SCANNING/CONNECTING/TRIGGERING) - the label
alone can sit on screen for several seconds with nothing else moving,
which was mistaken for a stall rather than for work in progress.

Also: BleGattSoundTrigger.trigger is retried up to 3 times (800ms apart)
when it fails with a plain connection/write failure, not when no known
sound service was found on the device - a retry cannot fix the latter, only
the former is the kind of transient BLE flakiness a retry is for. Previously
one dropped connection meant an immediate failure with no second attempt.

BleAccessorySoundTrigger's three hardware-dependent seams (permission
check, scanner, GATT trigger) are now constructor-injected instead of
static calls to BlePermissions/NearbyAccessoryScanner/BleGattSoundTrigger,
the same reasoning AppDependencies already uses for HardwareDescriber -
those three need real Bluetooth to run, which a JVM test cannot arrange,
but the orchestration logic around them (the permission gate, the retry
count, what an empty candidate set or a scanner timeout maps to) does not
and is now covered by BleAccessorySoundTriggerTest (11 tests, fakes only).
The class is generic over the found-device type (<D>, fixed to
BluetoothDevice in the real forRealBluetooth() factory) because the real
Android class has no public constructor and this project has no
Robolectric to fabricate one for tests - a plain String stands in instead.

MapsActivity's handleContinuousPingUpdate now stops the loop outright on
MISSING_PERMISSION or NO_CANDIDATE_MACS instead of looping on them
forever - neither recovers by waiting and retrying, so continuing was pure
battery burn with no chance of succeeding. Found next to the permission-
request fix in the previous commit, but belongs here: this switch over
phases is what this commit introduced.

Verified: full JVM suite (135 tests, including the new suite), installed
and running - two full scan/connect/trigger cycles on a real release
build, one of them after a retry, plus the loading spinner confirmed
visually on a subsequent release build.
The write itself is near-instant, so the label went scan -> connect -> Stop
with no visible moment where it actually worked - which is what looked like
a missing step. A successful DONE now shows ring_status_success for the
whole pause before the next scan (roughly as long as an AirTag's chirp
lasts); any other outcome still goes straight back to "Stop".

Verified: full JVM suite, installed and running.
The BLE work needs RollingKeyPairSource.current_mac_addresses(), contributed
by Ulrich Barrot (@ubrt) and merged as parawanderer/FindMy.py#1. This moves
all four pin sites off the personal fork and onto 254a7624, which carries it.

The same bump also brings the keychain-export work that landed on that branch
meanwhile - including the fix for parawanderer#140, where a recovered peer was addressed
by its escrow label instead of the bottle's id and every share came back
unreadable.

Two things the app was getting wrong against that API:

A bare current_mac_addresses() searches with no margin, so an accessory whose
true index has drifted below where alignment believes it is is simply absent
from its own candidate set - no error, no match, "not found nearby" for a tag
sitting on the desk. FindMy.py's own is_from takes 12 hours; so does this now.

And the call discarded the indices the map exists to provide. A sighting is an
observation of the same kind as a decrypted report, so it now feeds back
through update_alignment and is persisted the same way, which collapses the
next scan from a 12-hour range to three keys.

Co-Authored-By: Ulrich Barrot <12350410+ubrt@users.noreply.github.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ches

recordAccessorySeen fed back whatever index currentMacAddresses paired an
address with, primary or secondary. A secondary key covers 96 consecutive
primary indices, so its index is only the first one a search happened to
reach, not the true one - and update_alignment only refuses a move
backwards. Fed that index, it can ratchet alignment past the true index
in the wrong direction, permanently. Measured on a real accessory that
drifted 114 indices (28.5 hours) ahead this way and then needed a
multi-day margin just to be found at all.

The fix moves what crosses the bridge from the key index to the raw
address: recordAccessorySeen now re-derives the key at that address
itself and only accepts a match through its primary key, where the index
is unambiguous. BleSoundTriggerResult.matchedKeyIndex becomes matchedMac
throughout, since only Python can tell a primary key from a secondary
one from an address - this side of the bridge never could, regardless of
what shape crossed it.

A primary match also corrects downward, which update_alignment itself
cannot do (it only ever moves forward, correct for a fetch's own forward
search but not for a BLE match that can legitimately land below the
stored alignment - proof the alignment had already drifted too far
ahead). recordAccessorySeen writes the corrected index straight into the
serialized accessory instead of going through update_alignment for this.
@parawanderer

Copy link
Copy Markdown
Owner

@ubrt so far, so good.

I'm thinking it might be nice to reuse some free-to-use battery icon for the battery state in the main UI in place of the current "Battery" text to make the main map page easier to parse. I'm sure there should be something like this available already for free reuse, with 4 battery states. WDYT?

Also, I guess that if we merge your work into the app, the limitation will remain that when the tags are in their owner-connected state they will not be able to be locatable, right? Maybe it's worth typing up an issue summarising the state of that limitation in this repository, so somebody who is interested in investigating that further has a nice basis to start from. I suspect you might have a bit more context on this than me, so maybe you could have e.g. Claude Code summarise your knowledge on that in a new issue post here (and I can add anything I ran into over the last year with this topic below it - but really I haven't touched the Bluetooth all that much at all).

I think that after merging your changes here, that would leave the app's limitations at:

@parawanderer

Copy link
Copy Markdown
Owner

Heads up on an overlap with #173, which is likely to merge first.

LocationReport.provenance says its reason for existing is the export:

The column exists because the history is exported. Without it the CSV hands somebody a file
where their own phone's positions sit unlabelled among Apple's, and nothing in the file says
which is which.

This PR does not touch HistoryCsvWriter or HistoryZipWriter, so that is currently unwired.

Meanwhile #173 rewrites the history CSV (adds beacon_id, a requiredHeaders() contract shared
between writer and reader) and adds an importer that restores rows from it. When both are in:

  • the CSV still will not carry provenance, so the column's own justification goes unmet
  • and importing a history exported after this PR would restore every row with provenance
    defaulting to apple, silently relabelling the user's own local sightings as network reports.
    Those are different claims: Bluetooth range against a hundred metres or worse.

Nothing to change here yet. Just flagging that once #173 lands, this PR owns adding provenance to
requiredHeaders(), the writer and the importer, since neither PR can see the problem on its own.


This comment was written by Claude Code.

@ubrt

ubrt commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

I did two tests this week while keeping the AirTag offline, and both were successful. The signal was detected and the sound was triggered.

In my case, there was a slight delay of around 30 seconds after I unwrapped the AirTag. It could be that the device uses a longer interval between pings when it has been disconnected from its host for an extended period of time.

I’ll finish up the remaining work next weekend. There is currently a bug with the notifications when a tag hasn’t been seen for some time, the alert never stops in some cases.

If there are no showstoppers on your side, we could merge this next week?

@parawanderer

Copy link
Copy Markdown
Owner

Hi @ubrt, nice! For me I've been testing it around town when I remember and it's been looking good so far. I haven't found any bugs with your mechanism though I did not test the alert. Totally cool to merge this after you fix the bug you found and then release this as part of the 1.1.0 version of this app!

parawanderer and others added 2 commits September 3, 2026 19:24
Two things, both about the same value being read from the wrong place.

**The long-fetch banner shows for tags that are perfectly aligned.**
60c1aee added SlowFirstFetch so the banner only appears when the key
search is genuinely wide, and it asks KeyAlignmentPlist, which reads
lastIndexObservationDate out of the export's KeyAlignmentRecord. That
column is written by refreshFromImport and refreshFromAccount and by
nothing else. It is frozen at import.

The alignment that actually decides the search width lives in
accessory_json: FindMy.py serialises alignment_date and alignment_index,
and updateAccessoryJson writes the whole blob back after every fetch. So
anybody whose export is more than seven days old sees "Locating your tags
(x of y)" on every single refresh, however recently their tags updated.
Reported from a phone on this branch with tags that updated today.

KeyAlignmentPlist's own docstring says what it is for - "the one thing
known about a tag before anybody has ever scanned for it" - and ScanOrder
honours that, with a comment saying the record is "only ever consulted
for a tag with no scan history". SlowFirstFetch was the one caller that
did not.

AccessoryAlignment reads both fields out of the accessory state, and
SlowFirstFetch.laterOf takes whichever of the two timestamps is newer:
the live one normally, the export's record before the first fetch, and a
re-import's record when it is fresher than a stale blob.

Jackson rather than org.json, because org.json lives in android.jar and
the JVM test runtime stubs it - a test would read zero from a document
saying otherwise and pass. Rule 13.

**And the tag page now shows the alignment, in the debug panel.** Index
and date, or a line saying none is stored and the next fetch will search
from the pairing date. That is the first thing worth quoting in a report
about a fetch that takes minutes or comes back empty, and until now there
was no way to see it short of reading the database.

Nine JVM tests. Three strings in ten locales via add_strings.py.

Not verified locally: no Android SDK on this machine, so the build and
the suites are CI's to run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts:
#	app/src/main/res/values-de/strings.xml
#	app/src/main/res/values-en/strings.xml
#	app/src/main/res/values-fr/strings.xml
#	app/src/main/res/values-ja/strings.xml
#	app/src/main/res/values-ko/strings.xml
#	app/src/main/res/values-nl/strings.xml
#	app/src/main/res/values-ru/strings.xml
#	app/src/main/res/values-zh-rCN/strings.xml
#	app/src/main/res/values-zh-rTW/strings.xml
#	app/src/main/res/values/strings.xml
parawanderer added a commit to ubrt/OpenTagViewer that referenced this pull request Sep 3, 2026
The one case this app cannot handle: a tag that is with its owner, so it
is not in the Offline Finding network and there is nothing on Apple's
servers to fetch however far back anyone looks. Everything about it was
scattered across two blog series, four papers, three GitHub threads and a
FRIDA repository, and none of it says which parts are settled.

**The finding worth the whole document** is stek29's, in FindMy.py parawanderer#88:
the proof of concept everyone reaches for writes GATT to a characteristic
that is not present on a real AirTag with its owner nearby, because it is
the *unauthorised* sound command. Authorised ringing - the owner's own
tag, sitting next to them - is a different protocol over L2CAP. Somebody
starting from the obvious PoC would spend a week finding that out.

That has a bearing on parawanderer#139, which plays a nearby accessory's sound over
GATT, so it is said in the document rather than left to be noticed.

Also collects: what a nearby tag actually broadcasts and how that differs
from a separated one (primary key every 15 minutes, secondary key daily
at 04:00); Adam Catley on the first six key bytes travelling as the BLE
address; the WOOT'22 firmware work behind seemoo-lab/airtag and what its
jailbreak requirement really is; and AirGuard, which malmeloo says already
rings tags from Android - marked unverified, because that claim is the
strongest lead here and it should not be taken on trust.

Nothing was read out of rustpush, apple-private-apis or export-findmy,
per the clean-room note in docs/findmy-export/README.md. stek29's public
comments are quoted; his branches are not.

Index row added per rule 10.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ubrt

ubrt commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

"Also, I guess that if we merge your work into the app, the limitation will remain that when the tags are in their owner-connected state they will not be able to be locatable, right? Maybe it's worth typing up an issue summarising the state of that limitation in this repository, so somebody who is interested in investigating that further has a nice basis to start from."

From my knowledge, this is a hard limitation with no chance to work around. If the host device is in range the tag and host are connected with a real end to end bluetooth connection. So the tag stops sending beacons and disables the public service for ringing.

@parawanderer

Copy link
Copy Markdown
Owner

@ubrt right okay, fair enough, I will write up my own knowledge on the topic separately then.

Let me know when you feel comfortable to merge this and create a release including this then!

ubrt added 3 commits September 6, 2026 12:16
Everything about which addresses are worth scanning for rests on extrapolating
the stored alignment forward at one index every fifteen minutes, and on that
extrapolation staying near the tag. The margin, the bounded slice, how far back
a search should reach: all of it is sized by argument rather than by a number.

An alignment that moves is a real observation of where the tag was, so
extrapolating the previous one forward to that moment and subtracting gives the
drift directly. Positive means the extrapolation ran ahead of the tag, which is
the direction that loses it.

Reported from both paths, which is the point. A fetch reading only exists for a
tag the Find My network has seen, and the network re-anchors such a tag long
before anyone loses it. The tags the question is about are the ones no passing
iPhone ever reports, and for those the only observation that will ever be made
is a primary-key match over Bluetooth.

An unchanged alignment reports nothing rather than a drift of zero: a fetch
that found nothing to align to did not confirm the extrapolation, and a series
full of zeroes meaning "nothing checked" would answer the question wrongly and
confidently. A secondary match reports nothing either, since its index is a
lower bound rather than a position.

Written to a file in the app's external files directory as well as to logcat,
because the reading is worth something as a series over weeks and logcat on a
busy phone holds minutes. An index and a difference of two indices, nothing
else. Python is told where to write once per process, and only from somewhere
that is about to start an interpreter anyway.
Running the service on a phone produced an alert roughly every ninety seconds
for tags lying on a table, and once produced four for the same tag in two
minutes. Several defects, all presenting identically.

**Presence rode on a throttle meant for something else.** The sighting listener
fires at most once a minute per beacon, which is right for the alignment write
it exists for and wrong for recording when a tag was last heard: at that
resolution an eighty second wait has twenty seconds of margin, and one missed
window is enough. Measured as a ninety-two second gap between two sightings of
a tag that was advertising throughout. The subscription beside it received
every advertisement and discarded them, so that is where it lives now. Safe at
that rate, since the work only happens on the transition back from gone, and
the arrival is claimed with a single compute() so two advertisements arriving
together cannot both take it.

**The verification scan could not pass.** It looked for a few hundred addresses
derived around the stored alignment while the passive index held everything
ever derived, so it asked "is the tag at the index we believe" when the
question was "is the tag here at all". Not one succeeded in eleven minutes of
measured use. It also ran after the decision, lasted six seconds against a
radio that had just missed the tag for a minute, and put the app into a
start-stop cycle the platform punishes at about five scan starts per thirty
seconds.

So it is gone, and the scan already running is raised to
SCAN_MODE_LOW_LATENCY at half the wait instead, then dropped back once nothing
is missing or the alert has gone out. Leaving it raised after an alert would
have held the radio wide open all evening for keys left at the office, and the
escalation exists to prevent a wrong alert, so once the alert is out it has
nothing left to prevent. The controller confirms both halves: our client is
reported as mode[BALANCED, used=LOW_POWER] when several apps are scanning, and
as mode[LOW_LATENCY, used=LOW_LATENCY] when we ask.

**A tag's own screen deleted every other tag's derived addresses.** The watcher
retired anything outside the set it was handed, and the device screen watches
the one tag it is showing. They returned as a freshly derived narrow window,
which for a tag whose alignment has moved does not contain it, so it stopped
being heard at all. Retiring a file needs the full list of tags, so it moved to
the service, which reads every beacon before it starts watching.

**The alarm repeated until dismissed.** Tapping the notification dismisses it
through setAutoCancel without firing the delete intent, so the obvious reaction
removed the only control over the sound, and a re-trigger restarted the cap
before it could arrive. Alarm usage already answers being heard; looping on top
of it bought nothing and cost the user control. It plays once, cut off after
five seconds however long the chosen ringtone runs.

Verified live: two alerts for one tag with a genuine sighting between them, the
scan raised before each and dropped back four seconds after.
The slider ran from ten seconds and defaulted to thirty. Thirty was unusable:
tags in the same room were called left behind repeatedly, because the
background scan does not listen anywhere near continuously and the short scan
meant to catch the mistake never succeeded.

Raising the running scan instead changes what the wait has to do. It no longer
has to outlast the scan's gaps by itself, only leave the escalation time to
work. With that, thirty seconds held in a measured window, so the floor is
thirty and the default is two minutes.

The default matters more than the floor. It is the value for everybody who
never opens this screen, whose tags may be weaker and whose phones busier, and
erring long costs a later alert while erring short costs a wrong one. A wrong
alert is what teaches somebody to ignore the one that matters. An existing
choice below the floor is raised to it by the resolver, which already clamped.

The explainer under the slider ended "Shorter catches you sooner and checks
more often", which described the verification scan that no longer exists. What
a shorter wait now does is start the full-rate scan sooner, so it warns sooner
and costs more battery. That is the real trade and nobody can weigh it from a
number of seconds. Rewritten in all ten locales, since a stale sentence in nine
languages is the same stale sentence.

Thirty rests on one person's tags in one flat over a short period, not on a
distribution of sighting gaps, which is still unmeasured.
@ubrt
ubrt force-pushed the feature/beep-nearby-tag branch from 4c4ef1b to 2db382c Compare September 6, 2026 10:19
@ubrt

ubrt commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

Just pushed the remaining fixes for the lost device notifications. From my side this is now ready to merge. Please feel free to give everything a final test :).

@ubrt
ubrt marked this pull request as ready for review September 6, 2026 10:20
@ubrt
ubrt deployed to Android Build September 6, 2026 11:43 — with GitHub Actions Active
@parawanderer
parawanderer merged commit ddfc5f3 into parawanderer:main Sep 12, 2026
3 checks passed
parawanderer added a commit to hamza-lzr/OpenTagViewer that referenced this pull request Sep 12, 2026
parawanderer#139 landed while this was in review and both touch `AppDependencies`. Conflict resolved by
taking main's version and re-applying this branch's five additions: the two imports, the
`historyImporterFactory` field, `historyImporter(context)`, `replaceHistoryImporter` and the
line in `reset()`.

`OpenTagViewerDatabase` merged cleanly - this branch adds `historyImportDao()` and no schema, so
it sits beside parawanderer#139's migrations 6-7, 7-8 and 8-9 rather than competing with them. Database
version is 9 and every migration is registered.
parawanderer added a commit to hamza-lzr/OpenTagViewer that referenced this pull request Sep 12, 2026
parawanderer#139 added `LocationReport.provenance` and said in its own docstring that the column exists
because the history is exported - "without it the CSV hands somebody a file where their own
phone's positions sit unlabelled among Apple's, and nothing in the file says which is which".
The writer was never updated, and this branch's importer builds its rows without the field at
all, which is a `NOT NULL` column: Room refuses the insert and takes the whole merge transaction
with it. Five instrumented tests fail on it, and none of them could have failed before, because
each PR was tested against a main without the other.

So the round trip now carries it:

- `BeaconLocationReport` gains the field, because the export reads models rather than rows, and
  the three mappings in `BeaconRepository` carry it in both directions
- `HistoryCsvWriter` writes a `provenance` column. An Apple row is a stranger's iPhone estimating
  a position to within a hundred metres or worse; a `local` row is this phone hearing the tag
  directly and recording its own position as the tag's. Unlabelled, the second reads as the first
- `HistoryImporter` requires the column and refuses a row whose value is neither, rather than
  storing a third kind of report that nothing downstream has a branch for
- `HistoryImportDao.merge` sets it, which is the crash

**The column is required rather than optional**, and that is free exactly once: the history
export shipped in no release - `util/export/` does not exist in `android-app-v1.0.5` - so there
are no archives in the field to stay compatible with. Adding it after 1.1.0 would have meant
accepting both shapes forever.

The two round-trip tests now carry it, and the first uses `local` on purpose: it is the value a
missing column does not fall back to, so it cannot pass against a writer or reader that defaults
the field into place.

This branch was previously deployed

1 inactive deployment
Android Build — 2db382c8 Deployed Sep 6, 2026 by ubrt via build #214
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

@app Issues regarding the OpenTagViewer Android app enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants