Merge SadGare's fork: FindMy 0.9.x, map providers, and working key alignment

[parawanderer]

Merges SadGare's fork (#44) with their authorship preserved, plus the fixes needed to make it work end to end.

Closes #16, #30, #38. Likely helps #36, #23, #35.

Gare's work

Commit	What
29a15e4	Map provider abstraction + AMap implementation
962ee6a	FindMy 0.9.x backend, unicorn stub, zh locales, startup picker

Cherry-picked rather than squashed into my own commits, so git log --author still finds them.

One commit from the fork is deliberately excluded. Gare wrote the Migration(1,2) and the lazy accessory_json backfill correctly in 53c61d7. 4862cf7 ("remote old findmy backend related code"), 80 minutes later, deleted both and added a test asserting they stay deleted. Skipping that commit was the fix — none of their work needed rewriting.

Two things that looked like bugs in the fork are forced adaptations, verified against a real 0.9.8 install: fetch_last_reports/fetch_reports no longer exist, and LocationReport.published_at/.description are gone. The client-side filtering and getattr(report, "description", "") are correct.

Key alignment — the substantial fix

Without a KeyAlignmentRecord, FindMyAccessory starts at index 0 from its pairing date, so the first fetch searches the tag's entire life. Measured on real tags: 50,588 indices ≈ 175 sequential requests, per tag. That is the account-flagging risk in #30, quantified, and it applies to every import.

macOS does store it, confirmed on a real Sonoma install:

~/Library/com.apple.icloud.searchpartyd/KeyAlignmentRecords/<accessory-uuid>/<record-uuid>.record

Same nesting and same BeaconStore key as BeaconNamingRecord — it was excluded only by WHITELISTED_DIRS. Now wired end to end: the wizard exports it (format 0.0.2), the importer parses it, migration 2 → 3 persists it, and it reaches FindMyAccessory.from_plist(plist, key_alignment_plist).

Writing the tests found that this path was entirely dead code. AppleZipImporterUtil had the enum constant, the switch case, the map, the column and the migration — but no entry in MATCHERS, the only thing getAllowedFileType iterates. Every record was silently dropped. Nothing failed; imports just kept doing the full search. It survived because no real 0.0.2 zip has ever been produced, and on-device testing exercised the fallback probe, which works without a record.

For exports without one, main.py fetches the latest location instead of the history when the window is wide — one traversal, still narrows it. Verified live:

Key search window is 50588 indices wide; fetching latest location only...
Key search window narrowed from 50588 to 95 indices

Other fixes found by running it

• Fetch was all-or-nothing. Python returned one dict at the end, so quitting a long import discarded the alignment for every tag including finished ones. Java now drives the loop and stores each result as it arrives.
• Refreshes queued up. Serialising calls into Python fixed an event-loop crash but let the 60s refresh stack one entry per minute during a long fetch, then fire them all. The periodic path now skips its turn while Python is busy.
• Tag cards: text column offset by 51dp, ragged card heights, and the button row behind the navigation bar.
• AppCryptographyUtilTest asserted the whole keystore held one alias, so it passed once and failed forever after on any device that had signed in.
• Chinese language options, lowercase "password", doubled alignment probe.

Also in here

• Instrumented tests on a Gradle managed device — ./gradlew :app:testEmulatorDebugAndroidTest, 24/24 from cold in 1m37s with nothing booted first
• Contributor grid on the Information page, generated at build time so the app never calls the GitHub API at runtime

• Device list: empty state with an import button, long-press to remove

• String tooling (scripts/add_strings.py) with a --check that found 8 strings missing across 7 locales
• CI: translation check, pyright, and tests for the tooling; scripts/** added to the path filters, since the build already shelled out to it
• A pre-commit hook running the translation check plus flake8 and pyright on staged Python, scoped to match what CI actually gates so it cannot be stricter than the build

Added while testing the wizard on a real macOS 14 VM

Getting this branch onto a bare Mac turned out to be most of the work, and none of it was written down anywhere.

• scripts/bootstrap_macos.sh — one command from a freshly installed macOS to a running wizard: Command Line Tools, clone, virtualenv, dependencies, launch. Re-runnable.
  • Tests for the tools by running them, not by looking for them: a bare macOS ships stubs at /usr/bin/git and /usr/bin/python3 whose only job is to trigger the installer, so "is it on PATH" answers yes long before either works.
  • Picks an interpreter by whether it can actually construct a Tk() window. The Command Line Tools Python links the system Tk, which aborts during Tk_Init with Abort trap: 6 and no traceback if it decides the OS is too old — import tkinter succeeds right up until it doesn't. Falls back to printing the CLI route, which needs no GUI at all.
• The wizard now runs on Python 3.9, which is what the Command Line Tools install and therefore what anyone running from source on a stock Mac actually gets. It was crashing at import on MACOS_VER: tuple[int, int] | None — 3.10 syntax at module level. Fixed with from __future__ import annotations, and 3.9 added to the CI matrix so it cannot regress.
• TESTING.md → CONTRIBUTING.md (via git mv, history kept), with setup, every test suite, which Python each tree targets, and the macOS bootstrap.
• Recommends installing gh so an agent can read its own CI failures instead of guessing.

Exporter versioning

VERSION in python/main/wizard.py is the only place the exporter's version is written, and it is stamped into every export as via: OpenTagViewer.app:<version> — which is how a zip gets traced back to the exporter that built it. Nothing in the release pipeline rewrote it: the tag named the artifact, the source kept whatever was committed. Tagging macos-exporter-v1.0.5 against a tree still saying 1.0.4 would have shipped a build calling itself 1.0.4 everywhere a user can see, and python/README.md had already drifted to a version the source never declared.

scripts/exporter_version.py --tag <tag> now checks the two agree, in test-release-version, before either PyInstaller build starts. Both build jobs take the version from that job's output, which also removes the two copied awk/cut blocks that each re-parsed the tag. Releasing is deliberately two steps now — the bump is a commit, the tag only publishes it.

Fixed by making the source authoritative rather than patching the tag in at build time: the wizard also runs from source, and those exports stamp via: too, so a build-time patch would leave two artifacts from one commit disagreeing about their version.

VERSION bumped to 1.0.5, since this release changes what the zip contains.

Found by running the finished branch on a phone

Everything above was written before the branch had been used in anger. Importing a real 0.0.2 zip and clicking around found six more, none of which threw an exception, appeared in logcat, or would have been caught by any test that existed at the time. They are grouped here because the shape is the point: every one was a capability that quietly stopped being invoked.

Importing three tags produced one card. onImportFilePicked forked into an Observable.zip: one branch fetching reports, the other parsing metadata. zip completes when its shortest source does and disposes the rest — harmless while Python returned one dict for the whole batch, silently destructive once the fetch emitted per accessory. It took the first pair, completed, and cancelled accessories two and three before they were ever requested. A disposal is not an error, so nothing downstream could notice, and the tags sat in the database looking perfectly correct.

Selecting a tag did nothing visible. Tags kept in the same place resolve metres apart, which at any normal zoom is a single pile of markers. Raising the selected one existed before 29a15e4, as Marker.setZIndex, and was dropped when markers moved behind IMapProvider — MapPolyline kept a zIndex, MapMarker never gained one. The constants and the lastFocusedMarker field survived as unreferenced fossils, so the code still looked present. Restored for both card selection and marker taps, and MapMarker now carries a draw order that both providers honour.

The periodic refresh was off for the whole session. The startup stream completes without emitting when nothing is stored — a first run, or before any import — so initialFetchComplete, set in onNext, stayed false forever. The import path never set it either, so the one case guaranteed to hit both was a fresh install importing its first zip.

History showed one populated day and empty days behind it. _fetchReportsForAccessory took start and end and never passed them to the fetch, because there was nowhere to pass them: 0.9.x dropped the range parameters, and for a rolling-key accessory fetch_location_history delegates to _fetch_accessory_reports(..., only_latest=True), which walks back from now and returns at the first batch that yields anything — about one day. Every range returned today's reports, which the caller then filtered away to nothing. Rebuilt on accessory.keys_between(start, end) plus fetch_location_history(list_of_keys), generating the window's keys and feeding each report back into the accessory's alignment — the part the library cannot do, since it is handed bare keys and does not know which index each came from. Capped at 255 keys per request and 8 requests per fetch, because a history screen must not become a new route to the #30 problem.

A network timeout looked like a day with no reports. A day is one request; one aiohttp timeout emptied it. Requests are retried once now. More importantly, a wholly failed range used to return [], and mapResults only raises when Python returns None — so a dict merely missing a beacon read as "no reports for that day". The screen rendered a confident 0 reports for a day it had failed to fetch, and never reached the Retry button that already exists in view_history_bottom_sheet.xml. Total failure now returns None; partial results are still returned and logged as short.

Two latent bugs the new tests found on their own: the first fetch for a beacon stored Apple's response unsorted while every caller reads the last element as newest (a stale position on the map, for freshly imported tags only), and hoursToGoBack could return 0 when two fetches landed in the same millisecond — asking Apple for zero hours returns nothing, which is indistinguishable from a tag that has not been seen.

Tests, and why these ones

The branch started with an empty ExampleUnitTest. It now has 72 JVM, 33 instrumented, 43 Chaquopy bridge, 39 tooling and 29 wizard tests.

The bugs above all failed silently, so the tests are shaped to catch that specifically — they assert that a call happens, not that a value looks right:

Where	What it pins down
RxFlows	a fast branch must not truncate a slow one; accessories fetch one at a time; a failure skips one item, not the batch
MarkerFocus	selecting lowers the previous marker before raising the new one; a rebuilt marker stays raised across a refresh
RefreshPolicy	a tick during a long fetch is skipped rather than queued; the history window never asks for zero hours
BeaconLocationHistory	newest report is last; a beacon with no location cannot be drawn
LongFetchBannerState	the banner's counter survives overlapping fetches
TagCardLayoutTest	cards measure to a uniform height; the scroll row does not clip the shadows
test_main_ranged_fetch.py	the right keys are requested, batching stays under Apple's limit, alignment is updated, a failed range is an error rather than an empty day

Three of them are deliberately paired with an inverse assertion, because a guard that can only pass in one direction is not a guard: wrapContentIsWhatMadeTheRowRagged asserts the broken layout really is ragged, and the ranged-fetch tests assert that a genuinely empty range is not an error while a failed one is.

Much of this required extracting logic out of MapsActivity — 1,537 lines, every repository built in onCreate, nothing reachable from a test. What moved out is decision logic only; the remaining orchestration, and the UI tests it would unlock, are #50.

Verified

• assembleDebug green; 72 JVM, 33 instrumented, 43 Chaquopy bridge, 39 tooling, 29 wizard tests
• A 0.0.2 zip built by this branch's wizard, imported on a physical phone. main cannot export KeyAlignmentRecords at all, so the zip could only have come from here. All three records were read by the importer, and the key search window opened at 2,582 indices instead of 50,588, then narrowed to 95. Wizard → zip → importer → migration → FindMyAccessory → narrow first fetch, on live Apple data.
• History across multiple days, fetched and rendered per day against live Apple
• Clean 60s refreshes with correct de-duplication on overlap (8 held + 1 fetched = 8), and no app-level warnings or errors anywhere in logcat
• CI is green on all three jobs — Static checks, build, and instrumented tests on an emulator. Its first ever run found three latent bugs unrelated to this branch (a build step calling a script deleted months ago, pyright with no packages installed, pyright without Pillow); all three are fixed here.
• The wizard runs on a real macOS 14 install from bootstrap_macos.sh, against real com.apple.icloud.searchpartyd data
• KeyAlignmentRecords exists on real macOS with the expected structure

Not verified — please don't assume otherwise

• AMap. Needs a mainland-Chinese account with real-name verification. @SadGare — could you confirm it still works after the merge? The marker draw order added here had to go in reflectively like the rest of that class, so it degrades to an unordered marker rather than no marker if the method name ever changes. Replace AMapProvider's reflection with direct SDK calls #51 tracks replacing that reflection with direct calls.
• v1 → v3 on real data. Covered by tests, never run on a device holding real tags.

Note on AMap keys

AMap issues keys per developer account, bound to a package name and signing fingerprint, and expects the key holder to be the app's operator. Rather than bundling one, users supply their own in Settings; selecting AMap without a key does not save. There's a wiki page covering registration.

[pattontim]

I tried the debug release and the google map view is white but shows the control UI and google in the corner.
Will post logs when I get to a computer

Edit: tried 1d400ee

Just for awareness!

[parawanderer]

Hey @pattontim so there is a proper working release build of this now (well it should be working) that you should be able to update to.

It should look like this: