Skip to content

fix: read the macOS 27 Find My sidebar - #36

Merged
omarshahine merged 4 commits into
mainfrom
fix/macos27-sidebar
Oct 10, 2026
Merged

omarshahine merged 4 commits into
mainfrom
fix/macos27-sidebar

Conversation

@omarshahine

Copy link
Copy Markdown
Owner

Fixes the parsing half of #35 on macOS 27.0.1.

Root causes

  1. people/items never left Devices. SwitchTab clicked menu item X of menu "View" of menu bar 1. On macOS 27 that errors with Can't get menu "View" (-1728), and prepareTab discarded the error (_ = SwitchTab(tab)), so the CLI parsed whatever tab was open. It now goes through menu 1 of menu bar item "View" (which also works on older releases) and returns the error.
  2. devices returned 2 rows. Row text now starts at about 59pt (115-122px at 2x), right on the fixed 60pt (textColMinPx = 120px) cutoff. Lines at x≥120 survived and lines at 118-119 didn't, so names lost their location lines and the walk fused rows. DetectTextColumnMin finds the shared left edge of the row text and cuts 12px left of it, clamped between 4/5 of the fixed cutoff (which keeps the ~48pt icon column out) and the fixed cutoff itself, so older layouts are unchanged. The three parsers now share one sidebarRows filter, and findSidebarNameLine (--zoom / ring) uses the same detection.
  3. Smaller macOS 27 changes: Nearby in the distance column, a fourth Me tab (needed for the sidebar right-edge detection, otherwise distances fall outside it), and a trailing battery-glyph segment (Home • Now • CD) that ended up in staleness.

Tests

  • New macos27_sidebar_test.go: geometry copied from a real macOS 27 capture (generic names) runs through ParseDevices, plus DetectTextColumnMin cases for macOS 27, the older layout and a lone fragment.
  • New splitLocationStaleness cases for the trailing segments.
  • Parsed the real macOS 27 captures from this Mac: 9/9 visible devices and 9/9 visible items, with correct name, location, staleness and distance.

Not in this PR

Sidebars longer than the window: the CLI has never scrolled the sidebar, so only on-screen rows are read. Tracked separately.

🤖 Generated with Claude Code

- Tab switching addressed menu "View" of menu bar 1, which macOS 27 no
  longer resolves; prepareTab ignored the error, so people/items silently
  parsed the Devices tab. Click through the menu bar item instead and
  return the error.
- Row text now starts at ~59pt, on top of the fixed 60pt text-column
  cutoff, so Vision's left-edge jitter dropped about half the lines and
  split names from locations. Detect the column from the OCR lines
  (clamped between 48pt and the old 60pt cutoff).
- Treat the new "Nearby" distance label as a distance, count the new
  "Me" tab when finding the sidebar edge, and drop the trailing battery
  glyph segment after the staleness.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[Medium impact] Fixes macOS 27 compatibility in the Find My app interface.

Fix the watch cutoff before merging; it still removes the macOS 27 names and locations.

Fix All in Claude CodeFindings

  1. P1 Watch misses visible devices ▶
  2. P2 Translated labels lack coverage ▶
Fix with agent prompt
### Issue 1
internal/findmy/findmy.go:412-415
The new cutoff still drops macOS 27 row text in `watch`. `watchPixelLayout` passes a 160px fallback at 2× scale, so `DetectTextColumnMin` sets its floor to 128px. The captured names and locations at 118–122px are excluded before detection, and the returned cutoff cannot fall below 128px. `watch devices` therefore returns no devices for the new fixture, even though the ordinary command reads it correctly.

Align the watch cutoff with the 60pt cutoff and test this fixture through the watch path.

### Issue 2
internal/findmy/findmy.go:511
The new `Me` and `Nearby` checks only cover English, while the existing tabs use `GetAppStrings`. If macOS shows translated labels, an unrecognized fourth tab can leave the detected sidebar ending at `Items`, dropping distances. A translated `Nearby` that survives the bounds check falls through as a name or location instead of a distance.

Add the new labels to `AppStrings` and cover a non-English sidebar to protect supported languages from missing distances or incorrectly grouped rows.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR updates tab selection and sidebar parsing for macOS 27.

  • The CLI stops when Find My cannot open the requested tab.
  • Sidebar rows keep their names and locations together in macOS 27.
  • Sidebar records keep the new distance and location text in the right fields.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Ordinary command: 60pt cutoff] --> C[Shared sidebar filter]
  B[Watch: 80pt cutoff] --> C
  C --> D[Detect sidebar width and row start]
  D --> E[Detect text column within cutoff floor]
  E --> F[Parse people, devices, or items]
  B -. Floor excludes macOS 27 row text .-> E
Loading

Reviews (1) · Last reviewed commit: "fix: read the macOS 27 Find My sidebar (..." · Reviewed by Greptile

Comment thread internal/findmy/findmy.go
Comment on lines +412 to +415
floorPx := fallbackPx * 4 / 5
var xs []int
for _, l := range lines {
if strings.TrimSpace(l.Text) == "" || l.Y < rowStartY || l.X < floorPx {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Watch misses visible devices

The new cutoff still drops macOS 27 row text in watch. watchPixelLayout passes a 160px fallback at 2× scale, so DetectTextColumnMin sets its floor to 128px. The captured names and locations at 118–122px are excluded before detection, and the returned cutoff cannot fall below 128px. watch devices therefore returns no devices for the new fixture, even though the ordinary command reads it correctly.

Align the watch cutoff with the 60pt cutoff and test this fixture through the watch path.

Prompt To Fix With AI
This is a comment left during a code review.
Path: internal/findmy/findmy.go
Line: 412-415

Comment:
**Watch misses visible devices**

The new cutoff still drops macOS 27 row text in `watch`. `watchPixelLayout` passes a 160px fallback at 2× scale, so `DetectTextColumnMin` sets its floor to 128px. The captured names and locations at 118–122px are excluded before detection, and the returned cutoff cannot fall below 128px. `watch devices` therefore returns no devices for the new fixture, even though the ordinary command reads it correctly.

Align the watch cutoff with the 60pt cutoff and test this fixture through the watch path.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid, thanks. watch had its own 80pt cutoff, so it read no macOS 27 rows. Both paths now scale the shared SidebarRightPt / TextColumnMinPt constants (55dce50), and TestParseDevicesMacOS27SidebarAtSharedLayout runs the fixture at the shared layout.

Comment thread internal/findmy/findmy.go Outdated
case "People", ls.PeopleTab:
return true, true
case "Devices", "Items", ls.DevicesTab, ls.ItemsTab:
case "Devices", "Items", "Me", ls.DevicesTab, ls.ItemsTab:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Translated labels lack coverage

The new Me and Nearby checks only cover English, while the existing tabs use GetAppStrings. If macOS shows translated labels, an unrecognized fourth tab can leave the detected sidebar ending at Items, dropping distances. A translated Nearby that survives the bounds check falls through as a name or location instead of a distance.

Add the new labels to AppStrings and cover a non-English sidebar to protect supported languages from missing distances or incorrectly grouped rows.

Knowledge Base Used: Localization and native helper bridge

Prompt To Fix With AI
This is a comment left during a code review.
Path: internal/findmy/findmy.go
Line: 511

Comment:
**Translated labels lack coverage**

The new `Me` and `Nearby` checks only cover English, while the existing tabs use `GetAppStrings`. If macOS shows translated labels, an unrecognized fourth tab can leave the detected sidebar ending at `Items`, dropping distances. A translated `Nearby` that survives the bounds check falls through as a name or location instead of a distance.

Add the new labels to `AppStrings` and cover a non-English sidebar to protect supported languages from missing distances or incorrectly grouped rows.

**Knowledge Base Used:** [Localization and native helper bridge](https://app.greptile.com/shahine/-/custom-context/knowledge-base/omarshahine/findmy-cli/-/docs/localization-and-helper-bridge.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid. Added MeTab and NearbyLabel to AppStrings for all 41 locales, pulled from the system bundles: ME_TAB in FindMy.app's Localizable.loctable and DISTANCE_FROM_USER_NEARBY in FindMyCore.framework's Models.loctable. TestLocaleTableHasMacOS27Labels checks that every locale has both, plus the French values. Distances are now also classified by column, so a translated or misread label in the right-hand column can't become a location.

omarshahine and others added 3 commits October 10, 2026 09:03
…Nearby

- watch used its own 80pt text cutoff, which drops every macOS 27 row.
  Both paths now scale the same SidebarRightPt / TextColumnMinPt.
- Group OCR lines into rows by vertical gap instead of alternating
  name/location. Shared items add a third "Shared with X" line and a
  row scrolled under the header shows only its status line; either one
  shifted every following row. The distance is read by column, so an
  OCR slip like "2,317 ml" no longer becomes the location.
- Add MeTab and NearbyLabel for all 41 locales, from FindMy's
  Localizable.loctable (ME_TAB) and FindMyCore's Models.loctable
  (DISTANCE_FROM_USER_NEARBY).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The list commands only read rows that fit in the Find My window. With
Family Sharing the Devices list runs to dozens of rows (36 on the test
Mac against 9-11 visible). ScrollPages scrolls to the top, reads page by
page in third-of-a-list steps, backs up when a step jumped past rows (the
helper's trackpad gesture is accelerated), stops after two pages add
nothing, and scrolls back to the top. Without Accessibility it reads one
page and says so on stderr.

Rows now need a name line and a status line: a lone line is a row cut
off at a page edge or one whose name Vision missed, and a group starting
with a status line has lost its name the same way. Distances are taken
by column, so an OCR slip like "2,317 ml" no longer becomes the location.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@omarshahine

Copy link
Copy Markdown
Owner Author

Merge proof for #35 (macOS 27.0.1, en-US, run live on this Mac):

Root causes fixed

  1. people/items never left Devices: menu "View" of menu bar 1 doesn't resolve on macOS 27, and the error was swallowed. The switch now goes through the menu bar item and returns the error.
  2. Rows lost: the macOS 27 text column (~59pt) sits on the fixed 60pt cutoff. The column is now detected from OCR. watch used 80pt, and both paths now share the constants (Greptile P1).
  3. Row walk: a strict name/location alternation broke on shared items' third "Shared with" line, on a row half under the header, and on a name Vision missed. Rows are now grouped by vertical gap, and each needs a name line plus a status line. Distances are taken by column.
  4. Lists longer than the window: the CLI never scrolled. ScrollPages reads page by page in third-of-a-list steps, backs up if a step jumps past rows (the trackpad gesture accelerates), stops after two pages add nothing, and scrolls back to the top. Without Accessibility it reads one page and says so.
  5. Nearby and Me are localized for all 41 locales from the system loctables (Greptile P2).

Tests: go vet and go test pass, with new fixtures from real macOS 27 captures (generic names) for devices, scrolled shared items, OCR "ml", and a missing name. CI is green on 69d8589.

Live proof (real Find My, 1024x768 window):

  • findmy devices: 36 rows, identical across 3 consecutive runs (~18s each). Before this PR it read 2; a first paging attempt with big steps varied by run and peaked at a 29-row union.
  • findmy people: 9/9, matching the screenshot.
  • findmy items: 12/12, including the 2 rows under Items Shared With Me.
  • Names, locations, staleness and distances checked against screenshots; distances that are still loading (spinner) are correctly left empty.

Not in scope: watch stays single-page (documented in the README), because scrolling every poll would keep moving the pointer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant