🙏🏻🙏🏻🙏🏻 Thank you for doing this! iPhone Port & The Future?

[uberhaus9]

I just stumbled upon this project. Thank you for doing this! I'm working on building my own Jarvis always on AI companion to help me manage my life and work as someone who struggles with AuDHD. This project solves the biggest problem I haven't been about to find a solution for yet - an always on mic feed so Jarvis can hear and see what I'm up to and jump in if I'm getting off track or I'm not thinking things through or if I'm too much into hyperfocus - not eating, not sleeping - and might trigger another long period of burnout or a meltdown. I think this AI tech can really help a lot of people who struggle bad with neurodivergent issues, but only if the second brain can actually monitor what the person is doing all the time.

A few questions:

• Any reason this can't be ported to iOS by someone willing to do the work? Like are there any technical limitations imposed by Apple that would be a blocker?
• I SOOOOO need this to exist for the long-term. What are the chances Meta will shut this down and block the project and also block the ability for this to be able to help so many people who might need it?
• Any idea what the impact would be on battery life if the mic channel and audio channel were always on? I'm thinking through how to manage the visual part, but I think the best option might be just to have the camera snap photos every 30 seconds or so and send those to my private cloud hosting my Jarvis system to save on battery.

Godspeed!

[macedonga]

Thank you for the kind words! A second brain that notices hyperfocus before it turns into burnout is a good reason to want this.

First, so you do not plan around something Visor does not do: Visor does not keep the microphone open today. The glasses send audio after "Hey Meta", and Visor hands it to the model you choose. That can be any OpenAI-compatible endpoint, so your Jarvis could answer "Hey Meta" directly.

iPhone port. I do not have a Mac or an iPhone, so I cannot build or test one myself. For someone who wants to try, I know of no hard blocker, but a lot would change:

• The session runs over a Bluetooth LE L2CAP channel, and iOS apps can open those. The RFCOMM fallback would go, because iOS gives ordinary apps no RFCOMM.
• Photo and video import uses Wi-Fi Direct, which iOS does not have. The glasses can host their own Wi-Fi instead (the Live view already does this), and an iPhone app can ask to join it. I also noticed that the firmware has some iOS exclusive services, but haven't really looked into those.
• iOS does not let an app read other apps' notifications, so notification forwarding would not port.
• iOS limits what apps do in the background, so an always-listening app needs care there.
• Setup logs in to Meta with the Meta AI app's own client tokens. Apple wants permission under a service's terms (guideline 5.2.2), and Visor has none, so it would be a sideload.

Will Meta shut it down? I know for a fact that this app caused a lot of discussions around Meta (I don't know if i can talk about this, but I got contacted by someone from Meta Reality labs saying that they liked the app a lot and wanted to talk with me, they were really nice). So I don't think they'll deliberately try to block me or send a cease and desist, but they might update the protocol, and since I reported some "vulnerabilites" (they're not really vulnerabilities, but some stuff that shouldn't do what it does now), it might get harder in the future to reverse engineer it (not impossible though). So basically: I cannot promise it keeps working. Visor needs Meta's servers during setup (the login and the ownership step) and for language packs. After setup, the phone and the glasses talk directly. A firmware update or a change on Meta's side could still break parts of it, and I fix what I can when that happens.

For something that lasts, Meta's official Wearables Device Access Toolkit has an iOS SDK, and its docs list Ray-Ban Meta Gen 1 and Gen 2. It streams the camera and captures single frames. It is still a developer preview. It also works through the Meta AI app, so it replaces Visor rather than running next to it.

Battery. I have not measured either case, so I have no numbers.

• Microphone: some pairs also show up on the phone as a Bluetooth headset, so an app can record from them the way a call does. Meta's toolkit gets the microphone the same way, at 8 kHz mono. I would expect a drain like a long phone call.
• Photos every 30 seconds: Developer mode already does this on Android. Your app sends TAKE_PHOTO, waits for MEDIA_AVAILABLE, sends IMPORT_MEDIA and gets the file in NEW_MEDIA. Each photo comes to the phone over Wi-Fi, and I expect that transfer to cost more than the shot. The capture LED lights for every photo, so people around you will see it.
• The Live view streams camera and microphone without a break, but it is the heaviest option, and the glasses can stop it when they run hot.

Good luck. If you build on Developer mode, I would like to hear how it goes, and if you need any feature implemented feel free to open an issue and I'll see what i can do!