Add local and OGAD remote video generation to Mobile

[alichherawalla]

Changes

• Add native local video generation on iOS and Android with a pinned stable-diffusion.cpp runtime.
• Add Video models, Hugging Face search, complete model packs, downloads, settings, chat turns, generate_video, progress, Stop, retry, recovery, gallery, playback, and export.
• Support OGAD remote video jobs, model residency and memory admission, and media sync through the existing owners.
• Add the SD image backend on both platforms while preserving Core ML, QNN, and MNN paths. The initial mobile SD pack is Qwen Image 2.1 from the shared catalog.
• Verify pinned image-pack files before registration, reuse verified parts, recover completed packs after interrupted registration, and support mobile SD pack transfer through private Pro integration.
• Prefer usable Qualcomm HTP acceleration on Android, then Vulkan and CPU. Keep video VAE decode on the GPU with bounded spatial and temporal tiles. Avoid a confirmed Adreno crash in device-fault diagnostics.

Dependencies

• https://github.com/off-grid-ai/shared/pull/44
• https://github.com/off-grid-ai/mobile-pro/pull/84
• Add video as a Desktop modality with OGAD remote generation OGAD#164

All repositories use feature/local-video-generation.

Verification

• Android arm64 debug builds passed, including the decoder mitigation and driver diagnostic guard.
• iOS arm64 device app and runtime framework builds passed.
• Mobile production-source TypeScript checks passed. Shared Models and Sync builds passed.
• A physical iPhone generated and played a Wan 2.1 clip: 320×192, 17 frames, 20 steps, 11m47s.
• A later iPhone run uses 512×288, 33 frames, 30 steps. Its completed output is not yet verified.
• Android logs show HTP prompt/diffusion work and Vulkan VAE placement. The prior 320×192, 17-frame, 20-step run crashed after GPU device loss in the vendor fault-info callback. Smaller decoder graphs and the diagnostic guard are built; successful final output is still pending.
• No automated tests were written or run in this pass. Earlier branch commits contain previous test work.

Remaining checks and limits

• Keep this PR in draft. Verify a complete Android video, the higher-quality iPhone output, and a complete SD image on each phone.
• The mobile Image tab now queries Hugging Face and resolves compatible Qwen Image 2.1 GGUF variants into complete packs. It reuses the existing cards and download flow. Other SD image architectures are not yet resolved by this mobile path; existing Core ML/QNN/MNN catalogs remain available. This new search flow still needs device interaction verification.
• LTX and Hunyuan model-pack integration is present, but full generation has not been verified on devices. Image-to-video is not implemented.
• NPU support is specific to compatible Qualcomm hardware and operations. A preferred-backend label does not prove that each operation ran on the NPU. Apple video uses Metal; Apple Neural Engine video support is not implemented.
• Cross-device SD image-pack transfer, long-running background behavior, memory pressure, Stop/retry, and export still need focused device verification.

Summary by CodeRabbit

• New Features
  • Generate videos in chats using downloaded models or supported remote servers, with adjustable settings, progress updates, and cancellation.
  • Browse, download, manage, and select video models; view generated videos in the media gallery, play them in chats, and save or share them.
  • Generate images with compatible Stable Diffusion models and configure video models on supported remote servers.
  • Auto Setup can include a compatible video model when available.
• Improvements
  • Video downloads and generated media are restored and synced more reliably.
  • Storage usage includes downloaded video models, and image and video exports are supported.
  • Choose whether to show the development sync status bar in Settings.
• Bug Fixes
  • Fixed slider values reverting after editing them directly.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The pull request adds local and supported remote video generation. It adds video-model downloads, settings, chat integration, playback, export, and recovery. It also adds Stable Diffusion image-model support and native video-runtime builds for Android and iOS.

Changes

Video generation and model support

Layer / File(s)	Summary
Shared contracts and video model management src/types/*, src/stores/appStore.ts, src/types/remoteServer.ts, src/services/modelDownloadService/*, src/services/videoModelFiles.ts, src/screens/ModelsScreen/*	Shared types and persisted state include video turns, attachments, generated-video records, settings, video-model state, and remote video capability. The app adds video-pack discovery, downloads, and management.
Native runtime and platform builds android/app/src/main/cpp/*, android/app/src/main/java/ai/offgridmobile/video/*, native/video/*, scripts/*video*, scripts/video/*, ios/*, android/app/build.gradle, android/app/src/main/AndroidManifest.xml	Android and iOS add native video and image generation bindings, frame encoders, platform execution support, runtime build scripts, and runtime patches.
Generation orchestration and remote jobs src/services/videoGenerationService.ts, src/services/videoGenerator.ts, src/services/remoteMediaRuntime.ts, src/services/generationService.ts, src/services/generationRemoteHelpers.ts, src/services/tools/*, src/screens/ChatScreen/useChatGenerationActions.ts	Generation runs through a local video model or a supported remote server. The service tracks job state, progress, results, cancellation, and recovery. Tool flows can defer video requests.
Video controls and media presentation src/components/ChatInput/*, src/components/GenerationSettingsModal/*, src/components/ModelSelectorModal/*, src/components/VideoGenerationStatus.tsx, src/screens/GalleryScreen/*, src/components/ChatMessage/*	The UI adds video mode, model selection, generation settings and status, video attachments, and a gallery player with export and delete actions.
Stable Diffusion image support and app integration src/services/huggingFaceModelBrowser.ts, src/services/imageDownloadActions.ts, src/services/localDreamGenerator.ts, src/utils/imageModelIntegrity.ts, src/services/modelManager/*, App.tsx, src/screens/HomeScreen/*, src/screens/DownloadManagerScreen/*, src/screens/StorageSettingsScreen.tsx, src/services/sync/*, __tests__/unit/sync/sharedFileMaterializer.test.ts	Image-model discovery, validation, transfer, and local generation support the sd backend and multifile packs. Startup recovery, chat, storage, export, and sync flows account for video. A persisted development setting controls the sync-status strip.

Priority: ➖ Normal

Estimated code review effort: 5 (Critical) | ~90 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant ChatScreen
  participant VideoGenerationService
  participant VideoGenerator
  participant RemoteMediaRuntime
  participant AppStore
  ChatScreen->>VideoGenerationService: submit video request
  VideoGenerationService->>VideoGenerator: generate with local model
  VideoGenerationService->>RemoteMediaRuntime: generateVideo with remote server
  VideoGenerator-->>VideoGenerationService: return generated video path
  RemoteMediaRuntime-->>VideoGenerationService: return downloaded video and provenance
  VideoGenerationService->>AppStore: publish video and conversation attachment

Loading

Merge Risk: 🟡 Moderate · up to 6dc25

Several video-generation defects should be fixed before merge. An immediate re-download after cancelling can silently do nothing. Back-to-back Android jobs can cancel each other. iOS can crash if a negative prompt is missing. The Home screen shows a New Chat button that does nothing. Deferred video failures can break text turns and stall the queue.

Security Architecture Review

Security architecture risk: 🟡 Moderate · up to 6dc25

Local and remote video generation expands where prompts and downloaded model data flow. The available evidence shows several controls, but does not fully establish upstream model-integrity requirements or remote authorization.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The new boundaries are downloaded model files entering on-device native execution and user prompts and resulting media crossing to a configured remote service. The examined chat call does not supply an arbitrary remote endpoint.

Trust Boundaries and Controls

• observed — Remote video operations use job-specific routes; the client obtains the configured server API key for content download. This establishes client-side authenticated transport, not server-side authorization of job ownership.
• observed — Hash checking at pack resolution is conditional on supplied file metadata. The imported shared pack validator's requirements could not be established from the available repository source.

Resilience and Maintainability Implications

• observed — Remote cancellation aborts polling and transfer work and requests job cancellation; local cancellation reaches the native generator. Remote server-side cleanup after client interruption is not established by these client paths.

Hardening Proposals

• proposed — Establish whether the shared pack contract requires a trusted size and cryptographic hash for every native input, and require that property at the execution boundary if it does not.
• proposed — Confirm that the remote service authorizes job creation, polling, cancellation, previews, and content retrieval for the presented identity, including resumed jobs.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description provides detailed scope, dependencies, verification results, limitations, and remaining checks, but it does not follow the required template. It omits the Type of Change selection, man…	Restructure the description to use the required template. Select the applicable Type of Change options, add Android and iOS screenshots or recordings, complete the General, Testing, React Native Specific, Performance & Models, and Security …
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.51% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 109 functions across 66 files. (2 skipped…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the primary change: adding local and OGAD remote video generation to the mobile app.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description provides detailed scope, dependencies, verification results, limitations, and remaining checks, but it does not follow the required template. It omits the Type of Change selection, mandatory Android and iOS screenshots for the UI changes, the checklist sections, related issues, and additional notes.

Resolution

Restructure the description to use the required template. Select the applicable Type of Change options, add Android and iOS screenshots or recordings, complete the General, Testing, React Native Specific, Performance & Models, and Security checklists, and add related issues and additional notes. Keep the existing scope, verification results, and remaining limitations.

Full details: Docstring Coverage

Explanation

Docstring coverage is 16.51% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 109 functions across 66 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sonarqubecloud]

Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE