Skip to content

Add WebRTC ClientWindow to t2v app - #498

Draft
gtong-nv wants to merge 6 commits into
mainfrom
dev/gtong/t2v-webrtc-2
Draft

Add WebRTC ClientWindow to t2v app#498
gtong-nv wants to merge 6 commits into
mainfrom
dev/gtong/t2v-webrtc-2

Conversation

@gtong-nv

Copy link
Copy Markdown
Collaborator

No description provided.

@copy-pr-bot

copy-pr-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds persistent, prompt-driven WebRTC sessions to the v2 text-to-video runtime while keeping application state loaded between rollouts.

  • Introduces explicit session-description requests and application-owned initialized defaults.
  • Adds browser new_session events, persistent runner lifecycle handling, frame-generation flushing, and WebRTC reconnection.
  • Updates T2V prompt resolution, integration usage, tests, and WebRTC documentation.

Confidence Score: 4/5

The stale-peer callback must be fixed before merging because it can terminate a newly reconnected browser session.

Reconnection permits a replacement peer while callbacks from the prior peer may still run, and those callbacks mutate shared connection state and enqueue a close event before checking peer identity.

Files Needing Attention: flashdreams/flashdreams/runtime_v2/serving/webrtc_server.py

Important Files Changed

Filename Overview
flashdreams/flashdreams/runtime_v2/serving/webrtc_server.py Adds reusable media-track generations and peer reconnection, but stale peer callbacks can disconnect a replacement client.
flashdreams/flashdreams/runtime_v2/session_runner.py Adds replacement-session detection and coordinated session/window teardown while preserving windows between sessions.
flashdreams/flashdreams/runtime_v2/application_runner.py Separates application initialization from sessions and adds persistent session serving.
flashdreams/flashdreams/t2v_v2/application.py Allows prompts to come from per-session metadata while retaining an optional command-line default.
flashdreams/flashdreams/runtime_v2/serving/web/app.js Adds prompt submission and retries WebRTC admission conflicts to support persistent serving.

Sequence Diagram

sequenceDiagram
  participant Browser
  participant Server as WebRTCServer
  participant Runner as ApplicationRunner
  participant Session
  Browser->>Server: Connect and open controls channel
  Browser->>Server: new_session(prompt)
  Server->>Runner: Buffered NewSession event
  Runner->>Session: create/init/run
  Session-->>Server: Video frames
  Server-->>Browser: WebRTC video
  Browser--xServer: Disconnect
  Runner->>Session: close
  Browser->>Server: Reconnect and request next session
Loading

Comments Outside Diff (1)

  1. flashdreams/flashdreams/runtime_v2/serving/webrtc_server.py, line 383-387 (link)

    P1 Stale peer disconnects replacement

    When a replacement data channel opens before the old peer's terminal-state callback runs, this callback clears the shared connection state and queues a close event before _release_peer_connection rejects the stale peer, causing the newly connected session to stop.

    Knowledge Base Used: WebRTC Serving Flow

Reviews (1): Last reviewed commit: "Add persistent WebRTC sessions for v2 te..." | Re-trigger Greptile

@gtong-nv
gtong-nv marked this pull request as draft August 21, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant