Skip to content

Initial (simple) Threading Model + UI + Auxiliary Fixes - #506

Open
ArielG-NV wants to merge 5 commits into
NVIDIA:mainfrom
ArielG-NV:refactor-api-v2/adding-ui-simple
Open

Initial (simple) Threading Model + UI + Auxiliary Fixes#506
ArielG-NV wants to merge 5 commits into
NVIDIA:mainfrom
ArielG-NV:refactor-api-v2/adding-ui-simple

Conversation

@ArielG-NV

Copy link
Copy Markdown
Collaborator

Resolves: #503
Resolves: #468

Primary Changes:

  • We now only have only two threads: application-thread == io-thead == ui-thread && model-thread
  • Hoist free floating presentation logic into manager that fits in ISession; Event managment is in an EventBuffer object as well rather than freely floating in run_session logic
  • Clean up keyboard UserInputEvent logic missing implementation logic for key-down vs. key-up
  • Clean up incorrect python syntax (Enum are CAPS, etc...)
  • Fix webrtc bug that caused it to fail on msft edge
  • Adjust webrtc logic so that when waiting for startup, we see a pretty UI sharing status
  • Added imgui demos
  • Separated benchmarking from Mp4 output sink so that we can record metrics after model-thread processing (individually from emitting final frame result which comes from ui-thread -- which will eventually have its own metrics as well!)
  • Added threading API & modified ISession API around the changes described above. This means adding docs & adjusting AGENTS.md slightly as well.
  • Added utility ImGUIThread for users to use for simple "just run with it" UI support via our UIThread system.

Adding simple threading change:
- only two threads: application-thread == io-thead == ui-thread && model-thread
- Hoist free floating presentation logic into manager that fits in ISession; Event managment is in an EventBuffer object as well rather than freely floating in run_session logic
- clean up keyboard state missing implementation logic
- fix webrtc bug that caused it to fail on msft edge
- adjust webrtc logic so that when waiting for startup... we see a nice ui telling us what happened
- added imgui demos
- seperated benchmarking from Mp4 output sink so that we can record metrics after model-thread processing (individually from emitting final frame result which comes from ui-thread -- which will eventually have its own metrics as well!)
- Added threading API & modified ISession API around the changes described above. This means adding docs & adjusting AGENTS.md slightly as well.
- Added utility ImGUIThread for users to use for simple "just run with it" UI support via our UIThread system.
@copy-pr-bot

copy-pr-bot Bot commented Aug 23, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@greptile-apps

greptile-apps Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR introduces a two-thread runtime model, moving model generation to a worker thread while the calling thread manages UI, input, presentation, and output lifecycle.

  • Adds thread registration, asynchronous state operations, event buffering, and presentation modes.
  • Moves benchmark metric collection to the model-result publication path.
  • Adds ImGui support and demo integrations.
  • Updates WebRTC startup behavior, keyboard events, CLI options, tests, documentation, dependencies, and CI graphics packages.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported benchmark sample-loss issue is resolved, and the MP4 redraw concern was acknowledged as intentional behavior.

Important Files Changed

Filename Overview
flashdreams/flashdreams/api_v2/thread.py Defines model/UI thread lifecycle, pacing, reset handling, asynchronous state operations, and model-result publication.
flashdreams/flashdreams/api_v2/session.py Replaces direct session stepping with explicit model and UI thread registration and coordinated shutdown.
flashdreams/flashdreams/runtime_v2/session_runner.py Implements the new two-thread execution loop and records every successfully published model result independently of presentation selection.
flashdreams/flashdreams/runtime_v2/presentation_manager.py Adds bounded model-frame buffering with blocking, latest-frame, and lossless presentation behavior.
flashdreams/flashdreams/runtime_v2/event_buffer.py Adds synchronized multi-reader event retention and reset-generation tracking.
flashdreams/flashdreams/runtime_v2/imgui_thread.py Adds the optional ImGui-based UI thread and rendering integration.
flashdreams/flashdreams/runtime_v2/serving/webrtc_server.py Adjusts WebRTC serving and startup behavior for browser compatibility and connection status presentation.
uv.lock Updates the workspace lockfile for the ImGui demo and optional UI dependency.

Sequence Diagram

sequenceDiagram
    participant UI as UI/Application Thread
    participant Events as EventBuffer
    participant Model as Model Thread
    participant Metrics as Metrics Sink
    participant Buffer as PresentationManager
    participant Window as Client Window
    UI->>Window: Collect user input
    UI->>Events: Append events
    Model->>Events: Read unread events
    Model->>Model: Run model step
    Model->>Buffer: Publish model results
    Model->>Metrics: Write per-channel metrics
    UI->>Buffer: Advance presented frame
    UI->>UI: Run UI step
    UI->>Window: Write rendered result
Loading

Reviews (6): Last reviewed commit: "ad opengl to runner workfow install scri..." | Re-trigger Greptile

Comment thread flashdreams/flashdreams/runtime_v2/session_runner.py Outdated
Comment thread flashdreams/flashdreams/runtime_v2/session_runner.py
@ArielG-NV

Copy link
Copy Markdown
Collaborator Author

/ok to test ff71a57

@ArielG-NV
ArielG-NV force-pushed the refactor-api-v2/adding-ui-simple branch from a2154dc to 883f9ff Compare August 24, 2026 03:35
@ArielG-NV

Copy link
Copy Markdown
Collaborator Author

/ok to test 6fba61b

@ArielG-NV
ArielG-NV force-pushed the refactor-api-v2/adding-ui-simple branch from 6fba61b to d98a279 Compare August 24, 2026 04:13
@ArielG-NV

Copy link
Copy Markdown
Collaborator Author

/ok to test a2048da

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Presentation to Client Window Logic Should Be Tied Closely To ISession [API] Add Proper UI (input & output) Pass

1 participant