feat: capture platform views in session replay screenshots#453
feat: capture platform views in session replay screenshots#453turnipdabeets wants to merge 8 commits into
Conversation
bdeec5d to
76b62b3
Compare
|
d8919fb to
e491499
Compare
|
8cb2b93 to
fdb235a
Compare
| } | ||
| } | ||
|
|
||
| Future<Uint8List?> captureNativeScreenshot({ |
There was a problem hiding this comment.
I think this is worth discussing first before patching around it.
Today this is one call per captured view, per frame, each carrying a png data (could be low/high MB range?) Can we maybe design this to be 1 call per snapshot at least?
| try { | ||
| final bytes = await _channel.invokeMethod<Uint8List>( | ||
| 'captureNativeScreenshot', | ||
| {'x': x, 'y': y, 'width': width, 'height': height}, |
There was a problem hiding this comment.
Instead of geometry, can we be passing and threading flutter view id here? Then native side can just lookup per id?
There was a problem hiding this comment.
The challenge is on the native side: we'd need platform-specific lookup logic in each plugin, and the API for that (FlutterView.getPlatformViewsController() on Android) is internal. Geometry lets us use PixelCopy / WKWebView.takeSnapshot uniformly across all composition modes without touching Flutter internals.
Open to revisiting if you have a native lookup path in mind — what API on the Android or iOS side were you thinking of?
There was a problem hiding this comment.
Was thinking about getFlutterEngine().getPlatformViewsControllerDelegator().getPlatformViewById() for Android but you are right, this is not symmetric between iOS and Android. So seems possible on Android, but not on iOS. I'd rather have symmetric implementations between the two so we can drop the idea
| pixelRatio, | ||
| ); | ||
| } | ||
| for (final capturedRect in pvRects.captured) { |
There was a problem hiding this comment.
see https://github.com/PostHog/posthog-flutter/pull/453/changes#r3497391223
this essentially becomes final results = await _nativeCommunicator.captureNativeScreenshots() with one await, then we loop through the return list and composite
|
|
||
| private fun bitmapToPng(bitmap: Bitmap): ByteArray { | ||
| val outputStream = ByteArrayOutputStream() | ||
| bitmap.compress(Bitmap.CompressFormat.PNG, 100, outputStream) |
There was a problem hiding this comment.
Wonder if there is a way to avoid encoding/decoding PNG-100 during the native hop from dart? Batching will also help.
Maybe for this path we encode into something cheaper since it's just capturing the platform view snapshot?
There was a problem hiding this comment.
Updated — raw RGBA (skipping PNG entirely). Native returns raw pixel bytes, Dart decodes with decodeImageFromPixels. No codec round-trip.
posthog-flutter Compliance ReportDate: 2026-06-30 21:15:09 UTC ✅ All Tests Passed!45/45 tests passed Capture Tests✅ 29/29 tests passed View Details
Feature_Flags Tests✅ 16/16 tests passed View Details
|
Platform views (WebView, Maps, etc.) are masked by default in session replay — they appear as a black box rather than being captured or leaking native content. - Adds `maskAllPlatformViews` config flag (default `true`) - Adds `PostHogPlatformView` marker widget + `PostHogPlatformViewPrivacy` enum (`.mask` / `.capture`) for per-view override - iOS: `WKWebView.takeSnapshot` for capture; no `drawHierarchy` fallback (would leak masked CALayer-backed views into replay) - Android: PixelCopy + async bitmap compression; de-duplicates frames via `compositedBytesHash` - 5-second timeout on `captureNativeScreenshot` to prevent replay freeze - Example app: 8 test flows covering masked/captured combinations Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rame Previously _compositeRevealedView made one captureNativeScreenshot call per captured platform view per frame, blocking on each round-trip before starting the next. N views = N sequential channel crossings, each carrying a PNG payload. Replace with captureNativeScreenshots (plural): a single channel call per frame that accepts a list of view specs and returns a list of byte arrays. The Dart side computes all geometries upfront, dispatches one awaited call, then loops to composite. On Android and iOS the single- view work is extracted into an internal callback-based helper and the batch handler chains them sequentially. Also adds _viewSpec helper to screenshot_capturer.dart so geometry computation is separated from compositing, and updates tests for the new API. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ktlint auto-format: import ordering, statement wrapping, argument list wrapping, multi-space alignment in PosthogFlutterPlugin.kt - swiftformat auto-format: elseOnSameLine, wrapMultilineStatementBraces, preferKeyPath in PosthogFlutterPlugin.swift - Restore codec?.dispose() in finally block in _decodeImage (was lost when screenshot_capturer.dart was resynced during the batch refactor) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Android: replace bitmap.compress(PNG) with getPixels() + RGBA packing. iOS: replace pngData() with a CGBitmapContext raw-pixel extraction. Dart: replace ui.instantiateImageCodec with ui.decodeImageFromPixels, passing the already-known spec dimensions from _viewSpec. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
8182d17 to
8d17788
Compare
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…w capture - iOS imageToRawRgba: premultipliedLast to noneSkipLast; WKWebView snapshots are opaque so premultiplied bytes caused double-multiply against Skia - Dart: guard bytesList[i] with bounds check to prevent RangeError on short response - Android captureNext: add w<=0||h<=0 guard matching the singular handler - PostHogPlatformViewPrivacy.capture: document iOS WKWebView-only limitation - Add platform_view_privacy_test.dart for tree-walk privacy inheritance logic Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
WKWebView.takeSnapshot returns a UIImage at UIScreen.main.scale (2x/3x on Retina). cgImage.width/height are physical pixels; Dart decodes the buffer at logical-point dimensions from _viewSpec. The mismatch produced a byte buffer 4x/9x too large, garbling the WebView image in every replay frame on non-1x devices. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
| } | ||
|
|
||
| val svBitmap = Bitmap.createBitmap(svLogW, svLogH, Bitmap.Config.ARGB_8888) | ||
| PixelCopy.request( |
There was a problem hiding this comment.
Could this throw, and should we wrap?
💡 Motivation and Context
Platform views (WebView, Google Maps, etc.) rendered by Flutter are composited outside the Flutter layer tree. Without explicit handling, session replay either silently skips them (leaving blank gaps) or could inadvertently capture sensitive content. This PR adds first-class support: views are masked by default (black box), with opt-in capture on a per-view or global basis.
Closes #393.
Addresses only part 1 of #151
Note I'll address fully native views in a separate PR, that will require iOS and android SDK work as well.
💚 How did you test it?
See videos:
Android: https://us.posthog.com/shared/AYnlFG5M-DGczYUYyn27g1EC55HvDg?t=1
https://us.posthog.com/shared/kWlqsDn5LUynHgynC0Mp1t_wnAfK5w?t=2
iOS: https://us.posthog.com/shared/2G_GNlw70uTOwJQOdVcZFY_uiDasTA?t=5
📝 Checklist
If releasing new changes
pnpm changesetto generate a changeset file🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Authored with Claude Code (claude-sonnet-4-6). Anna Garcia directed the work throughout.
Key decisions made during the session: