Skip to content

feat: capture platform views in session replay screenshots#453

Open
turnipdabeets wants to merge 8 commits into
mainfrom
pr-393-platform-views
Open

feat: capture platform views in session replay screenshots#453
turnipdabeets wants to merge 8 commits into
mainfrom
pr-393-platform-views

Conversation

@turnipdabeets

@turnipdabeets turnipdabeets commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Platform views (WebView, Google Maps, etc.) rendered by Flutter are composited outside the Flutter layer tree. Without explicit handling, session replay either silently skips them (leaving blank gaps) or could inadvertently capture sensitive content. This PR adds first-class support: views are masked by default (black box), with opt-in capture on a per-view or global basis.

Closes #393.
Addresses only part 1 of #151

Note I'll address fully native views in a separate PR, that will require iOS and android SDK work as well.

💚 How did you test it?

  • iOS simulator (iPhone 17, iOS 26.4): ran all 8 example app test flows — masked views appear as black boxes in PostHog replay, captured WebViews show actual content
  • Android device (Pixel 4, Android 13): same flows verified
  • flutter test in posthog_flutter/ — all tests pass
  • Confirmed 5-second timeout fires correctly when native channel does not respond

See videos:

Android: https://us.posthog.com/shared/AYnlFG5M-DGczYUYyn27g1EC55HvDg?t=1
https://us.posthog.com/shared/kWlqsDn5LUynHgynC0Mp1t_wnAfK5w?t=2

iOS: https://us.posthog.com/shared/2G_GNlw70uTOwJQOdVcZFY_uiDasTA?t=5

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Authored with Claude Code (claude-sonnet-4-6). Anna Garcia directed the work throughout.

Key decisions made during the session:

  • iOS: no drawHierarchy fallback — when no WKWebView is found for the captured rect, we return nil rather than falling back to drawHierarchy over the full window. The fallback would composite any masked CALayer-backed platform view in the crop region and leak it into replay, defeating the privacy guarantee.
  • Android: async bitmap compression — PixelCopy runs on the main thread; bitmap compression was moved to a single-thread executor to avoid blocking the UI thread during screenshot capture.
  • 5-second timeout on captureNativeScreenshot — if the native side never calls result(), the Dart _isCapturing flag would stay true permanently and silently kill replay. The timeout degrades gracefully to null (frame skipped) rather than freezing.
  • hasCapturedPlatformViews gate — scheduleFrame() is only forced when at least one platform view was actually captured, avoiding unnecessary redraws on screens with only masked views.

@turnipdabeets turnipdabeets force-pushed the pr-393-platform-views branch 2 times, most recently from bdeec5d to 76b62b3 Compare June 25, 2026 17:05
@greptile-apps

greptile-apps Bot commented Jun 25, 2026

Copy link
Copy Markdown

Comments Outside Diff (2)

  1. posthog_flutter/test/posthog_test.dart, line 2193-2207 (link)

    P1 Broken test references non-existent property

    This test accesses config.sessionReplayConfig.capturePlatformViews, but PostHogSessionReplayConfig has no such property — the field added in this PR is maskAllPlatformViews. Dart will reject this at compile time, and the test cannot be one of the "47 passing tests" without this being caught. The final assertion also checks for a 'capturePlatformViews' key in the serialised map, but the config serialises under 'maskAllPlatformViews'. The test needs to be rewritten against the actual API: check that maskAllPlatformViews defaults to true, that it appears in the serialised map, and that setting it to false removes the masking.

  2. example/ios/Runner.xcodeproj/project.pbxproj, line 208-218 (link)

    P1 Personal developer credentials committed to a public repository

    DEVELOPMENT_TEAM = QU5XHRZES9 and PRODUCT_BUNDLE_IDENTIFIER = com.annagarcia.posthogFlutterExample appear to be personal Apple Developer credentials. Any contributor who clones this repo and tries to build the iOS example app will immediately hit a signing error (Xcode can't use someone else's team ID), and the personal name in the bundle identifier is clearly not intended to be canonical. These should revert to DEVELOPMENT_TEAM = "" (or the original value) and PRODUCT_BUNDLE_IDENTIFIER = com.example.posthogFlutterExample. The same values appear in the Release and profile build configurations in this file.

Reviews (1): Last reviewed commit: "feat: capture platform views in session ..." | Re-trigger Greptile

Comment thread posthog_flutter/lib/src/replay/screenshot/screenshot_capturer.dart Outdated
Comment thread posthog_flutter/lib/src/replay/mask/posthog_platform_view.dart Outdated
@turnipdabeets turnipdabeets force-pushed the pr-393-platform-views branch 2 times, most recently from d8919fb to e491499 Compare June 25, 2026 17:15
@turnipdabeets turnipdabeets marked this pull request as ready for review June 25, 2026 17:21
@turnipdabeets turnipdabeets requested a review from a team as a code owner June 25, 2026 17:21
@greptile-apps

greptile-apps Bot commented Jun 25, 2026

Copy link
Copy Markdown

Security Review

  • PostHogPlatformView(privacy: .mask) silently ignored when maskAllPlatformViews = false (screenshot_capturer.dart line 390): When the global flag is false, _collectPlatformViewRects() is never called, so per-view mask overrides are dropped. A developer who opts out of global masking but wraps a specific sensitive WebView in PostHogPlatformView(privacy: .mask) will find that view captured unmasked in replay. Fix: always traverse the element tree; pass the default privacy based on the global flag so undecorated views default to capture while explicit mask wrappers still apply.

Reviews (2): Last reviewed commit: "feat: capture platform views in session ..." | Re-trigger Greptile

Comment thread posthog_flutter/lib/src/replay/screenshot/screenshot_capturer.dart Outdated
@turnipdabeets turnipdabeets force-pushed the pr-393-platform-views branch 2 times, most recently from 8cb2b93 to fdb235a Compare June 25, 2026 19:11
}
}

Future<Uint8List?> captureNativeScreenshot({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is worth discussing first before patching around it.

Today this is one call per captured view, per frame, each carrying a png data (could be low/high MB range?) Can we maybe design this to be 1 call per snapshot at least?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

batched in ae7a7e3

try {
final bytes = await _channel.invokeMethod<Uint8List>(
'captureNativeScreenshot',
{'x': x, 'y': y, 'width': width, 'height': height},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of geometry, can we be passing and threading flutter view id here? Then native side can just lookup per id?

@turnipdabeets turnipdabeets Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The challenge is on the native side: we'd need platform-specific lookup logic in each plugin, and the API for that (FlutterView.getPlatformViewsController() on Android) is internal. Geometry lets us use PixelCopy / WKWebView.takeSnapshot uniformly across all composition modes without touching Flutter internals.

Open to revisiting if you have a native lookup path in mind — what API on the Android or iOS side were you thinking of?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was thinking about getFlutterEngine().getPlatformViewsControllerDelegator().getPlatformViewById() for Android but you are right, this is not symmetric between iOS and Android. So seems possible on Android, but not on iOS. I'd rather have symmetric implementations between the two so we can drop the idea

pixelRatio,
);
}
for (final capturedRect in pvRects.captured) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see https://github.com/PostHog/posthog-flutter/pull/453/changes#r3497391223

this essentially becomes final results = await _nativeCommunicator.captureNativeScreenshots() with one await, then we loop through the return list and composite


private fun bitmapToPng(bitmap: Bitmap): ByteArray {
val outputStream = ByteArrayOutputStream()
bitmap.compress(Bitmap.CompressFormat.PNG, 100, outputStream)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if there is a way to avoid encoding/decoding PNG-100 during the native hop from dart? Batching will also help.

Maybe for this path we encode into something cheaper since it's just capturing the platform view snapshot?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated — raw RGBA (skipping PNG entirely). Native returns raw pixel bytes, Dart decodes with decodeImageFromPixels. No codec round-trip.

@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

posthog-flutter Compliance Report

Date: 2026-06-30 21:15:09 UTC
Duration: 96921ms

✅ All Tests Passed!

45/45 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 150ms
Format Validation.Event Has Uuid 123ms
Format Validation.Event Has Lib Properties 117ms
Format Validation.Distinct Id Is String 118ms
Format Validation.Token Is Present 116ms
Format Validation.Custom Properties Preserved 120ms
Format Validation.Event Has Timestamp 119ms
Retry Behavior.Retries On 503 5334ms
Retry Behavior.Does Not Retry On 400 2121ms
Retry Behavior.Does Not Retry On 401 2117ms
Retry Behavior.Respects Retry After Header 8126ms
Retry Behavior.Implements Backoff 15458ms
Retry Behavior.Retries On 500 5228ms
Retry Behavior.Retries On 502 5234ms
Retry Behavior.Retries On 504 5231ms
Retry Behavior.Max Retries Respected 15447ms
Deduplication.Generates Unique Uuids 126ms
Deduplication.Preserves Uuid On Retry 5226ms
Deduplication.Preserves Uuid And Timestamp On Retry 10329ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 5232ms
Deduplication.No Duplicate Events In Batch 126ms
Deduplication.Different Events Have Different Uuids 119ms
Compression.Sends Gzip When Enabled 118ms
Batch Format.Uses Proper Batch Structure 116ms
Batch Format.Flush With No Events Sends Nothing 111ms
Batch Format.Multiple Events Batched Together 125ms
Error Handling.Does Not Retry On 403 2119ms
Error Handling.Does Not Retry On 413 2119ms
Error Handling.Retries On 408 5224ms

Feature_Flags Tests

16/16 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 14ms
Request Payload.Flags Request Uses V2 Query Param 10ms
Request Payload.Flags Request Hits Flags Path Not Decide 9ms
Request Payload.Flags Request Omits Authorization Header 10ms
Request Payload.Token In Flags Body Matches Init 9ms
Request Payload.Groups Round Trip 10ms
Request Payload.Groups Default To Empty Object 9ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It 10ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 10ms
Request Payload.Disable Geoip Omitted Defaults To False 10ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 9ms
Request Lifecycle.No Flags Request On Init Alone 5ms
Request Lifecycle.No Flags Request On Normal Capture 112ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 15ms
Request Lifecycle.Mock Response Value Is Returned To Caller 10ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 115ms

turnipdabeets and others added 4 commits June 30, 2026 14:30
Platform views (WebView, Maps, etc.) are masked by default in session
replay — they appear as a black box rather than being captured or
leaking native content.

- Adds `maskAllPlatformViews` config flag (default `true`)
- Adds `PostHogPlatformView` marker widget + `PostHogPlatformViewPrivacy`
  enum (`.mask` / `.capture`) for per-view override
- iOS: `WKWebView.takeSnapshot` for capture; no `drawHierarchy` fallback
  (would leak masked CALayer-backed views into replay)
- Android: PixelCopy + async bitmap compression; de-duplicates frames
  via `compositedBytesHash`
- 5-second timeout on `captureNativeScreenshot` to prevent replay freeze
- Example app: 8 test flows covering masked/captured combinations

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rame

Previously _compositeRevealedView made one captureNativeScreenshot call
per captured platform view per frame, blocking on each round-trip before
starting the next. N views = N sequential channel crossings, each
carrying a PNG payload.

Replace with captureNativeScreenshots (plural): a single channel call
per frame that accepts a list of view specs and returns a list of byte
arrays. The Dart side computes all geometries upfront, dispatches one
awaited call, then loops to composite. On Android and iOS the single-
view work is extracted into an internal callback-based helper and the
batch handler chains them sequentially.

Also adds _viewSpec helper to screenshot_capturer.dart so geometry
computation is separated from compositing, and updates tests for the
new API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ktlint auto-format: import ordering, statement wrapping, argument
  list wrapping, multi-space alignment in PosthogFlutterPlugin.kt
- swiftformat auto-format: elseOnSameLine, wrapMultilineStatementBraces,
  preferKeyPath in PosthogFlutterPlugin.swift
- Restore codec?.dispose() in finally block in _decodeImage (was lost
  when screenshot_capturer.dart was resynced during the batch refactor)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Android: replace bitmap.compress(PNG) with getPixels() + RGBA packing.
iOS: replace pngData() with a CGBitmapContext raw-pixel extraction.
Dart: replace ui.instantiateImageCodec with ui.decodeImageFromPixels,
passing the already-known spec dimensions from _viewSpec.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@turnipdabeets turnipdabeets force-pushed the pr-393-platform-views branch from 8182d17 to 8d17788 Compare June 30, 2026 18:30
@turnipdabeets turnipdabeets requested a review from ioannisj June 30, 2026 19:31
turnipdabeets and others added 4 commits June 30, 2026 15:31
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…w capture

- iOS imageToRawRgba: premultipliedLast to noneSkipLast; WKWebView snapshots
  are opaque so premultiplied bytes caused double-multiply against Skia
- Dart: guard bytesList[i] with bounds check to prevent RangeError on short response
- Android captureNext: add w<=0||h<=0 guard matching the singular handler
- PostHogPlatformViewPrivacy.capture: document iOS WKWebView-only limitation
- Add platform_view_privacy_test.dart for tree-walk privacy inheritance logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
WKWebView.takeSnapshot returns a UIImage at UIScreen.main.scale (2x/3x on
Retina). cgImage.width/height are physical pixels; Dart decodes the buffer
at logical-point dimensions from _viewSpec. The mismatch produced a byte
buffer 4x/9x too large, garbling the WebView image in every replay frame
on non-1x devices.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
}

val svBitmap = Bitmap.createBitmap(svLogW, svLogH, Bitmap.Config.ARGB_8888)
PixelCopy.request(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this throw, and should we wrap?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants