Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions src/main/lib/opsFlag.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@
* Ops flags are server config pushed TO the client (availability guards, rollout gates), not
* analytics collected FROM the user, so they read through `getOpsFlagResult`, which deliberately
* BYPASSES the consent gate — a user who declined telemetry still gets the override, and
* pre-consent surfaces can still resolve a value. The evaluation request supplies only the
* installation-stable key and the flag key; implicit flag events are disabled.
* pre-consent surfaces can still resolve a value. The evaluation request supplies the
* installation-stable key, the flag key and this build's `app_version` (plus `comfy_staff` for a
* consented staff install — see `opsFlagPersonProperties` in telemetry.ts); implicit flag events
* are disabled.
*
* Kept separate from `experiments.ts` (locked variant assignment, next-boot cache) so an
* operational override isn't accidentally consent-gated. Fetched once at boot; running apps
Expand Down
124 changes: 108 additions & 16 deletions src/main/lib/telemetry.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -549,9 +549,9 @@ describe('telemetry anonymous flag reads', () => {
{
key: 'desktop_core_beta_features',
distinctId: 'installation-id',
// Empty rather than absent: the request is made pre-consent by design, and this is
// Only the build's version: the request is made pre-consent by design, and this is
// where an email would sit if one were ever attached without one.
options: { sendFeatureFlagEvents: false, personProperties: {} }
options: { sendFeatureFlagEvents: false, personProperties: { app_version: '0.0.0' } }
}
])
})
Expand Down Expand Up @@ -622,7 +622,7 @@ describe('ops-flag person targeting', () => {

await evaluate()

expect(lastPersonProperties()).toEqual({ comfy_staff: 'true' })
expect(lastPersonProperties()).toEqual({ app_version: '0.0.0', comfy_staff: 'true' })
})

it('leaves the distinct id the installation hash, so bucketing is unchanged', async () => {
Expand Down Expand Up @@ -651,60 +651,60 @@ describe('ops-flag person targeting', () => {
expect(captured.map((event) => event.event)).not.toContain('$feature_flag_called')
})

it('sends nothing for a non-staff install', async () => {
it('sends no comfy_staff for a non-staff install', async () => {
// Absent rather than `comfy_staff: 'false'`: a `comfy_staff = true` condition does not match
// a missing property, so nothing goes on the wire for the majority of users.
// a missing property, so it stays off the wire for the majority of users.
setupTelemetry({ consent: 'granted' })
telemetry.setFlagEvaluationStaff(false)

await evaluate()

expect(lastPersonProperties()).toEqual({})
expect(lastPersonProperties()).toEqual({ app_version: '0.0.0' })
})

it('sends nothing when consent is undecided', async () => {
it('sends no comfy_staff when consent is undecided', async () => {
// The flag FETCH bypasses consent on purpose (ops flags are config pushed to the client);
// whether someone is an employee is a fact about them and does not inherit that exemption.
setupTelemetry({ consent: null })
telemetry.setFlagEvaluationStaff(true)

await evaluate()

expect(lastPersonProperties()).toEqual({})
expect(lastPersonProperties()).toEqual({ app_version: '0.0.0' })
})

it('sends nothing when consent is denied', async () => {
it('sends no comfy_staff when consent is denied', async () => {
setupTelemetry({ consent: 'denied' })
telemetry.setFlagEvaluationStaff(true)

await evaluate()

expect(lastPersonProperties()).toEqual({})
expect(lastPersonProperties()).toEqual({ app_version: '0.0.0' })
})

it('still evaluates the flag without consent, carrying no property', async () => {
it('still evaluates the flag without consent, carrying no staff property', async () => {
// The request itself must survive the consent gate — a declined user still gets ops
// overrides. Only the property is withheld.
// overrides. Only the staff property is withheld.
setupTelemetry({ consent: 'denied' })
telemetry.setFlagEvaluationStaff(true)
posthogClientMock.featureFlagResult = { enabled: true, variant: 'beta', payload: null }

await expect(
telemetry.getOpsFlagResult('desktop_core_beta_features', 'installation-id', 100)
).resolves.toMatchObject({ kind: 'value', value: 'beta' })
expect(lastPersonProperties()).toEqual({})
expect(lastPersonProperties()).toEqual({ app_version: '0.0.0' })
})

it('stops sending once the install is reclassified, as on sign-out', async () => {
setupTelemetry({ consent: 'granted' })
telemetry.setFlagEvaluationStaff(true)
await evaluate()
expect(lastPersonProperties()).toEqual({ comfy_staff: 'true' })
expect(lastPersonProperties()).toEqual({ app_version: '0.0.0', comfy_staff: 'true' })

telemetry.setFlagEvaluationStaff(false)
await evaluate()

expect(lastPersonProperties()).toEqual({})
expect(lastPersonProperties()).toEqual({ app_version: '0.0.0' })
})

it('does not survive a reset, so no classification leaks between launches in-process', async () => {
Expand All @@ -714,7 +714,7 @@ describe('ops-flag person targeting', () => {
setupTelemetry({ consent: 'granted' })
await evaluate()

expect(lastPersonProperties()).toEqual({})
expect(lastPersonProperties()).toEqual({ app_version: '0.0.0' })
})

// The bug in one test: with only a machine-derived distinct id, a person condition cannot
Expand Down Expand Up @@ -777,6 +777,98 @@ describe('ops-flag person targeting', () => {
).resolves.toEqual({ kind: 'value', value: false, payload: undefined })
})
})

// `app_version` describes the build, not the person, so unlike `comfy_staff` it rides on every
// evaluation. A grant only newer builds can apply must be gateable from the first launch, which
// is pre-consent.
describe('app_version', () => {
it.each([
['undecided', null],
['denied', 'denied'],
['granted', 'granted']
] as const)('is sent when consent is %s', async (_label, consent) => {
setupTelemetry({ consent, appVersion: '1.1.4' })

await evaluate()

expect(lastPersonProperties()).toEqual({ app_version: '1.1.4' })
})

it('rides alongside comfy_staff for a consented staff install', async () => {
setupTelemetry({ consent: 'granted', appVersion: '1.1.4' })
telemetry.setFlagEvaluationStaff(true)

await evaluate()

expect(lastPersonProperties()).toEqual({ app_version: '1.1.4', comfy_staff: 'true' })
})

it('is the version initTelemetry was given, verbatim, prerelease included', async () => {
// No normalisation: the flag condition compares the same string the events carry.
setupTelemetry({ consent: null, appVersion: '1.1.4-rc.1' })

await evaluate()

expect(lastPersonProperties()).toEqual({ app_version: '1.1.4-rc.1' })
})

it('is omitted rather than sent empty when the version is unknown', async () => {
// An empty string is not a version; absent is what a semver condition treats as no match.
setupTelemetry({ consent: null, appVersion: '' })

await evaluate()

expect(lastPersonProperties()).toEqual({})
})

// What the property is for: a payload carrying args older builds reject, gated server-side.
describe('against a stand-in `app_version semver_gte 1.1.4` condition', () => {
/** `version >= min` for a release `min`, enough for the versions below. Like PostHog's
* server (the `semver` crate), a prerelease sorts BELOW its release, and a missing value
* never matches. */
function isAtLeast(version: string | undefined, min: [number, number, number]): boolean {
if (!version) return false
const [release = '', ...prerelease] = version.split('-')
const [major = 0, minor = 0, patch = 0] = release.split('.').map(Number)
if (major !== min[0]) return major > min[0]
if (minor !== min[1]) return minor > min[1]
if (patch !== min[2]) return patch > min[2]
return prerelease.length === 0
}

function serveVersionGatedFlag(): void {
posthogClientMock.evaluateCondition = (options) => {
if (!isAtLeast(options?.personProperties?.['app_version'], [1, 1, 4])) {
return { enabled: false }
}
return {
enabled: true,
variant: 'beta',
payload: { flags: [{ arg: '--enable-assets-output-scanning' }] }
}
}
}

it.each(['1.1.4', '1.2.0'])('matches %s, even before consent', async (appVersion) => {
setupTelemetry({ consent: null, appVersion })
serveVersionGatedFlag()

await expect(
telemetry.getOpsFlagResult('desktop_core_beta_features', 'installation-id', 100)
).resolves.toMatchObject({ kind: 'value', value: 'beta' })
})

it.each(['1.1.3', '1.1.4-rc.1'])('does not match %s', async (appVersion) => {
// An RC of the gated release is below it, so it needs its own `semver_gte 1.1.4-rc.0`.
setupTelemetry({ consent: 'granted', appVersion })
serveVersionGatedFlag()

await expect(
telemetry.getOpsFlagResult('desktop_core_beta_features', 'installation-id', 100)
).resolves.toEqual({ kind: 'value', value: false, payload: undefined })
})
})
})
})

// A cold `/flags` POST measured ~2572 ms on Windows and is always cold at boot, so a 2000 ms
Expand Down
53 changes: 44 additions & 9 deletions src/main/lib/telemetry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ export function _resetForTest(): void {
installationIdProperty = null
consentState = 'undecided'
flagEvaluationStaff = false
flagEvaluationAppVersion = null
pendingSessionStart = null
pendingFirstLaunch = null
pendingPersonSet = null
Expand Down Expand Up @@ -365,6 +366,20 @@ let consentState: ConsentState = 'undecided'
*/
let flagEvaluationStaff = false

/**
* This build's version, sent with every ops-flag evaluation as `app_version` so
* a release condition can gate on it with PostHog's semver operators. Set once
* by `initTelemetry`; `null` until then, and no request goes out before it,
* because `getOpsFlagResult` needs the client that `initTelemetry` creates.
*
* Unlike `flagEvaluationStaff` it is NOT consent-gated. It describes the binary,
* not the person running it: every install of a release sends the same string.
* A grant that only newer builds can apply has to be gateable on the very first
* launch, and pre-consent is exactly when a version gate would otherwise
* evaluate differently from the same build after consent.
*/
let flagEvaluationAppVersion: string | null = null

/**
* Bind whether ops-flag evaluation should present this install as staff.
*
Expand All @@ -378,7 +393,23 @@ export function setFlagEvaluationStaff(isStaff: boolean): void {
/**
* Person properties for an ops-flag evaluation request.
*
* Empty unless consent is `'granted'` AND the account is staff. The flag FETCH
* `app_version` always (see `flagEvaluationAppVersion`). Builds from before it
* was added send none, and a semver condition does not match a missing
* property, so a `>=` gate evaluates closed for them — the direction a grant
* that older builds would reject needs. That covers the live evaluation only:
* `ops-flags.json` keeps the last answer without the version that earned it, so
* after a downgrade an `unreachable` launch still applies a grant a newer build
* persisted.
*
* PostHog orders prereleases by SemVer, so `1.1.5-rc.1` is below `1.1.5`; gate
* on `X-0` (e.g. `semver_gte 1.1.5-0`) to include X's prereleases. Unpackaged
* builds send `git describe` output (`1.1.4-5-gabc1234`, a prerelease of the
* last tag), or a bare commit id when no tag is reachable.
*
* The version is self-asserted, so a version gate is a compatibility gate, not
* a trust boundary: never gate anything a user could want withheld on it.
*
* `comfy_staff` only when consent is `'granted'` AND the account is staff. The flag FETCH
* itself bypasses the consent gate on purpose — ops flags are config pushed TO
* the client, and a user who declined telemetry still gets the override — but
* that argument covers the installation-stable key and nothing else. Whether a
Expand All @@ -392,8 +423,10 @@ export function setFlagEvaluationStaff(isStaff: boolean): void {
* overwhelming majority of users.
*/
function opsFlagPersonProperties(): Record<string, string> {
if (consentState !== 'granted' || !flagEvaluationStaff) return {}
return { comfy_staff: 'true' }
const properties: Record<string, string> = {}
if (flagEvaluationAppVersion) properties['app_version'] = flagEvaluationAppVersion
if (consentState === 'granted' && flagEvaluationStaff) properties['comfy_staff'] = 'true'
return properties
}

const PRE_CONSENT_ALLOWED_EVENTS: ReadonlySet<string> = new Set([
Expand Down Expand Up @@ -597,6 +630,7 @@ export function initTelemetry(opts: InitOptions): void {
if (initialized) return
initialized = true
bootstrapTimeMs = Date.now()
flagEvaluationAppVersion = opts.appVersion

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — For unpackaged builds opts.appVersion is git describe output (e.g. 1.1.1-5-gabc1234 or a bare commit SHA), which is local repo state rather than a shared release identifier. Since app_version is deliberately not consent-gated, that per-developer-machine string is sent to PostHog alongside the installation-stable id even when the user denied telemetry, which weakens the "every install of a release sends the same string" justification in the docblock above. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k2.7-code adversarial).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rejected: unpackaged builds only. Those are developers running from source, the value is a public-repo tag plus commit id with nothing personal in it, and sending it is what lets a developer exercise a version gate at all. Release builds send the shared release version.


// Set the per-event defaults BEFORE the early-return so disabled
// telemetry still gets a coherent snapshot (useful when tests stub
Expand Down Expand Up @@ -1670,18 +1704,19 @@ const OPS_FLAG_DEADLINE: unique symbol = Symbol('ops-flag-deadline')
* creates a PostHog person behind the capture policy.
*
* The evaluation call supplies the installation-stable evaluation key, the flag
* key, and — only once consent is `'granted'` and the account is staff —
* `comfy_staff`, via `opsFlagPersonProperties`. That property is
* request-scoped: the SDK puts it in the `/flags` POST body as
* `person_properties`, where the server evaluates release conditions against it
* and nothing is stored. The `distinct_id` remains the installation hash, so
* key, `app_version`, and — only once consent is `'granted'` and the account is
* staff — `comfy_staff`, via `opsFlagPersonProperties`. Those properties are
* request-scoped: the SDK puts them in the `/flags` POST body as
* `person_properties`, where the server evaluates release conditions against
* them and nothing is stored. The `distinct_id` remains the installation hash, so
* bucketing is unchanged and the machine is still not linked to any account in
* the person store.
*
* It exists because the installation hash CANNOT be resolved to a person: a
* condition on any person attribute can never match a machine-derived id, so
* staff targeting silently returned nothing for every install. Supplying a
* property on the request is what makes such a condition evaluable at all.
* property on the request is what makes such a condition evaluable at all. The
* same holds for `app_version`: it is what lets a grant be version-gated.
Comment thread
synap5e marked this conversation as resolved.
Resolved
*
* This is the ONLY evaluation. An earlier design added a second, authenticated
* one once a cloud view resolved auth; it could not work, because the
Expand Down
Loading