Skip to content

feat(telemetry): send app_version with ops-flag evaluation - #1602

Merged
synap5e merged 6 commits into
mainfrom
synap5e/feat/ops-flag-app-version
Oct 2, 2026
Merged

synap5e merged 6 commits into
mainfrom
synap5e/feat/ops-flag-app-version

Conversation

@synap5e

@synap5e synap5e commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Desktop now sends its version as app_version when it evaluates ops flags. A PostHog release condition can then gate a flag payload on the Desktop version with the semver operators, e.g. app_version semver_gte 1.1.5-0.

Problem

Some Core-arg grants only work on newer Desktops. #1594 adds the --enable/--disable-assets-output-scanning grants, and older Desktops reject args they don't know. The payload that carries those grants has to reach only the builds that accept them. Right now the server has nothing to compare against.

Cause

Ops flags (desktop_core_beta_features, read through makeOpsFlag → getOpsFlagResult) are evaluated with the installation id and, at most, comfy_staff. No version is sent. The experiments path already sends app_version, but ops flags never did.

Scope
  • opsFlagPersonProperties() now always includes app_version. It is the same value the events carry (getAppVersion(), via initTelemetry).
  • Sent regardless of consent. This follows the reasoning the file already uses. The ops-flag fetch is exempt from consent because it pushes config to the client. comfy_staff stays consent-gated because it is a fact about the person. The app version describes the build: every install of a release sends the same string. A grant that only newer builds can apply also has to be gateable on the first launch, and that launch happens before consent.
  • As before, the properties are request-scoped person_properties. sendFeatureFlagEvents stays false and nothing is stored.

Not access control: app_version is reported by the client, so a version gate is a compatibility gate only.

Unchanged: comfy_staff is still sent only with consent granted and a staff account. The distinct id is unchanged, and so are the value/unreachable classification, persistence and revocation.

Trade-offs:

  • Desktops released before this change send no app_version. A semver condition does not match a missing property, so a semver_gte gate excludes them. That is the intended fail-closed direction.
  • PostHog's server orders prereleases by SemVer: 1.1.4-rc.1 is below 1.1.4. To include a release's prereleases, gate on X-0, e.g. semver_gte 1.1.5-0.
  • ops-flags.json persists the last answer. If a user downgrades from a build that received a version-gated grant and the older build's boot fetch is unreachable, the older build holds the cached grant. That takes a downgrade and an offline launch, and it is not addressed here.
  • Unpackaged dev builds send git describe output (e.g. 1.1.1-5-gabc1234). It compares as a prerelease of the last tag, so test version gates with a packaged build or a hand-built /flags request.

QA

  • Server-side semver check against PostHog /flags, using a throwaway flag with app_version semver_gte 1.1.4-rc.0. 1.1.4, 1.2.0 and 1.1.4-rc.1 match. 1.1.3, the dev form 1.1.1-5-gabc1234, and no app_version do not.
  • Confirm a packaged build of this branch sends app_version in the /flags request body, both before consent and after.
Implementation details
  • telemetry.ts: a new module variable, flagEvaluationAppVersion. initTelemetry sets it before its config early-return, and _resetForTest clears it. It is always set before any request can go out, because getOpsFlagResult returns unreachable until initTelemetry has created the client.
  • opsFlagPersonProperties() builds { app_version }, plus comfy_staff: 'true' under the unchanged consent-and-staff condition. An empty version is omitted rather than sent as ''.
  • The docstrings in telemetry.ts and the header of opsFlag.ts are updated to say what the request carries, and how to write a version gate: prereleases sort below their release, so gate on X-0 to include them.

Change breakdown (total changed lines = added + deleted = 183):

Category Files Added Deleted Share
Product code 2: src/main/lib/telemetry.ts, src/main/lib/opsFlag.ts (comment-only) 48 11 32.2%
Test code 1: src/main/lib/telemetry.test.ts 108 16 67.8%
Tests

src/main/lib/telemetry.test.ts, in "ops-flag person targeting":

  • app_version is sent when consent is undecided, denied and granted. It rides alongside comfy_staff for a consented staff install.
  • The version is sent verbatim, prerelease included. An empty version is omitted.
  • The existing staff tests now expect app_version next to or instead of comfy_staff. They still show comfy_staff is withheld without consent or for non-staff.
  • A stand-in app_version semver_gte 1.1.4 condition, with strict SemVer prerelease ordering like PostHog's server:
    • 1.1.4 and 1.2.0 match, even before consent.
    • 1.1.3 and 1.1.4-rc.1 do not.

These tests fail if app_version is consent-gated: 9 failures when the change is reverted to send it only with consent granted.

Passing: vitest for telemetry, opsFlag, coreBetaGrants, staffFlagTargeting and cloudFreeRuns (434 tests), plus typecheck:node, eslint and prettier.

Ops flags were evaluated with comfy_staff as the only person property, so a
release condition could not gate a payload on the Desktop version. Send the
build's app_version on every evaluation, independent of consent: it describes
the binary, not the person, and a grant only newer builds can apply must be
gateable before consent is given. comfy_staff keeps its consent+staff gate.

Builds from before this change send no version, so a semver_gte condition
excludes them.
PostHog's server evaluates semver conditions with the semver crate, so
1.1.4-rc.1 sorts below 1.1.4. Make the test stand-in match, cover an RC,
document how to include RCs, and correct the dev-build version example
(getAppVersion strips the leading v).
@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Team

Run ID: defeb55a-6436-48f8-8dda-42f7341b1b5d

📥 Commits

Reviewing files that changed from the base of the PR and between 6c7b78d and 4d05936.

📒 Files selected for processing (1)
  • src/main/lib/telemetry.ts

Included review availability: This review used your included allowance. 3 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour. Your free on-demand review promotion remains active until October 9, 2026 at 6:00 PM UTC.


📝 Walkthrough

Walkthrough

Operational-flag requests now include the app version when available, regardless of consent. The comfy_staff property remains limited to staff accounts with granted consent. Tests cover consent states, version comparisons, prerelease versions, and missing versions.

Changes

Ops-flag evaluation properties

Layer / File(s) Summary
Build-version and staff properties
src/main/lib/telemetry.ts, src/main/lib/telemetry.test.ts, src/main/lib/opsFlag.ts
Telemetry stores the app version during initialization and adds it to flag-evaluation properties when available. It adds comfy_staff only for consented staff accounts. Tests cover consent states, version conditions, prerelease versions, and missing versions. Documentation describes the request properties and version-targeting behavior.

Priority: ⬇️ Low

Merge Risk: 🔵 Low · up to 4d059

Some unpackaged users who deny consent will still send a locally derived Git build identifier to PostHog alongside the stable install ID. This is bounded metadata exposure; merging is possible with owner awareness.

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Comment @coderabbitai help to get the list of available commands.

@synap5e synap5e added the cursor-review Trigger multi-model Cursor code review label Sep 29, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @synap5e.

Found 4 finding(s).

Severity Count
🟡 Medium 1
🟢 Low 2
⚪ Nit 1

Panel: 8/8 reviewers contributed findings.

Comment thread src/main/lib/telemetry.ts Resolved
Comment thread POSTHOG_TELEMETRY_EVENTS.md Resolved
Comment thread src/main/lib/telemetry.ts
if (initialized) return
initialized = true
bootstrapTimeMs = Date.now()
flagEvaluationAppVersion = opts.appVersion

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low — For unpackaged builds opts.appVersion is git describe output (e.g. 1.1.1-5-gabc1234 or a bare commit SHA), which is local repo state rather than a shared release identifier. Since app_version is deliberately not consent-gated, that per-developer-machine string is sent to PostHog alongside the installation-stable id even when the user denied telemetry, which weakens the "every install of a release sends the same string" justification in the docblock above. Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k2.7-code adversarial).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rejected: unpackaged builds only. Those are developers running from source, the value is a public-repo tag plus commit id with nothing personal in it, and sending it is what lets a developer exercise a version gate at all. Release builds send the shared release version.

Comment thread src/main/lib/telemetry.ts Resolved
…s not access control

A semver gate evaluates closed for builds without app_version, but the
persisted ops-flags.json answer is reapplied on an unreachable launch, so a
downgraded build can still apply a cached grant. The version is
client-reported, so a gate on it is a compatibility gate only. Also note
the bare-commit fallback of git describe --always for dev builds.
@synap5e
synap5e marked this pull request as ready for review September 30, 2026 17:15
@synap5e
synap5e requested review from a team as code owners September 30, 2026 17:15
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-30T17:18:24.572072Z 6c7b78d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

…-app-version

# Conflicts:
#	POSTHOG_TELEMETRY_EVENTS.md
…ormat

The section in POSTHOG_TELEMETRY_EVENTS.md went with that file; keep the two
notes the docblock did not already carry.
@synap5e-bot

synap5e-bot Bot commented Oct 2, 2026

Copy link
Copy Markdown

/swarmhost evaluate

@synap5e

synap5e commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

/swarmhost evaluate

@comfy-greenlight-bot

comfy-greenlight-bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Swarmhost agentic review

The detailed evaluation is available to employees in the internal Slack review thread.

Evaluation budget remaining for this pull request: 98 automatic and 99 manual.

Updated by Swarmhost's agentic review process.

@comfy-greenlight-bot comfy-greenlight-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pull request at head commit 4d05936 has been reviewed and approved by Swarmhost's agentic review process.

The detailed evaluation is available to employees in the internal Slack review thread.

@synap5e
synap5e added this pull request to the merge queue Oct 2, 2026
Merged via the queue into main with commit 0a6272a Oct 2, 2026
31 of 38 checks passed
@synap5e
synap5e deleted the synap5e/feat/ops-flag-app-version branch October 2, 2026 18:08
artifyfun added a commit to artifyfun/Comfy-Desktop that referenced this pull request Oct 8, 2026
…药丸 + 遥测强化)

上游功能
- feat(auth): Desktop 账号会话共享给本地 ComfyUI 视图 (Comfy-Org#1657)
- feat(settings): 启动参数中 Core beta 参数以 "+N beta" 药丸展示 (Comfy-Org#1626)
- feat(templates): 支持可信输入素材下载 (Comfy-Org#1435)
- feat(benchmarks): 性能测试示例工作流 + 相对速度对比 (Comfy-Org#1607/Comfy-Org#1593)
- fix(launch): 启动前清理上一次 Desktop 遗留的孤儿 ComfyUI 进程 (Comfy-Org#1601)
- fix(comfybuilder)/fix(models): 组织规则 Build 的 manager 与共享模型路径口径 (Comfy-Org#1636/Comfy-Org#1646/Comfy-Org#1664)
- fix(ops-flag): beta 授权 flag 等待 5s/3s,保存的授权 7 天后过期 (Comfy-Org#1649)
- fix(announcement): 公告弹窗换成 Comfy API (Comfy-Org#1608)
- feat(telemetry): workflow_name、typed asset metrics、error_kind、app_version (Comfy-Org#1647/Comfy-Org#1596/Comfy-Org#1623/Comfy-Org#1602)
- 其余:bridge types 发布改 Node 26、CI actions 刷新、多处测试稳定性修复
- Comfy-Org#1612 共享模型根目录改动已被 Comfy-Org#1616 revert,净效果为零

冲突解决
- package.json:保留 fork 品牌(artify-desktop / 1.10.0 / Artify Desktop 描述),丢弃上游 comfyui-desktop-2 / 1.1.6
- src/main/popups/titlePopup.ts:合并两侧 import(上游 PICKER_SETTINGS_CHANNELS + fork McpConfigInfo)

验证
- pnpm run typecheck:node / :web / :e2e / :integration 全绿
- npx vitest run src/main/artifylab --root . → 824 passed / 6 skipped
artifyfun added a commit to artifyfun/Comfy-Desktop that referenced this pull request Oct 8, 2026
v1.10.0 以来的 31 个提交(fork 侧 2 + 上游同步 29 / 2 次 merge):

fork 侧:
- fix(version): dev 版本串改用 `git describe --first-parent`,避免同步上游后报成上游版本
- test: 修同步上游后 Windows 上的 3 例红灯(均为既有红,非本次合并引入)——两处上游测试写死 POSIX 绝对路径(`path.resolve` 在 Windows 会补盘符)、一处 fork 自身的 `--preview-method latent2rgb` 注入让上游期望值失准

上游同步(upstream/main → artifylab-v2,29 提交 / 2 次 merge,官方 1.1.3 → 1.1.6 及之后 Comfy-Org#1671):
- feat(auth): 本地 ComfyUI 视图共享 Desktop 账号会话;hosted view 可切换 Desktop 工作区(Comfy-Org#1657 / Comfy-Org#1671)
- fix(launch): 启动前先清掉上次 Desktop 遗留的 ComfyUI 孤儿进程(Comfy-Org#1601)
- feat(settings): 启动参数里以「+N beta」药丸展示 Core beta args、beta 开关改文案「Opt-in to beta features」(Comfy-Org#1626 / Comfy-Org#1614)
- fix(ops-flag): 保存 grant 时等 5 s、其余 3 s;已保存的 grant 7 天后过期(Comfy-Org#1649)
- feat(telemetry): ops-flag 评估带上 app_version、assets 指标按命名约定上报、资产扫描失败转发 Core 的 error_kind、性能测试事件加 workflow_name;修占位 firmware UUID 产生共享 installation id(Comfy-Org#1602 / Comfy-Org#1596 / Comfy-Org#1623 / Comfy-Org#1647 / Comfy-Org#1617)
- fix(comfybuilder): 受组织策略约束的 Build 不再带上其 ComfyUI 会拒绝的 manager 与 extra-model-paths 参数;模型目录与其余 Build 一致(Comfy-Org#1664 / Comfy-Org#1646 / Comfy-Org#1636)
- fix(models): 共享模型目录作为 ComfyUI models root(Comfy-Org#1612,随后 Comfy-Org#1616 回退)
- feat(templates): 支持可信输入资产下载、视频页 LTX-2.5 I2V 换 MiniMax H3 I2V、四个页签启动卡片刷新(Comfy-Org#1435 / Comfy-Org#1611 / Comfy-Org#1606)
- feat(announcement): 启动公告弹窗改用 Comfy API(Comfy-Org#1608)
- feat(benchmarks): 性能测试示例工作流与相对速度基准(Comfy-Org#1607 / Comfy-Org#1593)
- chore: 官方版本号 1.1.3 → 1.1.4 → 1.1.5 → 1.1.6(Comfy-Org#1613 / Comfy-Org#1633 / Comfy-Org#1641);ci 用 Node 26 发布 bridge types(Comfy-Org#1599);数据库锁提示文案修正(Comfy-Org#1622);waitForPort/waitForUrl 中断测试改为按原因而非墙钟(Comfy-Org#1619);相对解释器替身启动去掉 `-s`(Comfy-Org#1621)

依赖变化:无新增(`pnpm-lock.yaml` 未动;子包 `@comfyorg/comfyui-desktop-bridge-types` 随上游 0.4.0 → 0.5.0)

验证:`typecheck:node` / `typecheck:web` 通过;主进程 vitest 372 文件 7251 passed / 0 failed;前端 vitest 29 文件 460 passed。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cursor-review Trigger multi-model Cursor code review greenlight

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants