Skip to content

fix(benchmarks): drop the Ideogram 4 int8 example removed upstream - #1737

Closed
benceruleanlu wants to merge 1 commit into
mainfrom
bl/drop-ideogram-benchmark
Closed

benceruleanlu wants to merge 1 commit into
mainfrom
bl/drop-ideogram-benchmark

Conversation

@benceruleanlu

Copy link
Copy Markdown
Member

Removes the image_ideogram4_t2i_int8 example from assets/benchmark-templates.json.

Why

Comfy-Org/workflow_templates#1338 (merged 2026-10-15 17:53 UTC) deleted benchmarks/image_ideogram4_t2i_int8.json from its main. Desktop still offered that example on the performance test page, so picking it fails for users. The Benchmark templates / validate check catches this, but it only runs when its watched paths change, so main still lists the example. Every PR that touches the workflow file now fails, e.g. #1699 (run 38797982317):

✗ users could not download these examples:
  image_ideogram4_t2i_int8: benchmarks/image_ideogram4_t2i_int8.json is not on GitHub main

Behavior

The performance test page offers 11 examples instead of 12; image drops from 4 examples to 3. No code changes: performanceTestExampleWorkflows.ts and its tests read the bundled list rather than fixed ids.

The file is normally produced by scripts/benchmark-templates.mjs. I edited it by hand instead of regenerating it, because a regeneration from upstream main would also add the new gated image_flux3_klein_distilled_t2i example, which needs the download support from the closed #1736.

Validation

  • node scripts/benchmark-templates.mjs validate: ✓ 11 examples, all downloadable from GitHub main
  • Pre-commit hook: typecheck (node, web, e2e, integration), ESLint and Prettier pass
  • Unit tests were not run locally; CI's Test job covers performanceTestExampleWorkflows.test.ts

Change breakdown

Total changed lines = additions + deletions (10). No merge-only changes.

Category Files Added Deleted Share
Product code 0 0 0 0%
Test code 0 0 0 0%
Generated files (hand-edited) 1 0 10 100%

Generated files (1)

  • assets/benchmark-templates.json (+0 / -10)

Comfy-Org/workflow_templates#1338 removed benchmarks/image_ideogram4_t2i_int8.json
from main, so the performance test page offers an example users can no
longer download and the Benchmark templates check fails for every PR that
touches its watched paths.

Edited by hand rather than regenerated: regenerating from upstream main
would also pull in the new gated Flux.3 Klein example, which needs the
download support from the closed #1736.
@benceruleanlu
benceruleanlu requested review from a team as code owners October 15, 2026 21:23
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T21:24:24.761947Z 23668bb PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: fef98267-355e-4e21-a627-8bb3fdaf061f
📥 Commits

Reviewing files that changed from the base of the PR and between 0758183 and 23668bb.

📒 Files selected for processing: 1
  • assets/benchmark-templates.json
💤 Files with no reviewable changes: 1
  • assets/benchmark-templates.json

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

📝 Walkthrough

The benchmark catalog removes the Ideogram v4 Int8 text-to-image template, including its image modality, snapshot metadata, description, size, and media subtype.

Changes: Benchmark catalog

Layer / File(s) Summary
Remove Ideogram template
assets/benchmark-templates.json
The catalog entry and its associated metadata are removed.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 23668

Removing the unavailable template leaves no identified broken catalog reference. The change is mergeable after normal checks.

Pre-merge checks: Passed 2
Check name Status Explanation
Linked Issues check Passed No active directly linked issue remains. Issue #1736 is closed and supplies historical context only. Therefore, no linked-issue coding requirements apply.
Out of Scope Changes check Passed The pull request changes only assets/benchmark-templates.json. It removes the unavailable image_ideogram4_t2i_int8 entry, which matches the stated intent. The change is not unrelated; the scope is…
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants