Repository navigation
[feature] 迁移 cache-dit 的 examples 守护管线到 docs 仓 - #207
Merged
Merged
Conversation
yzeyu71
changed the title
| on: | ||
| # Temporarily disabled to save NPU runner time. Re-enable when ready: | ||
| # schedule: | ||
| # - cron: '30 */6 * * *' |
参考 PR Ascend#200（accelerate）迁移范式，将 workflows 仓 projects/cache-dit 的 examples 守护管线平移到 docs 仓 examples-guards/cache-dit： - examples_manifest.yaml：1 supported + 7 unsupported 逐字段平移 （保留 scan.unit: files 与 supported/unsupported 的有意重叠）， overlay_args 补充参数注释 - scripts/setup_example.sh：模型下载 HF 优先（HF_ENDPOINT 缺省 hf-mirror.com）；FLUX.1-dev 为 HF gated repo 且本仓无 HF_TOKEN， 按约定保留 ModelScope 镜像兜底（AI-ModelScope/FLUX.1-dev，注释 明确边界与持久修复路径），非 gated 模型一律走 HF - scripts/run_example.sh：原样平移（含 EXEC/torchrun 支持） - .github/workflows/cache-dit-examples.yml：薄触发器对接 examples-template.yml 共享引擎，schedule 保持注释（bring-up 策略）
Signed-off-by: yangzeyu <yangzeyu7@huawei.com>
Add a pull_request trigger to cache-dit-examples.yml so changes to the guard assets (examples-guards/cache-dit/**) or this workflow file re-run the guard against the latest upstream release, catching regressions before merge. Mirrors the peft-quick-start example. The PR branch is handled by the shared examples-template engine (upstream/main already supports pull_request in the Decide target step). Co-Authored-By: CodeBuddy <https://codebuddy.ai>
yzeyu71
force-pushed
the
feat/cache-dit-examples-guard
branch
from
October 14, 2026 06:11
4e8332b to
b5356e3
Compare
Run 38520772429 hit the 120-minute job timeout in "Setup example environment" (119.7 min, "Run example" never started). The setup was downloading FLUX.1-dev from ModelScope into a per-job workspace dir: the HuggingFace leg 401s (gated repo, no HF_TOKEN in this repo) and the ModelScope fallback was never published into the shared runner cache, so every run re-downloaded all 57.89 GB, 23.8 GB of it the single-file flux1-dev.safetensors the pipeline never reads. - scripts/hub_cache.py: resolve FLUX_MODEL_PATH the way peft does. A snapshot in HF_HOME/hub that holds every required file is used as is, with zero network; a miss is downloaded (HuggingFace first, ModelScope fallback) into this job's staging dir on the same volume, checked, and renamed file by file into snapshots/<sha>, with refs/main written last so a half-published snapshot is never a hit. - allow_patterns trims the fetch to the diffusers component tree FluxPipeline.from_pretrained reads: 57.89 GB -> 33.75 GB, dropping the duplicate single-file weights (flux1-dev.safetensors 23.8 GB, ae.safetensors), dev_grid.jpg and the repo metadata. - setup_example.sh: drop the HF_ENDPOINT=hf-mirror override (the repo convention is huggingface.co / the engine default), replace the inline resolve_model helper with the shared hub_cache.py, and install the ModelScope fallback up front. - timeout_minutes 120 -> 180 so a cold-cache fill fits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
参考 #200（accelerate examples guard）迁移范式，将 workflows 仓
projects/cache-dit的 examples 守护管线迁移到 docs 仓，新增 4 个文件、共 379 行。迁移清单
examples-guards/cache-dit/examples_manifest.yamlscan.unit: files与 supported/unsupported 的有意重叠（generate.py 单卡可用、torchrun 分布式变体不可用）examples-guards/cache-dit/scripts/setup_example.shexamples-guards/cache-dit/scripts/run_example.sh.github/workflows/cache-dit-examples.ymlexamples-template.yml共享引擎模型下载策略（HF 优先 + gated 例外）
resolve_model与其他项目一致——共享 HF hub 缓存命中 → 冷池经 HF_ENDPOINT（缺省 hf-mirror.com）snapshot_download自填。manifest 注释补全
overlay_args8 个参数逐一注释：位置参数flux（示例选择）、--model-path、--attn _native_npu（NPU attention 后端）、--cache（DBCache 被测能力）、--cpu-offload（bf16 全量权重超 910B4 单卡显存）、--warmup/--repeat/--steps（smoke 最小开销取值）。关键参数（与 workflows 仓一致）
project: cache-dit、upstream_repo: vipshop/cache-dit、max_parallel: 4cann:9.1.0-910b-ubuntu22.04-py3.12、runnerlinux-aarch64-a2-1、timeout 120minTest plan