feat(llama.cpp): 将 example 看护迁到共用引擎

.github/workflows/llama.cpp-examples.yml

@@ -0,0 +1,62 @@
+# llama.cpp examples guard - project thin trigger.
+#
+# Calls the common engine .github/workflows/examples-template.yml;
+# this file only declares what varies per project: schedule,
+# concurrency, upstream repo, dispatch input forwarding, and matrix
+# parallelism. The guard loop lives in the engine. See
+# examples-guards/README.md.
+
+name: llama.cpp-examples
+
+concurrency:
+  # Three groups: schedule runs serialize; each PR serializes per number
+  # so a new push cancels the stale run; each manual run stands alone.
+  # Group names are repository-wide, so the PR group carries the project
+  # name. A bare pr-<number> would let two projects on one PR cancel
+  # each other.
+  group: ${{ github.event_name == 'schedule' && 'llama.cpp-examples-schedule' || (github.event_name == 'pull_request' && format('llama.cpp-examples-pr-{0}', github.event.pull_request.number)) || format('manual-{0}', github.run_id) }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+on:
+  # Daily at 03:00 Beijing time (UTC 19:00), one hour after
+  # accelerate-examples. The source repo's llama.cpp-examples cron should
+  # be turned off once this is on main, so the two guards do not both
+  # take NPUs.
+  schedule:
+    - cron: '0 19 * * *'
+  workflow_dispatch:
+    inputs:
+      target_ref:
+        description: Branch, tag, or SHA to test (empty = latest upstream release; falls back to master on API failure)
+        required: false
+        default: ''
+        type: string
+  # PR runs need no merge: GitHub executes the workflow files from the
+  # PR merge ref. paths lists only this project's files, so a PR that
+  # touches only the shared engine starts no matrix here.
+  pull_request:
+    branches: [main]
+    types: [opened, synchronize, reopened]
+    paths:
+      - 'examples-guards/llama.cpp/**'
+      - '.github/workflows/llama.cpp-examples.yml'
+
+permissions:
+  actions: read
+  contents: read
+
+jobs:
+  # Job ids cannot contain '.'. The project name stays llama.cpp; that
+  # string is what artifacts and examples-guards paths use.
+  llama-cpp-examples:
+    uses: ./.github/workflows/examples-template.yml
+    with:
+      project: llama.cpp
+      upstream_repo: ggml-org/llama.cpp
+      # Upstream's default branch is master. The engine's own default is
+      # main, which 404s when a release lookup falls back.
+      default_branch: master
+      target_ref: ${{ inputs.target_ref }}
+      # 9 supported entries, all single-card. 4 leaves room on the
+      # shared pool while the Qwen GGUF lock serializes the first download.
+      max_parallel: 4

examples-guards/llama.cpp/README.md

@@ -0,0 +1,48 @@
+# llama.cpp example 看护
+
+本目录是 ggml-org/llama.cpp 的 example 看护数据，给共用引擎 `.github/workflows/examples-template.yml` 用。薄触发器是 `.github/workflows/llama.cpp-examples.yml`。上游默认分支是 `master`。触发器把 `default_branch` 设成 `master`，发布查询失败时回落到这条分支。
+
+被测 ref 优先取上游最新 release。清单路径按 release `v0.6.0` 的 `examples/` 核对过，列出的目录都还在。
+
+## 清单里跑什么
+
+`supported` 只保留不改上游源码就能跑完、并且用了昇腾的条目能在日志里看到 CANN 设备行的目录。PR 运行 38228583901 上，`examples/simple` 已经这样跑通。
+
+不跑的条目：
+
+- `examples/idle`：Metal 空闲微基准
+- `examples/sycl`：要 Intel oneAPI
+- `examples/training`：全量 FP32 微调，CANN 没有对应反向算子
+- `examples/simple-chat`：上游把低于 ERROR 的日志丢掉，CANN 设备行不会出现。要看到这行就得改上游源码。只看退出码 0 会把没落到昇腾的运行当成成功
+- `examples/retrieval`：上游读循环在输入结束后不返回，喂一行会一直挂到超时。让它返回就得改 `retrieval.cpp`
+- `examples/batched`、`examples/embedding`、`examples/lookahead`、`examples/lookup`、`examples/speculative`、`examples/speculative-simple`：同一轮里 ggml-cann 在 `aclnnWeightQuantBatchMatmulV2` 报 y 不连续并中止
+- `examples/parallel`：同一轮里 `FusedInferAttentionScore` 要求非 PA 路径上的 key 连续，实际不连续并中止
+- `examples/diffusion`：运行 38249312071 里 Dream 权重已从 Hugging Face 下完并转成 Q8_0 GGUF，随后在同一个 key 不连续错误上中止
+
+每条不支持的依据写在 `examples_manifest.yaml` 里该路径的上方，带源码位置或这次 CI 作业链接，不把几条合成一段注释。
+
+`v0.6.0` 里另外有 `examples/deprecation-warning`、`examples/llama-eval`、`examples/test-cmake`。源仓清单没有它们，这次不调度。
+
+## profile
+
+`scripts/setup_example.sh` 用函数 `setup_<profile>` 解释清单里的 profile。未知 profile 在编译和下载之前退出。
+
+- `cann`：打开 `GGML_CANN`，只编 `exec` 指向的那一个 target。Qwen2.5 0.5B Instruct 的 Q4_0 GGUF 先从 huggingface.co 下，失败再从 ModelScope 下。量化选 Q4_0，因为上游 CANN 文档里 910B 的数据类型表写了 FP16、Q8_0、Q4_0、BF16。
+- `cpu`：cmake 不开 `GGML_CANN`。绿灯只表示这个目录在 aarch64 上能编完、能跑完。外部汇总不要把这些条目读成昇腾推理成功。
+- `cmake-pkg`：安装之后再用 `find_package` 编 `examples/simple-cmake-pkg`，确认安装包还能链上 CANN 后端。`LD_LIBRARY_PATH` 指到 `inst/lib`，因为消费方二进制的 RUNPATH 盖不住 `libggml` 再加载的 `libggml-cann`。
+
+镜像是 `swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12`。编译器用镜像自带的 cmake、g++、make，不改 apt 源。
+
+## 缓存
+
+下载落到 runner 的 `~/.cache`，香港池会把这棵目录留到下一次运行。workflow 不写容器 volume，也不依赖源仓的预投递。
+
+多条 `cann` 腿共用同一个 Qwen GGUF。脚本先下到本 job 的暂存文件，核对 GGUF 文件头之后再改名进缓存，并用 mkdir 锁避开并发追加写。
+
+## 设备锚点
+
+`cann`、`cmake-pkg` 的运行日志里必须出现 `CANN` 后接数字的那一行。ggml 在卡没挂上时会静默退回 CPU，进程仍然可以 0 退出。看护只在自己的运行脚本里检查这行，不改上游 example 文件。
+
+## 触发
+
+定时已经打开，每天一次，UTC 19:00，也就是北京时间 03:00，比 accelerate 的 example 看护晚一小时。源仓同项目的定时仍开着的话会和本仓抢 NPU，合入后由维护者关掉源仓那条。

examples-guards/llama.cpp/examples_manifest.yaml

@@ -0,0 +1,184 @@
+# llama.cpp example guard for the shared engine in examples-template.yml.
+# Profiles are implemented by scripts/setup_example.sh. Timeouts include a
+# cold-cache budget: this runner pool does not have the source repo's
+# pre-seeded weights. Entries that crash in ggml-cann, or that cannot
+# prove the CANN device without editing upstream sources, are unsupported.
+version: 1
+scan:
+  root: examples
+  unit: directories
+  marker: CMakeLists.txt
+  max_depth: 1
+supported:
+  - path: examples/simple
+    profile: cann
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: build/bin/llama-simple
+    overlay_args:
+      - -m ${LLAMA_CI_MODEL}
+      - -n 16
+      - -ngl 99
+    timeout_minutes: 90
+  - path: examples/passkey
+    profile: cann
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: build/bin/llama-passkey
+    overlay_args:
+      - -m ${LLAMA_CI_MODEL}
+      - --junk 16
+      - -ngl 99
+      - -v
+    timeout_minutes: 90
+  - path: examples/debug
+    profile: cann
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: build/bin/llama-debug
+    overlay_args:
+      - -m ${LLAMA_CI_MODEL}
+      - -p hello
+      - -ngl 99
+      - -c 256
+      - --save-logits
+      - --logits-output-dir ${CI_OUTPUT_DIR}
+      - --no-warmup
+      - -v
+    timeout_minutes: 90
+  - path: examples/eval-callback
+    profile: cann
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: build/bin/llama-eval-callback
+    overlay_args:
+      - -m ${LLAMA_CI_MODEL}
+      - -p hello
+      - -ngl 99
+    timeout_minutes: 90
+  - path: examples/convert-llama2c-to-ggml
+    profile: cpu
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: build/bin/llama-convert-llama2c-to-ggml
+    overlay_args:
+      - --llama2c-model ${FIXTURE_DIR}/stories260K.bin
+      - --copy-vocab-from-model ${FIXTURE_DIR}/tok512.bin
+      - --llama2c-output-model ${CI_OUTPUT_DIR}/stories260K.gguf
+    timeout_minutes: 90
+  - path: examples/gguf
+    profile: cpu
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: build/bin/llama-gguf
+    timeout_minutes: 90
+  - path: examples/gguf-hash
+    profile: cpu
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: build/bin/llama-gguf-hash
+    overlay_args:
+      - ${FIXTURE_DIR}/ci-demo.gguf
+    timeout_minutes: 90
+  - path: examples/gen-docs
+    profile: cpu
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: build/bin/llama-gen-docs
+    timeout_minutes: 90
+  - path: examples/simple-cmake-pkg
+    profile: cmake-pkg
+    runner: linux-aarch64-a2-1
+    npu_devices: '0'
+    image: swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-910b-ubuntu22.04-py3.12
+    exec: examples/simple-cmake-pkg/build/llama-simple-cmake-pkg
+    overlay_args:
+      - -m ${LLAMA_CI_MODEL}
+      - -n 16
+      - -ngl 99
+    timeout_minutes: 120
+unsupported:
+  # examples/idle/README.md links PR 17766, titled
+  # "metal : add residency sets keep-alive heartbeat". That PR's log is
+  # "ggml_metal_init: found device: Apple M2 Ultra". idle.cpp:78 sleeps
+  # to "simulate idle GPU", then times llama_process. The example measures
+  # Metal residency after idle, not a CANN workload.
+  - examples/idle
+  # ls-sycl-device.cpp:8 includes ggml-sycl.h and line 13 calls
+  # ggml_backend_sycl_print_sycl_devices(). CMakeLists.txt builds
+  # llama-ls-sycl-device. README.md says to build with GGML_SYCL_TARGET
+  # and run `source /opt/intel/oneapi/setvars.sh`. Intel oneAPI, not CANN.
+  - examples/sycl
+  # finetune.cpp:85 calls llama_opt_epoch. Lines 33-39 force an F32 KV
+  # cache because OUT_PROD has no f16 support. README.md:6-7 documents
+  # only a CPU build with no extra backend, or CUDA with -ngl 999.
+  # README.md:9 says FLASH_ATTN_EXT has no backward pass. v0.6.0
+  # ggml/src/ggml-cann/ggml-cann.cpp has no "backward" string, so the
+  # CANN backend does not implement the training backward this binary needs.
+  - examples/training
+  # simple-chat.cpp:72-77 installs llama_log_set and prints only when
+  # level >= GGML_LOG_LEVEL_ERROR. The CANN device line is below ERROR,
+  # so the guard cannot see it unless that filter is edited. Exit 0
+  # would not show that the NPU was used.
+  - examples/simple-chat
+  # retrieval.cpp:265-268 is `while (true)` then getline and
+  # common_tokenize, with no break on an empty line or EOF. One prompt
+  # does not end the process. Exiting would mean editing this loop.
+  - examples/retrieval
+  # llama-batched on v0.6.0, CANN 9.1.0, Ascend910B3 aborted in
+  # ggml-cann.cpp:70 with Invalid_Argument(EZ0026): Parameter y of
+  # aclnnWeightQuantBatchMatmulV2GetWorkspaceSize is not contiguous.
+  # Exit 134. Fixing it means editing ggml-cann, not this example.
+  # https://github.com/Ascend/docs/actions/runs/38228583901/job/114742295302
+  - examples/batched
+  # llama-embedding on run 38228583901 aborted in ggml-cann.cpp:70 with
+  # Invalid_Argument(EZ0026): Parameter y of
+  # aclnnWeightQuantBatchMatmulV2GetWorkspaceSize is not contiguous.
+  # Exit 134.
+  # https://github.com/Ascend/docs/actions/runs/38228583901/job/114742295286
+  - examples/embedding
+  # llama-lookahead on run 38228583901 aborted in ggml-cann.cpp:70 with
+  # Invalid_Argument(EZ0026): Parameter y of
+  # aclnnWeightQuantBatchMatmulV2GetWorkspaceSize is not contiguous.
+  # Exit 134.
+  # https://github.com/Ascend/docs/actions/runs/38228583901/job/114742295384
+  - examples/lookahead
+  # llama-lookup on run 38228583901 aborted in ggml-cann.cpp:70 with
+  # Invalid_Argument(EZ0026): Parameter y of
+  # aclnnWeightQuantBatchMatmulV2GetWorkspaceSize is not contiguous.
+  # Exit 134.
+  # https://github.com/Ascend/docs/actions/runs/38228583901/job/114742295213
+  - examples/lookup
+  # llama-speculative on run 38228583901 aborted in ggml-cann.cpp:70 with
+  # Invalid_Argument(EZ0026): Parameter y of
+  # aclnnWeightQuantBatchMatmulV2GetWorkspaceSize is not contiguous.
+  # Exit 134.
+  # https://github.com/Ascend/docs/actions/runs/38228583901/job/114742295204
+  - examples/speculative
+  # llama-speculative-simple on run 38228583901 aborted in ggml-cann.cpp:70
+  # with Invalid_Argument(EZ0026): Parameter y of
+  # aclnnWeightQuantBatchMatmulV2GetWorkspaceSize is not contiguous.
+  # Exit 134.
+  # https://github.com/Ascend/docs/actions/runs/38228583901/job/114742295188
+  - examples/speculative-simple
+  # llama-parallel on run 38228583901 aborted in ggml-cann.cpp:70 with
+  # Invalid_Argument(EZ0037): Parameter key of FusedInferAttentionScore
+  # is invalid because in non-PA scenarios the key tensor must be
+  # contiguous. Exit 134.
+  # https://github.com/Ascend/docs/actions/runs/38228583901/job/114742295315
+  - examples/parallel
+  # llama-diffusion-cli on run 38249312071 downloaded Dream-v0-Instruct-7B
+  # and wrote Dream-v0-Instruct-7B.Q8_0.gguf, then aborted in
+  # ggml-cann.cpp:70 with Invalid_Argument(EZ0037): Parameter key of
+  # FusedInferAttentionScore is invalid because in non-PA scenarios the
+  # key tensor must be contiguous. Exit 134.
+  # https://github.com/Ascend/docs/actions/runs/38249312071/job/114811506183
+  - examples/diffusion

examples-guards/llama.cpp/scripts/expand_overlay.py

@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+"""Print OVERLAY_ARGS as a shell-quoted token list.
+
+The workflow passes manifest.overlay_args as a JSON array. Each item is
+expanded against this job's environment, so ${LLAMA_CI_MODEL} and
+${CI_OUTPUT_DIR} resolve here and nowhere else.
+"""
+from __future__ import annotations
+
+import json
+import os
+import shlex
+import sys
+
+
+def main() -> int:
+    raw = os.environ.get('OVERLAY_ARGS', '').strip()
+    if not raw or raw in ('null', '""'):
+        return 0
+    try:
+        items = json.loads(raw)
+    except json.JSONDecodeError as exc:
+        print(f'OVERLAY_ARGS is not valid JSON: {exc}', file=sys.stderr)
+        return 1
+    if items in (None, ''):
+        return 0
+    if not isinstance(items, list):
+        print(f'OVERLAY_ARGS must be a JSON array, got {type(items).__name__}',
+              file=sys.stderr)
+        return 1
+    tokens: list[str] = []
+    for item in items:
+        if not isinstance(item, str) or not item.strip():
+            print('OVERLAY_ARGS items must be non-empty strings', file=sys.stderr)
+            return 1
+        tokens.extend(shlex.split(os.path.expandvars(item), posix=True))
+    print(' '.join(shlex.quote(token) for token in tokens))
+    return 0
+
+
+if __name__ == '__main__':
+    raise SystemExit(main())