Skip to content

Add optimizing-models-with-quark entry-point skill - #147

Open
amd-zhaofeng wants to merge 2 commits into
amd:mainfrom
amd-zhaofeng:optimizing-models-with-quark
Open

Add optimizing-models-with-quark entry-point skill#147
amd-zhaofeng wants to merge 2 commits into
amd:mainfrom
amd-zhaofeng:optimizing-models-with-quark

Conversation

@amd-zhaofeng

@amd-zhaofeng amd-zhaofeng commented Aug 17, 2026

Copy link
Copy Markdown

What

Adds optimizing-models-with-quark, an entry-point skill that connects an agent to AMD Quark's own quantization skills. Implements #139.

Why

amd-skill-finder already routes "quantize this model to FP8" to amd/Quark, but all it can return is a repository address — the agent still has to read the docs and guess the flags. Quark ships skills of its own for exactly this work, and catalog users cannot reach them.

The skill is deliberately thin: it quantizes nothing, installs no pip package, and registers nothing as a skill. The quantization knowledge stays upstream in Quark, versioned with the product, so there is no vendored copy here to go stale.

How it works

  1. Check whether Quark's skill tree is already in the cache directory (~/.cache/amd-quark-skills, or %LOCALAPPDATA%/amd-quark-skills on Windows).
  2. If not, explain what will happen and fetch it with the user's consent — a shallow, blobless, sparse clone, nothing written to their workspace.
  3. Verify the starting-point files are readable; stop and report if not.
  4. Route to one of three starting points and follow it, checkpoints included.
The user wants to Starting point
Quantize a PyTorch / HuggingFace / safetensors model quark-torch-llm-ptq-workflow
Quantize an .onnx model quark-onnx-ptq-workflow
Install or verify the Quark environment quark-install

Interim design for amd-quark v0.12

The proposal in #139 assumed pip install amd-quark would deliver the skills.
On amd-quark v0.12 it does not, so this PR takes a different route and says so plainly:

  • The pip package does not carry the skills. setup.py packages quark and quark.* only; the skills live in the repository, so they are fetched from GitHub instead.
  • A skill is not self-contained. .claude/skills/ holds thin stubs that defer to .claude/skills-impl/, whose files reference each other by relative path. The tree therefore travels as a unit — fetching individual files is not reliable.
  • Read as files, not installed as skills. Nothing is written into the user's workspace or any harness skills directory, and no restart is required. The trade-off is stated in the skill body: because nothing is registered, a later question will not route to a Quark skill on its own and every request goes through this entry point. That cost is temporary, and it buys a clean removal path — a single rm -rf of the cache directory.

Once Quark ships its skills inside the quark package, the retrieval step goes away and this entry point can become what #139 originally described.

Test plan

  • ./.github/scripts/check.sh passes — 8 skills, 0 errors, both generated manifests report up to date
  • Fetch commands smoke-tested end to end: git clone --sparse plus sparse-checkout set yields all four required files, ~2 MB on disk
  • Description is third person and 824 characters, with negative triggers for serving-llms-on-instinct / serving-llms-on-epyc and magpie-kernel-evaluator
  • SKILL.md body is 164 lines, well under the 500-line cap; reference is linked one level deep
  • Not included this round: evals/evals.py, a walkthrough, and routing eval prompts — happy to add in a follow-up

The catalog had no answer for "quantize this model": `amd-skill-finder`
routes such requests to amd/Quark, but could only hand back a repository
address. Quark ships its own quantization skills, yet they live in the
Quark repository and are not part of the `amd-quark` pip package, so an
agent cannot reach them by installing the package.

This skill is the bridge. With the user's consent it fetches Quark's
skill tree into a cache directory outside the workspace, then routes to
one of three starting points: Torch PTQ, ONNX PTQ, or Quark environment
setup. It quantizes nothing itself and registers nothing as a skill, so
the quantization knowledge stays upstream, versioned with the product,
and nothing is left in the user's repository to clean up.

The sparse checkout takes `.claude` plus the `llm_ptq` example, because
the Torch workflow runs `quantize_quark.py` from the latter and the pip
package does not ship it.
The clone and the pip install serve different purposes, and nothing said
so outright. An agent reading the skill could plausibly conclude the
cache directory is a Quark checkout to install from, which would leave
the user with a broken environment: the `quark/` package source is not
in the sparse checkout, so an install run from there cannot produce a
working Quark. `quark-install` also documents a from-source option, so
say plainly that PyPI is the endorsed path and source installs need the
user to ask for them.
@amd-zhaofeng
amd-zhaofeng force-pushed the optimizing-models-with-quark branch 2 times, most recently from 97d85b4 to 932cf40 Compare August 17, 2026 06:57
@amd-zhaofeng
amd-zhaofeng marked this pull request as ready for review August 17, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant