ggml-openvino: forward NPU compilation mode parameters - #292
Open
zhaixuejun1993 wants to merge 2 commits into
Open
ggml-openvino: forward NPU compilation mode parameters#292zhaixuejun1993 wants to merge 2 commits into
zhaixuejun1993 wants to merge 2 commits into
Conversation
Owner
|
@zhaixuejun1993 |
ravi9
force-pushed
the
dev_backend_openvino
branch
from
August 24, 2026 16:49
8ba9348 to
d691121
Compare
zhaixuejun1993
force-pushed
the
xuejun/enable_npu_compilation_mode_params
branch
2 times, most recently
from
August 25, 2026 02:11
09d8c7c to
27ec9b1
Compare
Add GGML_OPENVINO_NPU_COMPILE_PARAMS to the backend's cached environment so callers can configure the NPU compiler without using the generic property escape hatch. When the value is non-empty, pass it to OpenVINO as NPU_COMPILATION_MODE_PARAMS. This enables settings such as optimization-level=3 for NPU compilation while preserving the existing behavior when the variable is unset and leaving CPU and GPU configuration unchanged. Document the variable, its NPU-only scope, and the optimization-level=3 example in the OpenVINO backend runtime configuration table.
wine99
approved these changes
Aug 25, 2026
ravi9
force-pushed
the
dev_backend_openvino
branch
from
August 25, 2026 16:53
d691121 to
407e209
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds support for passing custom NPU compilation mode parameters to OpenVINO via a new environment variable. This enables advanced users to fine-tune NPU compilation behavior by specifying parameters such as optimization level.
New NPU compilation mode parameter support:
ggml/src/ggml-openvino/ggml-openvino-extra.cpp: Added support for theGGML_OPENVINO_NPU_COMPILATION_MODE_PARAMSenvironment variable, which forwards custom NPU compiler parameters to OpenVINO. [1] [2]docs/backend/OPENVINO.md: Documented the newGGML_OPENVINO_NPU_COMPILATION_MODE_PARAMSvariable, including its purpose and usage example.Add GGML_OPENVINO_NPU_COMPILATION_MODE_PARAMS to the backend's cached environment so callers can configure the NPU compiler without using the generic property escape hatch.When the value is non-empty, pass it to OpenVINO as NPU_COMPILATION_MODE_PARAMS. This enables settings such as optimization-level=3 for NPU compilation while preserving the existing behavior when the variable is unset and leaving CPU and GPU configuration unchanged.
Document the variable, its NPU-only scope, and the optimization-level=3 example in the OpenVINO backend runtime configuration table.
Overview
Additional information
Requirements