You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-up to #151. Most of that PR's feedback (Mode A trace synthetic ids, default_used vs. rationale as the fallback signal, classifier label
mismatches scoring 0.0 silently) was already addressed by #156, which is
now merged into main.
The one piece #156 didn't cover: examples.md's bundled policies name models
a given host may not have. scripts/validate.py can't check model existence
(needs a live server), so registering one of these examples unmodified fails
at the server with 400 Collection component not registered. This documents
that the examples are shape references, not runnable as-is, and that models
should be swapped for ones the target host actually has before registering.
Thank you for creating this Matt!
I addressed this in #156 in a different location[skill.md] for better visibility:
SKILL.md Step 1: "A name the user did give may still not exist on the target host - the offline validator can't check that (Step 8b closes the gap)."
SKILL.md Step 8b: "ready: true from the validator only means the JSON is schema-valid - it says nothing about whether these models exist on the target host. Run #1 for every candidate/classifier model before #2, or /pull will 400 on a policy that just passed validation."
Thank you for creating this Matt! I addressed this in #156 in a different location[skill.md] for better visibility:
SKILL.md Step 1: "A name the user did give may still not exist on the target host - the offline validator can't check that (Step 8b closes the gap)."
SKILL.md Step 8b: "ready: true from the validator only means the JSON is schema-valid - it says nothing about whether these models exist on the target host. Run #1 for every candidate/classifier model before #2, or /pull will 400 on a policy that just passed validation."
Thanks for confirming - just didn't want this to disappear into the ether without confirmation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #151. Most of that PR's feedback (Mode A trace synthetic ids,
default_usedvs.rationaleas the fallback signal, classifier labelmismatches scoring
0.0silently) was already addressed by #156, which isnow merged into
main.The one piece #156 didn't cover:
examples.md's bundled policies name modelsa given host may not have.
scripts/validate.pycan't check model existence(needs a live server), so registering one of these examples unmodified fails
at the server with
400 Collection component not registered. This documentsthat the examples are shape references, not runnable as-is, and that models
should be swapped for ones the target host actually has before registering.