feat: add candidate quality gates for solution promotion - #477
Open
lucabello wants to merge 2 commits into
Open
Conversation
Add automated testing and promotion workflow for the COS and COS Lite solutions. On a schedule, each supported branch deploys its solutions via Terraform, runs solution tests, and promotes every pinned charm from beta to candidate once all solutions pass. Solutions are discovered dynamically from the tests/solution directory so new solutions or branches require no workflow changes. The quality-gates.just module can also be run locally to test or promote a solution without needing CI.
mmkay
approved these changes
Aug 25, 2026
mmkay
left a comment
mmkay
left a comment
Contributor
There was a problem hiding this comment.
A couple smaller comments, but as this will be WIP in the next PRs, they aren't blocking.
| # Init/apply a solution's Terraform module, then run its solution test suite | ||
| [group("test")] | ||
| [working-directory("./tests/solution")] | ||
| test-solution solution="cos-lite": |
Contributor
There was a problem hiding this comment.
In the promote tests, we likely want to be passing the risk to the module in order to really test the beta risk here. I think we don't do that yet?
| charm_keys=$(awk ' | ||
| /data[[:space:]]*"juju_charm"[[:space:]]*"/ { in_block=1; next } | ||
| in_block && /^\}/ { in_block=0; charm=""; key=""; next } | ||
| in_block && match($0, /charm[[:space:]]*=[[:space:]]*"([^"]*)"/, m) { charm=m[1] } |
Contributor
There was a problem hiding this comment.
Worth checking if the matchers will work with github runners. We might be using mawk there, this might not support the three-param match.
| charms_tracks=$(just charm-tracks "{{solution}}" "{{ref}}") | ||
| if [ -z "$charms_tracks" ]; then | ||
| echo "No pinned charms found for solution '{{solution}}'; nothing to promote." | ||
| exit 0 |
Contributor
There was a problem hiding this comment.
I wonder if this shouldn't exit with a non-zero exit code. What are the odds we'll trigger promotion but won't actually want to promote anything?
Co-authored-by: Mateusz Kulewicz <mateusz.kulewicz@canonical.com> Signed-off-by: Luca Bello <36242061+lucabello@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a scheduled quality gate that deploys the COS and COS Lite solutions from each supported branch, runs their solution tests, and promotes every pinned charm from beta to candidate once all solutions pass on that branch.
Closes canonical/observability#489
Changes
quality-gates.just:justmodule (imported into the mainjustfileasquality-gates) with recipes to test a solution, promote a solution's charms, and list a product module's charm/track pinstests/solution/{cos,cos-lite}: minimal Terraform wrapper modules and solution test entrypoints.github/workflows/quality-gates-candidate.yaml: branch matrix trigger, runs on a monthly schedule or manual dispatch.github/workflows/_quality-gate-candidate-branch.yaml: reusable workflow that discovers solutions dynamically per branch, tests each one, then promotes once all passNotes
Solutions are discovered from
tests/solution/*/terraform, so adding a new solution or branch requires no workflow changes.