HYPERFLEET-1108 - feat: Add e2e.hyperfleet.io/run-id label for e2e-gcp environment, label resources created by adapters#57
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
📝 WalkthroughSummary by CodeRabbit
WalkthroughCWE-703 (Improper Check for Unusual Conditions): base-adapter.yaml.gotmpl weakens RUN_ID injection gating from truthy Changes propagate Estimated code review effort: 2 (Simple) | ~12 minutes Sequence Diagram(s)Skipped — this is a templating/labeling change (Helm/Helmfile YAML), not a multi-component runtime control-flow change; no sequence diagram warranted per criteria. Related Issues: None referenced in diff. Related PRs: None referenced in diff. Suggested labels: helmfile, e2e, supply-chain-review Suggested reviewers: Platform/Adapter and Sentinel template owners (CWE-703 fix on RUN_ID gating requires sign-off before merge). Poem: 🚥 Pre-merge checks | ✅ 11✅ Passed checks (11 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@helmfile/configs/e2e/adapters/cl-deployment/adapter-task-config.yaml`:
- Around line 9-12: The e2eRunId input is only checked for non-empty, but it is
later used as a Kubernetes label value, so invalid characters can still reach
templating. Update the boundary handling for the e2eRunId mapping sourced from
env.E2E_RUN_ID to enforce a strict label-safe regex before it is rendered or
stamped into manifests. Keep the validation close to the adapter-task-config
input definition so invalid values are rejected early and cannot flow into label
generation.
In
`@helmfile/configs/e2e/adapters/cl-maestro/adapter-task-resource-manifestwork.yaml`:
- Around line 31-50: The YAML template in the manifestwork adapter has an
indentation regression that breaks the parent mappings and produces invalid
YAML. Re-indent the `hyperfleet.io/platform-type` conditional, the `annotations`
block, the ConfigMap `data` conditionals, and the ConfigMap `metadata` block so
they remain nested under the correct `metadata`/resource sections in this
template. Use the surrounding manifest structure in
`adapter-task-resource-manifestwork.yaml` to keep the `annotations`, `data`, and
`metadata` keys aligned with their parent objects.
In `@helmfile/helmfile.yaml.gotmpl`:
- Around line 30-32: The run-id label value is being emitted unquoted in the
helmfile template, which can let a numeric E2E_RUN_ID be serialized as a
non-string scalar and break Kubernetes label rendering. Update the
helmfile.yaml.gotmpl entry for labels.e2e.hyperfleet.io/run-id so the value is
always quoted as a string, while keeping e2eRunId consistent with the downstream
string handling.
In `@Makefile`:
- Around line 320-327: The E2E_RUN_ID check in the Makefile only blocks empty
values, but it should also reject strings that are not valid Kubernetes label
values before they reach metadata.labels. Update the install-hyperfleet
validation block to verify E2E_RUN_ID against Kubernetes label-value rules
(allowed characters, length, and format) in addition to the existing non-empty
check, and fail fast with a clear error message when invalid. Keep the logic in
the same HELMFILE_ENV=e2e-gcp guard so the validation happens at the system
boundary before any Helm/Kubernetes apply step.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Central YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: dec1e2d2-0e70-4017-9915-192667816ec1
📒 Files selected for processing (12)
Makefilehelmfile/configs/e2e/adapters/cl-deployment/adapter-task-config.yamlhelmfile/configs/e2e/adapters/cl-deployment/adapter-task-resource-deployment.yamlhelmfile/configs/e2e/adapters/cl-job/adapter-task-config.yamlhelmfile/configs/e2e/adapters/cl-job/adapter-task-resource-job.yamlhelmfile/configs/e2e/adapters/cl-maestro/adapter-task-config.yamlhelmfile/configs/e2e/adapters/cl-maestro/adapter-task-resource-manifestwork.yamlhelmfile/configs/e2e/adapters/cl-namespace/adapter-task-config.yamlhelmfile/helmfile.yaml.gotmplhelmfile/values/base-adapter.yaml.gotmplhelmfile/values/base-api.yaml.gotmplhelmfile/values/base-sentinel.yaml.gotmpl
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
openshift-hyperfleet/architecture(manual)openshift-hyperfleet/hyperfleet-api(manual)openshift-hyperfleet/hyperfleet-sentinel(manual)openshift-hyperfleet/hyperfleet-adapter(manual)openshift-hyperfleet/hyperfleet-broker(manual)
| - name: "e2eRunId" | ||
| source: "env.E2E_RUN_ID" | ||
| type: "string" | ||
| required: false |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Enforce label-safe validation for E2E_RUN_ID before templating.
e2eRunId is sourced from env and later stamped into Kubernetes labels. Current gating only checks non-empty, so invalid characters can break manifests or inject malformed YAML/labels (CWE-20, CWE-74). Add strict label-value regex validation at the boundary.
As per path instructions, “Validate input at system boundaries (HTTP handlers, CLI parsers, webhook receivers).”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@helmfile/configs/e2e/adapters/cl-deployment/adapter-task-config.yaml` around
lines 9 - 12, The e2eRunId input is only checked for non-empty, but it is later
used as a Kubernetes label value, so invalid characters can still reach
templating. Update the boundary handling for the e2eRunId mapping sourced from
env.E2E_RUN_ID to enforce a strict label-safe regex before it is rendered or
stamped into manifests. Keep the validation close to the adapter-task-config
input definition so invalid values are rejected early and cannot flow into label
generation.
Source: Path instructions
| e2eRunId: {{ env "E2E_RUN_ID" }} | ||
| labels: | ||
| e2e.hyperfleet.io/run-id: {{ env "E2E_RUN_ID" }} |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== helmfile references =="
rg -n 'E2E_RUN_ID|e2e\.hyperfleet\.io/run-id|e2eRunId|requiredEnv "E2E_RUN_ID"|env "E2E_RUN_ID"' helmfile . || true
echo
echo "== nearby context in helmfile/helmfile.yaml.gotmpl =="
nl -ba helmfile/helmfile.yaml.gotmpl | sed -n '1,80p'Repository: openshift-hyperfleet/hyperfleet-infra
Length of output: 5082
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== adapter references =="
rg -n 'testRunId|run-id|E2E_RUN_ID|TEST_RUN_ID|e2e\.hyperfleet\.io/(run-id|test-run-id)' . || trueRepository: openshift-hyperfleet/hyperfleet-adapter
Length of output: 437
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== architecture docs references =="
rg -n 'test-run-id|run-id|E2E_RUN_ID|TEST_RUN_ID|e2e\.hyperfleet\.io/(run-id|test-run-id)' hyperfleet/docs/e2e-testing || trueRepository: openshift-hyperfleet/architecture
Length of output: 2602
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '20,38p' helmfile/helmfile.yaml.gotmpl
echo
sed -n '1,90p' helmfile/values/base-adapter.yaml.gotmpl
echo
sed -n '312,328p' MakefileRepository: openshift-hyperfleet/hyperfleet-infra
Length of output: 2577
🏁 Script executed (no clone):
python3 - <<'PY'
import yaml, json, textwrap
doc = yaml.safe_load(textwrap.dedent("""
labels:
e2e.hyperfleet.io/run-id: 123
"""))
print(type(doc["labels"]["e2e.hyperfleet.io/run-id"]).__name__, doc["labels"]["e2e.hyperfleet.io/run-id"])
PY
Length of output: 133
Quote the run-id label value (CWE-20).
e2eRunId is re-quoted downstream, but labels.e2e.hyperfleet.io/run-id is later serialized via toYaml and can become a non-string YAML scalar if E2E_RUN_ID is numeric, which breaks Kubernetes label rendering.
Suggested fix
- e2e.hyperfleet.io/run-id: {{ env "E2E_RUN_ID" }}
+ e2e.hyperfleet.io/run-id: {{ env "E2E_RUN_ID" | quote }}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| e2eRunId: {{ env "E2E_RUN_ID" }} | |
| labels: | |
| e2e.hyperfleet.io/run-id: {{ env "E2E_RUN_ID" }} | |
| e2eRunId: {{ env "E2E_RUN_ID" }} | |
| labels: | |
| e2e.hyperfleet.io/run-id: {{ env "E2E_RUN_ID" | quote }} |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@helmfile/helmfile.yaml.gotmpl` around lines 30 - 32, The run-id label value
is being emitted unquoted in the helmfile template, which can let a numeric
E2E_RUN_ID be serialized as a non-string scalar and break Kubernetes label
rendering. Update the helmfile.yaml.gotmpl entry for
labels.e2e.hyperfleet.io/run-id so the value is always quoted as a string, while
keeping e2eRunId consistent with the downstream string handling.
Source: Path instructions
| @if [ "$(HELMFILE_ENV)" = "e2e-gcp" ]; then \ | ||
| if [ -z "$(E2E_RUN_ID)" ]; then \ | ||
| echo "ERROR: E2E_RUN_ID must be set when HELMFILE_ENV=e2e-gcp"; \ | ||
| echo " Usage: E2E_RUN_ID=<run-id> make install-hyperfleet"; \ | ||
| exit 1; \ | ||
| fi; \ | ||
| echo "OK: E2E_RUN_ID=$(E2E_RUN_ID)"; \ | ||
| fi; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Enforce Kubernetes label-value validation for E2E_RUN_ID (CWE-20).
Current check only rejects empty values. Invalid label values (illegal chars/length) still pass here and fail later when applied as metadata.labels.
Suggested fix
check-e2e-run-id: ## Verify E2E_RUN_ID is set for e2e-gcp environment
`@if` [ "$(HELMFILE_ENV)" = "e2e-gcp" ]; then \
if [ -z "$(E2E_RUN_ID)" ]; then \
echo "ERROR: E2E_RUN_ID must be set when HELMFILE_ENV=e2e-gcp"; \
echo " Usage: E2E_RUN_ID=<run-id> make install-hyperfleet"; \
exit 1; \
fi; \
+ if ! printf '%s' "$(E2E_RUN_ID)" | grep -Eq '^[A-Za-z0-9]([A-Za-z0-9._-]{0,61}[A-Za-z0-9])?$$'; then \
+ echo "ERROR: E2E_RUN_ID must be a valid Kubernetes label value (1-63 chars, alnum boundaries, [A-Za-z0-9._-] allowed)"; \
+ exit 1; \
+ fi; \
echo "OK: E2E_RUN_ID=$(E2E_RUN_ID)"; \
fi;As per path instructions, "Validate input at system boundaries (HTTP handlers, CLI parsers, webhook receivers)."
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| @if [ "$(HELMFILE_ENV)" = "e2e-gcp" ]; then \ | |
| if [ -z "$(E2E_RUN_ID)" ]; then \ | |
| echo "ERROR: E2E_RUN_ID must be set when HELMFILE_ENV=e2e-gcp"; \ | |
| echo " Usage: E2E_RUN_ID=<run-id> make install-hyperfleet"; \ | |
| exit 1; \ | |
| fi; \ | |
| echo "OK: E2E_RUN_ID=$(E2E_RUN_ID)"; \ | |
| fi; | |
| `@if` [ "$(HELMFILE_ENV)" = "e2e-gcp" ]; then \ | |
| if [ -z "$(E2E_RUN_ID)" ]; then \ | |
| echo "ERROR: E2E_RUN_ID must be set when HELMFILE_ENV=e2e-gcp"; \ | |
| echo " Usage: E2E_RUN_ID=<run-id> make install-hyperfleet"; \ | |
| exit 1; \ | |
| fi; \ | |
| if ! printf '%s' "$(E2E_RUN_ID)" | grep -Eq '^[A-Za-z0-9]([A-Za-z0-9._-]{0,61}[A-Za-z0-9])?$$'; then \ | |
| echo "ERROR: E2E_RUN_ID must be a valid Kubernetes label value (1-63 chars, alnum boundaries, [A-Za-z0-9._-] allowed)"; \ | |
| exit 1; \ | |
| fi; \ | |
| echo "OK: E2E_RUN_ID=$(E2E_RUN_ID)"; \ | |
| fi; |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Makefile` around lines 320 - 327, The E2E_RUN_ID check in the Makefile only
blocks empty values, but it should also reject strings that are not valid
Kubernetes label values before they reach metadata.labels. Update the
install-hyperfleet validation block to verify E2E_RUN_ID against Kubernetes
label-value rules (allowed characters, length, and format) in addition to the
existing non-empty check, and fail fast with a clear error message when invalid.
Keep the logic in the same HELMFILE_ENV=e2e-gcp guard so the validation happens
at the system boundary before any Helm/Kubernetes apply step.
Source: Path instructions
|
Similar change submitted at: #62 Biggest difference is not all deployed resources are tagged - check PR for more information. |
mliptak0
force-pushed
the
HYPERFLEET-1108
branch
2 times, most recently
from
8b48386 to
fa08e66
Compare
mliptak0
force-pushed
the
HYPERFLEET-1108
branch
4 times, most recently
from
21e0771 to
8cab4fe
Compare
…vironments, label all deployed resources - Use RUN_ID environment variable (following Mallory's approach from PR openshift-hyperfleet#62) - Label both HyperFleet components (API, Sentinel, Adapters) and adapter-created resources - Applied to both e2e-gcp and e2e-kind environments - RUN_ID defaults to namespace name in e2e environments (from env.gcp/env.kind) - No breaking changes - RUN_ID has sensible defaults Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
helmfile/helmfile.yaml.gotmpl (1)
30-41: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winLabel value still unquoted (CWE-20) — prior fix incomplete.
runIdgot| quotebutlabels.e2e.hyperfleet.io/run-id(Lines 32, 41) did not. IfRUN_IDis numeric, this scalar can be parsed as a non-string type and break Kubernetes label serialization downstream (label values must be strings). Same defect previously flagged, only half-fixed.Fix
- projectId: {{ env "PROJECT_ID" | default "hcm-hyperfleet" }} brokerType: googlepubsub serviceType: {{ env "API_SERVICE_TYPE" | default "LoadBalancer" }} runId: {{ requiredEnv "RUN_ID" | quote }} labels: - e2e.hyperfleet.io/run-id: {{ requiredEnv "RUN_ID" }} + e2e.hyperfleet.io/run-id: {{ requiredEnv "RUN_ID" | quote }}Apply the same fix at Line 41 for
e2e-kind.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@helmfile/helmfile.yaml.gotmpl` around lines 30 - 41, The label value in the Helmfile templates is only partially quoted, so the remaining `labels.e2e.hyperfleet.io/run-id` entry in `e2e-kind` still needs the same fix as `runId`. Update the `helmfile.yaml.gotmpl` values under the `e2e-kind` block so the `requiredEnv "RUN_ID"` value is passed through `quote`, matching the existing `runId` handling and keeping the label value a string. Use the `e2e-kind` and `labels.e2e.hyperfleet.io/run-id` keys to locate the unquoted scalar.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@helmfile/values/base-adapter.yaml.gotmpl`:
- Around line 40-43: The RUN_ID guard in the base-adapter template only checks
for the presence of .Values.runId, so an empty value still gets injected into
the container. Update the conditional around the RUN_ID env entry to reject
empty strings as well as missing values, using the existing runId/hasKey logic
in base-adapter.yaml.gotmpl so only non-empty RUN_ID values are rendered.
---
Duplicate comments:
In `@helmfile/helmfile.yaml.gotmpl`:
- Around line 30-41: The label value in the Helmfile templates is only partially
quoted, so the remaining `labels.e2e.hyperfleet.io/run-id` entry in `e2e-kind`
still needs the same fix as `runId`. Update the `helmfile.yaml.gotmpl` values
under the `e2e-kind` block so the `requiredEnv "RUN_ID"` value is passed through
`quote`, matching the existing `runId` handling and keeping the label value a
string. Use the `e2e-kind` and `labels.e2e.hyperfleet.io/run-id` keys to locate
the unquoted scalar.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Central YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 8b7090b7-194b-4d0f-aaff-5c38a558bb45
📒 Files selected for processing (6)
helmfile/configs/e2e/adapters/cl-job/adapter-task-resource-job.yamlhelmfile/configs/e2e/adapters/cl-maestro/adapter-task-resource-manifestwork.yamlhelmfile/helmfile.yaml.gotmplhelmfile/values/base-adapter.yaml.gotmplhelmfile/values/base-api.yaml.gotmplhelmfile/values/base-sentinel.yaml.gotmpl
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
openshift-hyperfleet/architecture(manual)openshift-hyperfleet/hyperfleet-api(manual)openshift-hyperfleet/hyperfleet-sentinel(manual)openshift-hyperfleet/hyperfleet-adapter(manual)openshift-hyperfleet/hyperfleet-broker(manual)
✅ Files skipped from review due to trivial changes (3)
- helmfile/values/base-sentinel.yaml.gotmpl
- helmfile/values/base-api.yaml.gotmpl
- helmfile/configs/e2e/adapters/cl-job/adapter-task-resource-job.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
- helmfile/configs/e2e/adapters/cl-maestro/adapter-task-resource-manifestwork.yaml
| {{- if hasKey .Values "runId" }} | ||
| - name: RUN_ID | ||
| value: {{ .Values.runId | quote }} | ||
| {{- end }} |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
CWE-703: hasKey check lets empty RUN_ID through.
requiredEnv "RUN_ID" only verifies the var is set, not non-empty. RUN_ID="" produces runId: "", which now passes hasKey .Values "runId" and injects RUN_ID="" into the container — a silently corrupted trust-boundary value instead of a hard failure.
Fix
- {{- if hasKey .Values "runId" }}
+ {{- if and (hasKey .Values "runId") .Values.runId }}
- name: RUN_ID
value: {{ .Values.runId | quote }}
{{- end }}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| {{- if hasKey .Values "runId" }} | |
| - name: RUN_ID | |
| value: {{ .Values.runId | quote }} | |
| {{- end }} | |
| {{- if and (hasKey .Values "runId") .Values.runId }} | |
| - name: RUN_ID | |
| value: {{ .Values.runId | quote }} | |
| {{- end }} |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@helmfile/values/base-adapter.yaml.gotmpl` around lines 40 - 43, The RUN_ID
guard in the base-adapter template only checks for the presence of
.Values.runId, so an empty value still gets injected into the container. Update
the conditional around the RUN_ID env entry to reject empty strings as well as
missing values, using the existing runId/hasKey logic in
base-adapter.yaml.gotmpl so only non-empty RUN_ID values are rendered.
| hyperfleet.io/resource-type: "job" | ||
| e2e.hyperfleet.io/run-id: "{{ .runId }}" | ||
| app: test-job | ||
| e2e.hyperfleet.io/run-id: "{{ .runId }}" |
| fieldRef: | ||
| fieldPath: metadata.namespace | ||
| {{- if and (hasKey .Values "runId") .Values.runId }} | ||
| {{- if hasKey .Values "runId" }} |
There was a problem hiding this comment.
No need the hasKey to prevent issues. Addressed this yesterday in the first pr.
2 participants
Summary
This PR adds support for labeling all resources in the
e2e-gcpenvironment with a uniquee2e.hyperfleet.io/run-idlabel, enabling better tracking and cleanup of resources created during E2E test runs.Caution
TODO: Prepare change for openshift/release repo CI for hyperfleet-e2e
What changed:
E2E_RUN_IDenvironment variable fore2e-gcpdeploymentse2e-gcpare automatically labeled withe2e.hyperfleet.io/run-id: <run-id>E2E_RUN_IDas an environment variablee2eRunIdparameter to resources they create (Namespaces, Jobs, Deployments, ConfigMaps, ManifestWorks)E2E_RUN_IDis set when deploying toe2e-gcpWhy this is needed:
Impact on users:
E2E_RUN_IDto be set when deploying (enforced by validation)E2E_RUN_IDwhen runningmake install-hyperfleetUsage:
Test Plan
make test-allpasses - N/A (no test suite for helmfile templates)make lintpasses - Validated withmake ci-validatemake test-helm(if applicable) - N/A (changes are in helmfile values, not charts)make template-helmfileManual Testing Performed:
e2e-gcpdeployment fails withoutE2E_RUN_IDe2e-gcpdeployment succeeds withE2E_RUN_IDe2e-gcpare labeled withe2e.hyperfleet.io/run-idE2E_RUN_IDenvironment variablegcp,kind, ande2e-kindenvironments build without changese2e.hyperfleet.io/run-idlabelBreaking Changes
For e2e-gcp environment only:
E2E_RUN_IDenvironment variable is now required when deploying toe2e-gcpE2E_RUN_IDis not setHELMFILE_ENV=e2e-gcpmust be updated to passE2E_RUN_ID