flaky: container-safe flaky-filter + record-results; add stale (max adoptability) - #2
Merged
Merged
Conversation
…-safe; add stale Maximize adoptability of the shared flaky tooling so consumers can drop their local flaky_db.py entirely: - flaky-filter: new composite action that fetches active marks and filters them out of a caller-provided test list (the caller still enumerates its own tests, e.g. LIST=1 make pytest -- that part is product-specific). Fail-open: empty output means "run the full suite". - flaky-record-results: fetch the DB CLI into the mounted workspace via checkout instead of github.action_path, which is NOT mounted inside container jobs (RediSearch tests run in devcontainers). Added a ci-common-ref input. - stale: re-add as an opt-in reusable workflow so issue-tracking repos (e.g. the public repo) can adopt it too. With flaky-mark / flaky-unmark / flaky-filter / flaky-record-results, the full flaky pipeline is now shared and container-safe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
🛡️ Jit Security Scan Results✅ No security findings were detected in this PR
Security scan by Jit
|
When active marks filter out every enumerated test in a shard, an empty TESTFILE alone reads as "run the full suite" (the runner can't express "run nothing"), so the quarantined tests would run anyway. Emit a distinct `all-quarantined` output (true only when marks existed, filter succeeded, the input had tests, and the filtered set is empty) so a caller can skip the test step for that shard. Purely additive; callers that ignore it keep the prior fail-open behavior. Verified across all-filtered / partial / no-marks cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
flaky_db.py filter drops lines that aren't valid RLTest ids, so a junk input list also yields an empty filtered output (exit 0) -- which previously set all-quarantined=true even though nothing was quarantined. Now, when the filtered output is empty, re-filter with no marks (same id-validation code path) to get the baseline set that would run; only call it all-quarantined when that baseline is non-empty. Verified: valid/all-filtered -> true, partial -> false, junk-input -> false. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 94d84fb. Configure here.
The reusable stale workflow exposed days-before-close but the close-issue / close-pr messages hardcoded "7 days" (inherited from the OSS copy). Interpolate inputs.days-before-close so the posted close comment matches the configured delay. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Enhances the shared repo so consumers can fully adopt the flaky-test tooling
(and drop their local
flaky_db.py), plus re-addsstaleas opt-in.Why
The flaky composite actions are used inside container test jobs (RediSearch
runs tests in devcontainers).
${{ github.action_path }}points at the runnerhost, which is not mounted in containers, so the previously-merged
flaky-record-resultswould fail to find the bundledflaky_db.pythere. Andthe
fetch/filterhalf of the pipeline had no shared equivalent, so a repocouldn't remove its local
flaky_db.py.Changes
flaky-filter(new composite action): fetch active marks for the branchand filter them out of a caller-provided test list → filtered TESTFILE.
The caller still enumerates its own tests (
LIST=1 make pytestetc. — that'sproduct-specific); this action does only the generic fetch + filter.
Fail-open: empty output ⇒ run the full suite.
flaky-record-results(fixed): fetch the DB CLI into the mountedworkspace via
actions/checkoutinstead ofgithub.action_path, so it worksin container jobs. New
ci-common-refinput (set it to your pinned ref).stale(re-added): opt-in reusable workflow (actions/stale) soissue-tracking repos (e.g. the public repo) can adopt it too.
Result
flaky-mark/flaky-unmark/flaky-filter/flaky-record-resultsnow coverthe whole flaky pipeline and are container-safe — a consumer keeps only its
product-specific test enumeration and needs no local
flaky_db.py. Thisunblocks the flaky portion of the OSS adoption (RediSearch/RediSearch#10245).
Validated: strict-YAML (incl. dup-key) +
compileall+ actionlint via the repo'sown CI.
🤖 Generated with Claude Code
Note
Medium Risk
Changes how CI talks to the flaky Redis DB and alters composite-action behavior in container test jobs; fail-open paths limit blast radius but misconfiguration could skip quarantine or record incorrectly.
Overview
Makes the shared flaky-test pipeline usable from container jobs and lets consumers drop a local
flaky_db.py, plus adds an opt-in stale reusable workflow.flaky-filter(new composite action) fetches branch-keyed marks from Redis and filters a caller-supplied test-id list into a TESTFILE. It fails open (emptytestfile⇒ run the full suite) and setsall-quarantinedwhen every enumerated test was quarantined so shards can be skipped.flaky-record-resultsno longer invokesflaky_db.pyviagithub.action_path; both actions checkoutredisearch-ci-commoninto the mounted workspace and run the CLI from.redisearch-ci-common, with a newci-common-refinput (defaultv1).stale.ymlis a reusableworkflow_callwrapper aroundactions/stale@v10with tunable stale/close days and runner label. README documentsflaky-filter, container-safe usage, and the end-to-end flaky pipeline without per-repo CLI copies.Reviewed by Cursor Bugbot for commit af541ba. Bugbot is set up for automated code reviews on this repo. Configure here.