Skip to content

flaky: container-safe flaky-filter + record-results; add stale (max adoptability) - #2

Merged
kei-nan merged 4 commits into
mainfrom
enhance-flaky-adoptability
Jun 24, 2026
Merged

flaky: container-safe flaky-filter + record-results; add stale (max adoptability)#2
kei-nan merged 4 commits into
mainfrom
enhance-flaky-adoptability

Conversation

@kei-nan

@kei-nan kei-nan commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Enhances the shared repo so consumers can fully adopt the flaky-test tooling
(and drop their local flaky_db.py), plus re-adds stale as opt-in.

Why

The flaky composite actions are used inside container test jobs (RediSearch
runs tests in devcontainers). ${{ github.action_path }} points at the runner
host, which is not mounted in containers, so the previously-merged
flaky-record-results would fail to find the bundled flaky_db.py there. And
the fetch/filter half of the pipeline had no shared equivalent, so a repo
couldn't remove its local flaky_db.py.

Changes

  • flaky-filter (new composite action): fetch active marks for the branch
    and filter them out of a caller-provided test list → filtered TESTFILE.
    The caller still enumerates its own tests (LIST=1 make pytest etc. — that's
    product-specific); this action does only the generic fetch + filter.
    Fail-open: empty output ⇒ run the full suite.
  • flaky-record-results (fixed): fetch the DB CLI into the mounted
    workspace
    via actions/checkout instead of github.action_path, so it works
    in container jobs. New ci-common-ref input (set it to your pinned ref).
  • stale (re-added): opt-in reusable workflow (actions/stale) so
    issue-tracking repos (e.g. the public repo) can adopt it too.

Result

flaky-mark / flaky-unmark / flaky-filter / flaky-record-results now cover
the whole flaky pipeline and are container-safe — a consumer keeps only its
product-specific test enumeration and needs no local flaky_db.py. This
unblocks the flaky portion of the OSS adoption (RediSearch/RediSearch#10245).

Validated: strict-YAML (incl. dup-key) + compileall + actionlint via the repo's
own CI.

🤖 Generated with Claude Code


Note

Medium Risk
Changes how CI talks to the flaky Redis DB and alters composite-action behavior in container test jobs; fail-open paths limit blast radius but misconfiguration could skip quarantine or record incorrectly.

Overview
Makes the shared flaky-test pipeline usable from container jobs and lets consumers drop a local flaky_db.py, plus adds an opt-in stale reusable workflow.

flaky-filter (new composite action) fetches branch-keyed marks from Redis and filters a caller-supplied test-id list into a TESTFILE. It fails open (empty testfile ⇒ run the full suite) and sets all-quarantined when every enumerated test was quarantined so shards can be skipped. flaky-record-results no longer invokes flaky_db.py via github.action_path; both actions checkout redisearch-ci-common into the mounted workspace and run the CLI from .redisearch-ci-common, with a new ci-common-ref input (default v1).

stale.yml is a reusable workflow_call wrapper around actions/stale@v10 with tunable stale/close days and runner label. README documents flaky-filter, container-safe usage, and the end-to-end flaky pipeline without per-repo CLI copies.

Reviewed by Cursor Bugbot for commit af541ba. Bugbot is set up for automated code reviews on this repo. Configure here.

…-safe; add stale

Maximize adoptability of the shared flaky tooling so consumers can drop their
local flaky_db.py entirely:

- flaky-filter: new composite action that fetches active marks and filters them
  out of a caller-provided test list (the caller still enumerates its own tests,
  e.g. LIST=1 make pytest -- that part is product-specific). Fail-open: empty
  output means "run the full suite".
- flaky-record-results: fetch the DB CLI into the mounted workspace via checkout
  instead of github.action_path, which is NOT mounted inside container jobs
  (RediSearch tests run in devcontainers). Added a ci-common-ref input.
- stale: re-add as an opt-in reusable workflow so issue-tracking repos (e.g. the
  public repo) can adopt it too.

With flaky-mark / flaky-unmark / flaky-filter / flaky-record-results, the full
flaky pipeline is now shared and container-safe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jit-ci

jit-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

Comment thread .github/actions/flaky-filter/action.yml
When active marks filter out every enumerated test in a shard, an empty
TESTFILE alone reads as "run the full suite" (the runner can't express "run
nothing"), so the quarantined tests would run anyway. Emit a distinct
`all-quarantined` output (true only when marks existed, filter succeeded, the
input had tests, and the filtered set is empty) so a caller can skip the test
step for that shard. Purely additive; callers that ignore it keep the prior
fail-open behavior. Verified across all-filtered / partial / no-marks cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread .github/actions/flaky-filter/action.yml Outdated
flaky_db.py filter drops lines that aren't valid RLTest ids, so a junk input
list also yields an empty filtered output (exit 0) -- which previously set
all-quarantined=true even though nothing was quarantined. Now, when the filtered
output is empty, re-filter with no marks (same id-validation code path) to get
the baseline set that would run; only call it all-quarantined when that baseline
is non-empty. Verified: valid/all-filtered -> true, partial -> false,
junk-input -> false.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 94d84fb. Configure here.

Comment thread .github/workflows/stale.yml Outdated
The reusable stale workflow exposed days-before-close but the close-issue /
close-pr messages hardcoded "7 days" (inherited from the OSS copy). Interpolate
inputs.days-before-close so the posted close comment matches the configured
delay.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kei-nan
kei-nan merged commit 5d3485c into main Jun 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant