Skip to content

CI failure-masking guard: file line numbers, an input budget and smaller functions #957

Open

Description

@cemililik

Follow-ups for tools/check_ci_failure_masking.py after #954. None of them is a masking shape the guard accepts; the shapes it deliberately leaves out (a required command run as $CMD, through make or docker exec, from eval "$(…)", in a remote action, or behind a job-level if: always()) are listed as out of scope in docs/standards/testing.md.

  • Line numbers. A finding counts lines within its run: block ((`run:` block line 3)), not within the workflow file. Report the file line instead; that needs the scalar's position, which yaml.compose keeps and yaml.safe_load drops.
  • Input budget. For every (( the lexer scans ahead for the matching )); text with thousands of unterminated (( takes seconds. Bound or memoise that scan, as matrix values and repository scripts already are.
  • Function size. The analysis is 2,033 lines; its largest functions are scan_steps (73 lines, 29 branch points), command (58, 30), trap_problem (52, 27) and _nested_script (51, 28). Split them the way Fix the wizard's Compliance step and field labels, guard CI against failure masking, correct monitoring copy #954 moved the parser to tools/_shell_syntax.py.
  • A conservative rejection. trap 'rc=$?; let rc=0; exit $rc' EXIT is reported, although bash ends that trap with status 1 (let returns 1 for a zero result and errexit is on). Harmless, but it could be modelled.

Acceptance criteria

  • Findings name the workflow file's line number for run: blocks.
  • A run block of 8,000 unterminated (( scans in well under a second.
  • No function in the guard exceeds roughly 40 lines; behaviour is unchanged, as tests/test_check_ci_failure_masking.py, including its bash comparison, shows.

Activity

  1. cemililik commented on Oct 16, 2026

    @cemililik
    CollaboratorAuthor

    Additions from the last verification round on #954, which fixed the rest of what it found (quoting inside "${…}", trap substitutions, DEBUG/RETURN traps, exec in an EXIT trap, exits written as { exit 0; }, x=1 && exit or a function, fallbacks that leave through eval, $PWD script paths, self-running scripts, if: failure() / cancelled(), ruff --fix-only, exit ², and a continue after the required command):

    • File size. The guard is now 2,274 lines and its _Analysis class about 1,000. Splitting the class (traps, nested scripts, the if and loop rules) into modules belongs with the function-size item above.
    • Left out by design, and listed in docs/standards/testing.md: a DEBUG trap that skips commands under shopt -s extdebug (under errexit it needs a ! [[ … ]] action); an if: on a job that calls a reusable workflow (the called workflow's own steps are scanned); conditions other than false, failure(), !success() and cancelled(), such as ${{ 1 == 2 }}; a captured status changed by a sourced file outside the repository. Model any of them here if a workflow ever needs it.
    • docker compose run / docker exec. Naming a required command through them is refused as an unknown wrapper. Both propagate the program's exit status, so they could be modelled like uv run; the script inside the container would still be out of reach.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: dev-toolingCode area: dev-toolingenhancementNew feature or requestseverity: lowMinor defect, inaccuracy or rough edge

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions