Skip to content

Hydra Gates reports SUCCESS having judged 0 files on every push to a repo's own default branch #163

Description

@rubenvdlinde

Summary

On a push event (not a PR), the Hydra Gates job resolves its diff base to
the branch it is running on, diffs it against itself, gets zero changed files,
and reports SUCCESS having inspected nothing. The green tick is
indistinguishable from a real one.

Measured 2026-08-05 across the fleet. Five repos' most recent Hydra Gates job is
this shape:

repo run branch conclusion duration scope line, verbatim
openregister 31004095287 development success 36s diff vs origin/development — 0 changed file(s). Base resolves; this PR changes nothing.
shillinq 30980005168 development success 31s diff vs origin/development — 0 changed file(s). Base resolves; this PR changes nothing.
opencatalogi 31004100713 development success 17s diff vs origin/development — 0 changed file(s). Base resolves; this PR changes nothing.
decidesk 30979933593 development success 20s diff vs origin/development — 0 changed file(s). Base resolves; this PR changes nothing.
doriath 31008292790 development success 16s diff vs origin/development — 0 changed file(s). Base resolves; this PR changes nothing.

Four of the five are on branch development itself. On a push to
development in those repos the gate tier can never fail
, for any content,
ever. That is not five coincidences; it is structural.

Mechanism

quality.yml, Resolve the diff base:

if [ "${{ github.event_name }}" = "pull_request" ] || ...; then
  BASE="origin/${{ github.base_ref }}"
else
  # Not a PR. We deliberately set NOTHING and let the package's own
  # precedence chain resolve origin/HEAD -> origin/development -> ...
  BASE=""
fi

The comment is right that inventing a base would be "the HEAD~1 mistake wearing
a different name". But the fallback it chooses instead resolves, for a push to
development in a repo whose origin/HEAD is development, to the branch
being tested
. git diff origin/development...HEAD is then empty by
construction.

The package behaves correctly and says so out loud —
0 changed file(s). Base resolves; this PR changes nothing. — and then exits 0,
because zero failing gates is zero failing gates. The job is what turns that
honest sentence into a green check nobody reads the log of.

The same defect in the other direction

Where origin/HEAD is not the branch being pushed, the fallback picks an
arbitrary long-lived branch and the "diff" becomes the entire delta between two
release lines:

repo branch base chosen changed files
portaliq development origin/beta 144
scholiq development origin/beta 780
pipelinq development origin/beta 900
openconnector development origin/beta 1447

A 1447-file "PR diff" is not diff scoping either. ADR-020 exists so inherited
debt cannot bounce an unrelated change; here every push to development is
judged against the whole unreleased backlog. So the push-event base is
unreliable in both directions, and which direction a repo gets is decided by
where its origin/HEAD happens to point.

Why this matters more than it looks

This is the "falsely GREEN" dead-gate shape: the absence of the check is
byte-identical to the check succeeding. It also silently weakens the fleet
measurement — five of eighteen repos' newest Hydra Gates results are currently
evidence of nothing, and they are counted alongside the eight genuine greens.

Suggested direction (not applied)

  1. On a push, say so in the verdict. If the resolved scope is zero files,
    the job should not report a plain success. Either report the step as
    SKIPPED — no diff to judge on a push to the base branch itself, or fail
    with that reason under an opt-in. A zero-file scope is a legitimate outcome;
    a green check is not the honest way to render it.
  2. Consider running full-repo on push instead of diff-scoped. A push to a
    mainline is exactly the moment the whole-tree debt number is wanted, and it
    cannot bounce anyone's PR. That turns today's vacuous green into the one
    measurement the fleet currently has no scheduled source for.
  3. Do not paper over it by pinning a base per repo. The arbitrary
    origin/beta diffs above are what that produces.

Filed rather than worked around, per the same reasoning as #153.

Found while measuring the v1.2.0 pin rollout.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions