Skip to content

Ruff CI is fragile to version drift: select=["ALL"] + no ruff version pin will silently break every PR #406

Closed

Description

@rhammen

What's happening

.ruff.toml uses select = ["ALL"], and neither pyproject.toml (required-version = ">=0.15.0", no upper bound) nor .github/workflows/validate.yaml pin an exact ruff version - the workflow's ruff format/ruff check steps just fetch whatever the constraint allows, which is currently ruff 0.16.0.

This makes CI fragile in two compounding ways whenever ruff cuts a new release:

  1. New formatting behavior: ruff 0.16.0 changed how it formats Python code blocks embedded in Markdown. Under 0.16.0, ruff format --diff . on current master reports DEVELOPMENT.md would be reformatted (a blank line inserted after import json in one code fence). This fails the first ("Run ruff format") step of the Ruff validation job.

  2. New rule auto-enabled by ALL: ruff 0.16.0 also added CPY001 (missing-copyright-notice). Because select = ["ALL"] opts into every rule ruff ships, this rule is now active repo-wide and isn't in .ruff.toml's ignore list. Running ruff check --exclude custom_components/zaptec/zaptec/api.py (the gating check step) with 0.16.0 reports 39 errors, one per file lacking a copyright header - across nearly the whole custom_components/zaptec tree and several test files.

CI hasn't visibly hit problem 2 yet only because problem 1 fails first and aborts the job before the ruff check steps run (confirmed: the failing job's log shows only the Run ruff format step executed). The moment DEVELOPMENT.md gets reformatted to satisfy step 1, the gating ruff check --exclude api.py step will immediately start failing on all 39 files from problem 2 - breaking every currently-open PR's CI, not just whichever PR happens to fix the Markdown formatting.

How I found this

Investigating why PR #404 and #405 (translation additions, unrelated Python changes) were failing "Ruff validation" in CI despite passing locally with ruff 0.15.9 (the version in my local dev environment). Reproduced locally by installing ruff 0.16.0 in an isolated location and running the same two commands CI runs, against plain master.

Suggested fixes (pick one, not prescribing an approach)

  • Pin an exact ruff version (e.g. ruff==0.15.9 or explicitly bump to 0.16.0 deliberately) in both pyproject.toml's required-version and wherever the CI workflow resolves the ruff version, so upgrades are a deliberate, reviewed step instead of silent drift.
  • If moving to 0.16.0 deliberately: reformat DEVELOPMENT.md, and either add copyright headers repo-wide or add CPY001 to .ruff.toml's ignore list (matching how this repo already opts out of other HA-inherited rules that don't fit a community-maintained HACS integration).

Happy to open a PR for whichever direction is preferred - flagging first since it's a repo-wide CI/config decision, not something scoped to any single feature PR.

Activity

  1. added this to the v0.8.8 milestone on Jul 24, 2026
  2. sveinse commented on Jul 24, 2026

    @sveinse
    Collaborator

    I think this is happened in master right now after merging #391. I see that the CI runs ruff 0.16.0 despite requirements.txt defining ruff ==0.15.9 which is precisely what's described in description.

    This issue is right now the most important thing to fix, as master should be clean before proceeding with the PR merges.

    The CI run in master (https://github.com/custom-components/zaptec/actions/runs/30088497464/job/89466165298) fails due to formatting error in DEVELOPMENT.md. I suggest also reducing the scope of what files are run through ruff. I am not able to replicate this error locally thou, I only get the CPY001 error.

  3. steinmn commented on Jul 24, 2026

    @steinmn
    Contributor

    Not entirely sure where the ruff-action-parser is looking for the ruff version in pyproject.toml, so I think the simplest short term solution is to switch over to specifying the action to use the version from requirements.txt (see #407).

    Longer term we should update to ruff v16, but lets reduce the number of open PRs first.

  4. sveinse commented on Jul 24, 2026

    @sveinse
    Collaborator

    Given that #407 has been merged, we have fixed the immediate issue. Thanks @steinmn . I understand it such that we should still keep this issue open, as it addresses the bigger issue. Agreed?

  5. rhammen commented on Jul 24, 2026

    @rhammen
    ContributorAuthor

    You can argue both ways wrt keeping the issue open or not.
    However: with the pinning of the version, the problem is solved. There is no harm in pinning the version, right?
    My view would therefore be: YAGNI, you ain't going to need it.

    Since the issue is solved, let's close the issue: keeps the issue list clean and short, and allows us to focus on what we really want to improve, fix or implement from the issue list first.
    And if at some point we really need/want to upgrade to ruff 16, or if the issue list is empty and we are bored, we make a new 'feature' issue the update. Likely we don't need ruff 16 in the near future.

  6. steinmn commented on Jul 24, 2026

    @steinmn
    Contributor

    There is no harm in pinning the version, right?

    It's called technical debt. Anyone who has had to maintain a project over an extended period of time will tell you that letting your dependencies get outdated is just pushing an increasing amount of pain down the road.

    We want our codebase to stay aligned with HA core, and they appear to adopt the new ruff version ~10 days after they are released. It might be a bit longer for this particular version that has breaking changes, but I doubt it will take long.

  7. sveinse commented on Jul 24, 2026

    @sveinse
    Collaborator

    As discussed in other threads, we should mirror and replicate what Home Assistant does. If they take use of ruff 0.16.0 we should follow suit.

  8. rhammen commented on Jul 25, 2026

    @rhammen
    ContributorAuthor

    Agreed on the principle — we should track HA core's lead here, that's the right long-term posture.

    Out of curiosity I checked where we actually stand against HA core's dev branch right now:

    • ruff: we're on 0.15.22 (post-Use requirements.txt for ruff version in ci-pipeline #407), HA core dev is on 0.15.21 — we're not behind at all here, actually one patch ahead. So there's no live drift to fix for ruff specifically at the moment; the next HA bump (to 0.16.0 or later) is the thing to watch for, not a current gap.
    • pydantic: we're on 2.13.1, HA core pins 2.13.4 — a small gap, low risk.
    • pytest / pytest-asyncio / pytest-mock / pytest-cov: these are unpinned in our requirements_test.txt. HA core pins all of theirs deliberately (pytest==9.0.3, pytest-asyncio==1.4.0, pytest-cov==7.1.0, plus pytest-rerunfailures==16.0.1 constrained repo-wide) specifically because, per their own comment in that file, linters/test tools "should be pinned, as new releases make new things fail." We don't follow that here.

    So ironically, the exact failure mode this issue describes for ruff — an unpinned tool silently updating and breaking CI — is currently a live, un-mitigated risk for our test tooling, not for ruff. Worth pinning requirements_test.txt the same way we did for ruff in #407? Happy to open a small PR for that if there's agreement.

  9. rhammen commented on Jul 25, 2026

    @rhammen
    ContributorAuthor

    Following up on this with something concrete: I tested what actually complying with ruff 0.16.0 requires against current master.

    ruff format --diff: only touches DEVELOPMENT.md — one blank line inserted in an embedded code block.

    ruff check: 31 CPY001 (missing-copyright-notice) errors — no other new rule fires. That's the only real decision point, and there are two ways to handle it:

    1. Ignore it — add CPY001 to .ruff.toml's ignore list, matching how we already opt out of several other HA-inherited rules that don't fit a community HACS integration. Only touches .ruff.toml, DEVELOPMENT.md, and the requirements.txt version bump — no overlap with any currently open PR, so this could go in any time.
    2. Fix it properly — add real copyright headers to all 31 flagged files. That list is basically the whole custom_components/zaptec core + zaptec/ subpackage + several test files (api.py, coordinator.py, entity.py, services.py, validate.py, tests/conftest.py, etc.) — nearly every file the current stack of open PRs is actively touching.

    Which direction do you both prefer — the ignore, or the full header fix? If it's the latter, I'd rather wait until the current PR backlog is merged first, since a header added to any of those files now would create an avoidable rebase/conflict on almost every open PR.

  10. sveinse commented on Jul 25, 2026

    @sveinse
    Collaborator

    Right now I don't consider this urgent, as there is a lot of PRs to review. We can come back to this a little later I think.

  11. steinmn commented on Jul 25, 2026

    @steinmn
    Contributor

    For the CPY001-errors, the simplest approach is to just wait for HA core to decide and do what they do.

  12. sveinse commented on Oct 6, 2026

    @sveinse
    Collaborator

    What is the status for this issue? Can we close it? Or should this be postponed to v0.9.0?

  13. rhammen commented on Oct 6, 2026

    @rhammen
    ContributorAuthor

    HA core is now on ruff 0.16.9, so I opened #440 to follow it. It bumps the pin and ignores CPY001 and PLR0917, which matches HA core. No code changes were needed. Merging #440 closes this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions