From 43abce959aac98ea0e3d53814d98826c99d1c6ab Mon Sep 17 00:00:00 2001 From: Chris Nussbaum Date: Sat, 1 Aug 2026 21:40:56 -0500 Subject: [PATCH 1/3] Align lint config with upstream ESPHome; validate and compile in CI Comparing this repo's tooling against esphome/esphome turned up one gap that mattered more than the rest: ruff was installed but never configured. With no pyproject.toml it fell back to its defaults (E4, E7, E9, F), which is roughly pyflakes. Upstream selects ~30 rule families. Most visibly, isort was off, so import ordering was never enforced despite ruff-format running on every commit -- formatting and import sorting are separate concerns in ruff. The CI side had a different problem. Nothing validated the device configs at all. yamllint checks whitespace; it cannot catch an undefined substitution, a broken package reference, or a component option renamed by an ESPHome upgrade. For a repo that is mostly device configs, that was the largest real risk, and nothing verified the custom C++ components still compiled either. Changes: - Add pyproject.toml mirroring upstream's [tool.ruff] and [tool.pylint]. One deliberate divergence: upstream sets known-first-party = ["esphome"] because it is esphome; here esphome is a dependency, so it belongs in the third-party section and this repo's code is reached by relative imports. - Unify the Python version on 3.12 (CI, pyupgrade, ruff target-version). It was previously three-way inconsistent: CI on 3.11, pyupgrade on py311-plus, and ruff defaulting to py39 with no requires-python. - Add pylint, with esphome as a hook dependency so the component imports resolve. esphome.components.econet is in ignored-modules because devices pull it from esphome-econet at config time; it can never resolve statically. - Widen the flake8 scope to cover scripts/ as well as components/. - Fix what the new rules found: import ordering (ruff --fix), one nested if (SIM102), one range(len(...)) loop (C0200). CI: - Merge lint.yml and the new config/compile jobs into a single ci.yml, so the triggers and the concurrency group are defined once. Check contexts are job names, so this rename does not affect required-check config. - Skip no-commit-to-branch in CI. It is a local guardrail against committing to master, and it failed every post-merge push to master -- the red X after each merge was always this and nothing else. - Validate every device with esphome config. devices/secrets.yaml is gitignored, so scripts/generate_ci_secrets.py writes a placeholder one from the !secret references found in tracked YAML. - Compile five devices covering all six custom components across both platforms. The components compile behind USE_ESP32/USE_ESP8266 guards, so an esp32 build says nothing about the esp8266 ones. Compiles are skipped on feature-branch pushes and run on PRs and master. Remove .deepsource.toml. Its Python analyzer had been failing master on 14 issues that this repo's own config deliberately ignores -- an 88-character line limit against the configured 120, and missing docstrings under D100-D107. Its C & C++ analyzer reported 0 issues across 3708 lines, which reflects having no build configuration to resolve the esphome or ESP-IDF headers rather than clean code. The compile matrix has not run yet; this PR is its first exercise. Co-Authored-By: Claude Opus 5 --- .deepsource.toml | 10 -- .github/workflows/ci.yml | 105 ++++++++++++++++ .github/workflows/lint.yml | 15 --- .pre-commit-config.yaml | 16 ++- components/bed_sensor/__init__.py | 5 +- components/pool_controller/__init__.py | 14 +-- components/pool_controller/schema/heater.py | 3 +- components/pool_controller/schema/pump.py | 69 +++++----- components/pool_controller/schema/schedule.py | 33 +++-- components/pool_controller/schema/types.py | 13 +- components/pzem6l24/sensor.py | 4 +- components/treo_led_pool_light/light.py | 1 - pyproject.toml | 118 ++++++++++++++++++ scripts/generate_ci_secrets.py | 95 ++++++++++++++ scripts/tag_esphome_version.py | 2 +- 15 files changed, 407 insertions(+), 96 deletions(-) delete mode 100644 .deepsource.toml create mode 100644 .github/workflows/ci.yml delete mode 100644 .github/workflows/lint.yml create mode 100644 pyproject.toml create mode 100644 scripts/generate_ci_secrets.py diff --git a/.deepsource.toml b/.deepsource.toml deleted file mode 100644 index 7099a92..0000000 --- a/.deepsource.toml +++ /dev/null @@ -1,10 +0,0 @@ -version = 1 - -[[analyzers]] -name = "python" - - [analyzers.meta] - runtime_version = "3.x.x" - -[[analyzers]] -name = "cxx" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..f729d73 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,105 @@ +--- +name: CI + +on: + push: + pull_request: + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + pre-commit: + name: pre-commit + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - uses: pre-commit/action@v3.0.1 + env: + # Local-only guardrail; being on master is expected in CI. + SKIP: no-commit-to-branch + + esphome-config: + name: esphome config + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + # Deliberately unpinned: the point of this job is to catch configs that + # break against the current ESPHome release. Pin here if you need to hold + # the repo at a specific version. + - name: Install ESPHome + run: pip install esphome + - name: Report ESPHome version + run: esphome version + # devices/secrets.yaml is gitignored, so CI generates a placeholder one. + - name: Generate placeholder secrets + run: python scripts/generate_ci_secrets.py + - name: Validate device configs + run: | + status=0 + for config in devices/*.yaml; do + if [ "$(basename "$config")" = "secrets.yaml" ]; then + continue + fi + echo "::group::$config" + if ! esphome config "$config" > /dev/null; then + echo "::error file=$config::esphome config failed" + status=1 + fi + echo "::endgroup::" + done + exit $status + + # Compiling proves the custom C++ components in components/ still build. The + # matrix is the smallest set of devices that covers every custom component, + # across both platforms -- the components are compiled per-platform, so an + # esp32 build says nothing about the esp8266 ones. + esphome-compile: + name: compile ${{ matrix.device }} + runs-on: ubuntu-latest + # Compiles are slow, so skip them on feature-branch pushes; the pull request + # and the post-merge master push both still get a full build. + if: github.event_name == 'pull_request' || github.ref == 'refs/heads/master' + env: + # Cache the ESP-IDF install rather than re-downloading it every run. + ESPHOME_ESP_IDF_PREFIX: ~/.esphome-idf + strategy: + fail-fast: false + matrix: + include: + - device: pool # esp32s3 / esp-idf: pool_controller, pzem6l24 + platform: esp32 + - device: water-heater # esp32 / esp-idf: high_temp_water_heater + platform: esp32 + - device: hvac # esp32 / esp-idf: econet_zone_control + platform: esp32 + - device: pool-lights # esp8266: treo_led_pool_light + platform: esp8266 + - device: master-bed # esp8266: bed_sensor + platform: esp8266 + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Cache PlatformIO and ESP-IDF + uses: actions/cache@v4 + with: + path: | + ~/.platformio + ~/.esphome-idf + key: build-${{ matrix.platform }}-${{ hashFiles('devices/packages/device_base*.yaml') }} + restore-keys: build-${{ matrix.platform }}- + - name: Install ESPHome + run: pip install esphome + - name: Generate placeholder secrets + run: python scripts/generate_ci_secrets.py + - name: Compile ${{ matrix.device }} + run: esphome compile devices/${{ matrix.device }}.yaml diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml deleted file mode 100644 index 8782934..0000000 --- a/.github/workflows/lint.yml +++ /dev/null @@ -1,15 +0,0 @@ -name: Lint - -on: - push: - pull_request: - -jobs: - pre-commit: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 - with: - python-version: "3.11" - - uses: pre-commit/action@v3.0.1 diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 4247ece..2ba480d 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -13,7 +13,7 @@ default_install_hook_types: repos: - repo: https://github.com/astral-sh/ruff-pre-commit # Ruff version. - rev: v0.15.15 + rev: v0.16.0 hooks: # Run the linter. - id: ruff @@ -27,7 +27,7 @@ repos: additional_dependencies: - flake8-docstrings==1.7.0 - pydocstyle==5.1.1 - files: ^components/.+\.py$ + files: ^(components|scripts)/.+\.py$ - repo: https://github.com/pre-commit/pre-commit-hooks rev: v5.0.0 hooks: @@ -42,7 +42,7 @@ repos: rev: v3.21.2 hooks: - id: pyupgrade - args: [--py311-plus] + args: [--py312-plus] - repo: https://github.com/adrienverge/yamllint.git rev: v1.37.1 hooks: @@ -53,6 +53,16 @@ repos: hooks: - id: clang-format types_or: [c, c++] + - repo: https://github.com/pylint-dev/pylint + rev: v4.0.6 + hooks: + - id: pylint + files: ^(components|scripts)/.+\.py$ + # esphome must be importable or every `from esphome...` import in the + # custom components resolves to import-error. Deliberately unpinned so + # components are linted against the current release. + additional_dependencies: + - esphome - repo: local hooks: - id: tag-esphome-version diff --git a/components/bed_sensor/__init__.py b/components/bed_sensor/__init__.py index 1a85ca9..6367480 100644 --- a/components/bed_sensor/__init__.py +++ b/components/bed_sensor/__init__.py @@ -1,9 +1,6 @@ import esphome.codegen as cg +from esphome.components import binary_sensor, output, sensor, text_sensor import esphome.config_validation as cv -from esphome.components import binary_sensor -from esphome.components import output -from esphome.components import sensor -from esphome.components import text_sensor from esphome.const import ( CONF_ID, CONF_OUTPUT, diff --git a/components/pool_controller/__init__.py b/components/pool_controller/__init__.py index 8057dcb..3a71809 100644 --- a/components/pool_controller/__init__.py +++ b/components/pool_controller/__init__.py @@ -1,24 +1,24 @@ import esphome.codegen as cg -import esphome.config_validation as cv from esphome.components import binary_sensor, time +import esphome.config_validation as cv from esphome.const import CONF_ID, CONF_NAME, CONF_TIME_ID from .const import ( - CONF_PRIMARY_PUMP, CONF_AUXILIARY_PUMPS, - CONF_SEQUENCE_DELAY, CONF_DISABLE_PUMPS_SENSOR, - CONF_STATE_SAVE_INTERVAL, CONF_MAX_RESUME_AGE, + CONF_PRIMARY_PUMP, + CONF_SEQUENCE_DELAY, + CONF_STATE_SAVE_INTERVAL, ) -from .schema.types import PoolController +from .schema.heater import CONF_POOL_HEATER, POOL_HEATER_SCHEMA, heater_to_code from .schema.pump import ( - PRIMARY_PUMP_SCHEMA, AUX_PUMP_SCHEMA, + PRIMARY_PUMP_SCHEMA, pump_to_code, schedule_select_to_code, ) -from .schema.heater import CONF_POOL_HEATER, POOL_HEATER_SCHEMA, heater_to_code +from .schema.types import PoolController AUTO_LOAD = [ "binary_sensor", diff --git a/components/pool_controller/schema/heater.py b/components/pool_controller/schema/heater.py index 197bd99..9da5ea1 100644 --- a/components/pool_controller/schema/heater.py +++ b/components/pool_controller/schema/heater.py @@ -1,7 +1,8 @@ import esphome.codegen as cg -import esphome.config_validation as cv from esphome.components import output, sensor, water_heater +import esphome.config_validation as cv from esphome.const import CONF_OUTPUT + from .types import PoolHeater # Constants are defined inline here because they are only used for the pool heater. diff --git a/components/pool_controller/schema/pump.py b/components/pool_controller/schema/pump.py index 83fe19f..73baa84 100644 --- a/components/pool_controller/schema/pump.py +++ b/components/pool_controller/schema/pump.py @@ -1,62 +1,69 @@ import uuid +from esphome import automation import esphome.codegen as cg +from esphome.components import ( + binary_sensor, + button, + output, + select as esphome_select, + sensor, + switch as esphome_switch, + text_sensor, +) import esphome.config_validation as cv -from esphome import automation -from esphome.components import binary_sensor, button, output, sensor, text_sensor -from esphome.components import switch as esphome_switch -from esphome.components import select as esphome_select from esphome.const import ( CONF_ID, CONF_NAME, CONF_OUTPUT, - DEVICE_CLASS_PROBLEM, CONF_TRIGGER_ID, + DEVICE_CLASS_PROBLEM, ENTITY_CATEGORY_DIAGNOSTIC, ) + from ..const import ( - CONF_SCHEDULES, - CONF_RUNTIMES, - CONF_START_TIME, - CONF_END_TIME, - CONF_MINUTES_PER_HOUR, - CONF_SCHEDULE_SELECT, - CONF_CURRENT_SENSOR, - CONF_ANOMALY_DETECTION_SWITCH, CONF_ANOMALY_BASELINE_RESET_BUTTON, - CONF_ANOMALY_STATUS_BINARY_SENSOR, + CONF_ANOMALY_DETECTION_SWITCH, CONF_ANOMALY_REASON_TEXT_SENSOR, + CONF_ANOMALY_STATUS_BINARY_SENSOR, CONF_ANOMALY_THRESHOLD_PCT, - CONF_LEARNING_SAMPLES, - CONF_ON_ANOMALY, CONF_CURRENT_ON_THRESHOLD, - CONF_NO_CURRENT_BINARY_SENSOR, - CONF_CURRENT_TIMEOUT, + CONF_CURRENT_SENSOR, CONF_CURRENT_STALE_BINARY_SENSOR, + CONF_CURRENT_TIMEOUT, + CONF_END_TIME, + CONF_FLOW_LOSS_BINARY_SENSOR, CONF_FLOW_SENSOR, CONF_FLOW_TIMEOUT, - CONF_FLOW_LOSS_BINARY_SENSOR, + CONF_LEARNING_SAMPLES, + CONF_MINUTES_PER_HOUR, + CONF_NO_CURRENT_BINARY_SENSOR, + CONF_ON_ANOMALY, + CONF_RUNTIMES, + CONF_SCHEDULE_SELECT, + CONF_SCHEDULES, + CONF_START_TIME, CONF_UNEXPECTED_FLOW_BINARY_SENSOR, ) +from .schedule import ( + SCHEDULE_SCHEMA, + _days_to_mask, + _time_to_minutes, + _validate_unique_schedule_names, +) from .types import ( - ScheduleSelect, - PrimaryPumpSwitch, AuxiliaryPumpSwitch, - PumpAnomalySwitch, - PumpAnomalyStatusBinarySensor, + PrimaryPumpSwitch, PumpAnomalyReasonTextSensor, PumpAnomalyResetButton, - PumpNoCurrentBinarySensor, + PumpAnomalyStatusBinarySensor, + PumpAnomalySwitch, + PumpAnomalyTrigger, PumpCurrentStaleBinarySensor, PumpFlowLossBinarySensor, + PumpNoCurrentBinarySensor, PumpUnexpectedFlowBinarySensor, - PumpAnomalyTrigger, -) -from .schedule import ( - SCHEDULE_SCHEMA, - _validate_unique_schedule_names, - _time_to_minutes, - _days_to_mask, + ScheduleSelect, ) DEFAULT_ANOMALY_DETECTION_SWITCH_PREFIX = ( diff --git a/components/pool_controller/schema/schedule.py b/components/pool_controller/schema/schedule.py index bc9f935..86dd5c5 100644 --- a/components/pool_controller/schema/schedule.py +++ b/components/pool_controller/schema/schedule.py @@ -1,5 +1,5 @@ -import esphome.config_validation as cv from esphome.components.time import validate_cron_days_of_week +import esphome.config_validation as cv from esphome.const import ( CONF_DAYS_OF_WEEK, CONF_HOUR, @@ -7,12 +7,8 @@ CONF_NAME, CONF_SECOND, ) -from ..const import ( - CONF_END_TIME, - CONF_MINUTES_PER_HOUR, - CONF_RUNTIMES, - CONF_START_TIME, -) + +from ..const import CONF_END_TIME, CONF_MINUTES_PER_HOUR, CONF_RUNTIMES, CONF_START_TIME def _time_to_minutes(time_val): @@ -84,21 +80,24 @@ def _validate_runtime(runtime): def _validate_no_runtime_overlaps(schedule): """Validate that no two runtimes in the same schedule overlap on the same days.""" runtimes = schedule[CONF_RUNTIMES] - for i in range(len(runtimes)): + for i, a in enumerate(runtimes): for j in range(i + 1, len(runtimes)): - a, b = runtimes[i], runtimes[j] + b = runtimes[j] a_start = _time_to_minutes(a[CONF_START_TIME]) a_end = _time_to_minutes(a[CONF_END_TIME]) b_start = _time_to_minutes(b[CONF_START_TIME]) b_end = _time_to_minutes(b[CONF_END_TIME]) - if a_start < b_end and b_start < a_end: - if _runtime_days(a) & _runtime_days(b): - raise cv.Invalid( - f"Runtime {i} ({_time_to_str(a[CONF_START_TIME])}-" - f"{_time_to_str(a[CONF_END_TIME])}) overlaps with " - f"runtime {j} ({_time_to_str(b[CONF_START_TIME])}-" - f"{_time_to_str(b[CONF_END_TIME])}) on shared days" - ) + if ( + a_start < b_end + and b_start < a_end + and _runtime_days(a) & _runtime_days(b) + ): + raise cv.Invalid( + f"Runtime {i} ({_time_to_str(a[CONF_START_TIME])}-" + f"{_time_to_str(a[CONF_END_TIME])}) overlaps with " + f"runtime {j} ({_time_to_str(b[CONF_START_TIME])}-" + f"{_time_to_str(b[CONF_END_TIME])}) on shared days" + ) return schedule diff --git a/components/pool_controller/schema/types.py b/components/pool_controller/schema/types.py index d311dc5..9eb0207 100644 --- a/components/pool_controller/schema/types.py +++ b/components/pool_controller/schema/types.py @@ -1,8 +1,13 @@ -import esphome.codegen as cg from esphome import automation -from esphome.components import binary_sensor, button, text_sensor, water_heater -from esphome.components import switch as esphome_switch -from esphome.components import select as esphome_select +import esphome.codegen as cg +from esphome.components import ( + binary_sensor, + button, + select as esphome_select, + switch as esphome_switch, + text_sensor, + water_heater, +) pool_controller_ns = cg.esphome_ns.namespace("pool_controller") diff --git a/components/pzem6l24/sensor.py b/components/pzem6l24/sensor.py index 9cb9dea..eb069f9 100644 --- a/components/pzem6l24/sensor.py +++ b/components/pzem6l24/sensor.py @@ -4,8 +4,8 @@ from esphome.components import modbus, sensor import esphome.config_validation as cv from esphome.const import ( - CONF_ID, CONF_FREQUENCY, + CONF_ID, DEVICE_CLASS_APPARENT_POWER, DEVICE_CLASS_CURRENT, DEVICE_CLASS_ENERGY, @@ -20,9 +20,9 @@ STATE_CLASS_TOTAL_INCREASING, UNIT_AMPERE, UNIT_HERTZ, - UNIT_KILOWATT_HOURS, UNIT_KILOVOLT_AMPS_HOURS, UNIT_KILOVOLT_AMPS_REACTIVE_HOURS, + UNIT_KILOWATT_HOURS, UNIT_VOLT, UNIT_VOLT_AMPS, UNIT_VOLT_AMPS_REACTIVE, diff --git a/components/treo_led_pool_light/light.py b/components/treo_led_pool_light/light.py index ce8b40d..8f96bfd 100644 --- a/components/treo_led_pool_light/light.py +++ b/components/treo_led_pool_light/light.py @@ -9,7 +9,6 @@ import esphome.config_validation as cv from esphome.const import CONF_EFFECTS, CONF_NAME, CONF_OUTPUT, CONF_OUTPUT_ID - treo_light_ns = cg.esphome_ns.namespace("treo_light") TreoPoolLightOutput = treo_light_ns.class_( "TreoPoolLightOutput", light.LightOutput, cg.Component diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..5271725 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,118 @@ +# Lint/format configuration for this ESPHome config repo. +# +# Mirrors the upstream ESPHome repo's tooling (esphome/esphome) so that custom +# components here are held to the same standard as components upstream. Any +# intentional divergence from upstream is called out in a comment below. + +[tool.pylint.MAIN] +py-version = "3.12" +persistent = false + +[tool.pylint.REPORTS] +score = false + +[tool.pylint."MESSAGES CONTROL"] +disable = [ + "format", + "missing-docstring", + "fixme", + "unused-argument", + "global-statement", + "too-few-public-methods", + "too-many-lines", + "too-many-locals", + "too-many-ancestors", + "too-many-branches", + "too-many-statements", + "too-many-arguments", + "too-many-positional-arguments", + "too-many-return-statements", + "too-many-instance-attributes", + "duplicate-code", + "invalid-name", + "cyclic-import", + "redefined-builtin", + "undefined-loop-variable", + "useless-object-inheritance", + "stop-iteration-return", + "import-outside-toplevel", + # Broken + "unsupported-membership-test", + "unsubscriptable-object", +] + +[tool.pylint.FORMAT] +expected-line-ending-format = "LF" + +[tool.pylint.TYPECHECK] +# esphome.components.econet is a third-party external component that devices pull +# from https://github.com/esphome-econet/esphome-econet at config time. It is not +# part of the esphome package, so pylint can never resolve it statically. +ignored-modules = [ + "esphome.components.econet", +] + +[tool.ruff] +required-version = ">=0.5.0" +target-version = "py312" + +[tool.ruff.lint] +# Preview mode is scoped: with explicit-preview-rules only rules named in +# select run in preview, prefixes like "PL" keep their stable set. +preview = true +explicit-preview-rules = true +select = [ + "B", # flake8-bugbear + "BLE", # flake8-blind-except + "C4", # flake8-comprehensions + "DTZ", # flake8-datetimez + "E", # pycodestyle + "EXE", # flake8-executable + "F", # pyflakes/autoflake + "FA", # flake8-future-annotations + "FLY", # flynt: convert string formatting to f-strings + "FURB", # refurb + "G", # flake8-logging-format + "I", # isort + "ICN", # flake8-import-conventions + "ISC", # flake8-implicit-str-concat + "LOG", # flake8-logging + "NPY", # numpy-specific rules + "PERF", # performance + "PGH", # pygrep-hooks + "PIE", # flake8-pie + "PL", # pylint + "PLW1514", # require explicit encoding on text file I/O (Windows defaults to cp1252) + "PTH", # flake8-use-pathlib + "PYI", # flake8-pyi + "Q", # flake8-quotes + "RSE", # flake8-raise + "SIM", # flake8-simplify + "SLOT", # flake8-slots + "RET", # flake8-ret + "T10", # flake8-debugger + "UP", # pyupgrade + "W", # pycodestyle warnings + "YTT", # flake8-2020 +] + +ignore = [ + "E501", # line too long + "PLC0415", # `import` should be at the top-level of a file + "PLR0911", # Too many return statements ({returns} > {max_returns}) + "PLR0912", # Too many branches ({branches} > {max_branches}) + "PLR0913", # Too many arguments to function call ({c_args} > {max_args}) + "PLR0915", # Too many statements ({statements} > {max_statements}) + "PLR0917", # Too many positional arguments ({c_pos} > {max_pos}) + "PLW1641", # Object does not implement `__hash__` method + "PLR2004", # Magic value used in comparison, consider replacing {value} with a constant variable + "PLW2901", # Outer {outer_kind} variable {name} overwritten by inner {inner_kind} target +] + +[tool.ruff.lint.isort] +force-sort-within-sections = true +# Diverges from upstream: upstream sets known-first-party = ["esphome"] because +# it *is* esphome. Here esphome is an external dependency, so it belongs in the +# third-party section; this repo's own code is reached via relative imports. +combine-as-imports = true +split-on-trailing-comma = false diff --git a/scripts/generate_ci_secrets.py b/scripts/generate_ci_secrets.py new file mode 100644 index 0000000..8fdb0a3 --- /dev/null +++ b/scripts/generate_ci_secrets.py @@ -0,0 +1,95 @@ +#!/usr/bin/env python3 +"""Generate a throwaway devices/secrets.yaml so CI can run `esphome config`. + +The real devices/secrets.yaml is gitignored, but every device config references it +via `!secret`, so validation cannot run without one. This scans the tracked YAML for +`!secret ` references and writes a placeholder value for each. + +The values are deliberately fake and are only ever used for config validation -- +nothing generated here is flashed to a device or connected to a network. Refuses to +overwrite an existing secrets.yaml so it can never clobber real secrets locally. +""" + +import base64 +from pathlib import Path +import re +import subprocess +import sys + +REPO_ROOT = Path(__file__).resolve().parent.parent +DEVICES_DIR = REPO_ROOT / "devices" +SECRETS_FILE = DEVICES_DIR / "secrets.yaml" + +SECRET_RE = re.compile(r"!secret\s+([A-Za-z0-9_]+)") + +# api `encryption.key` must be a valid base64-encoded 32-byte key. +PLACEHOLDER_API_KEY = base64.b64encode(b"esphome-ci-placeholder-key-32byt").decode() +# ota + wifi ap passwords must be non-empty; ap password needs >= 8 characters. +PLACEHOLDER_PASSWORD = "ci-placeholder-password" +PLACEHOLDER_SSID = "ci-placeholder-ssid" + + +def placeholder_for(name): + if name.endswith("_key"): + return PLACEHOLDER_API_KEY + if name.endswith("_ssid"): + return PLACEHOLDER_SSID + return PLACEHOLDER_PASSWORD + + +def tracked_yaml(): + """Yield tracked YAML files under devices/ and components/. + + Uses git rather than a filesystem walk so local build output (devices/build/, + **/.esphome/) is skipped -- those are gitignored, can hold thousands of files, + and never contain !secret references we care about. + """ + result = subprocess.run( + ["git", "ls-files", "-z", "devices/*.yaml", "components/*.yaml"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=True, + ) + for entry in result.stdout.split("\0"): + if entry: + yield REPO_ROOT / entry + + +def referenced_secrets(): + names = set() + for path in tracked_yaml(): + if path == SECRETS_FILE: + continue + names.update(SECRET_RE.findall(path.read_text(encoding="utf-8"))) + return sorted(names) + + +def main(): + if SECRETS_FILE.exists(): + print( + f"generate_ci_secrets: {SECRETS_FILE.name} already exists, refusing to overwrite" + ) + return 1 + + names = referenced_secrets() + if not names: + print("generate_ci_secrets: no !secret references found") + return 1 + + lines = [ + "# Generated by scripts/generate_ci_secrets.py for CI config validation.", + "# These are placeholders, not real secrets. Do not commit this file.", + "", + ] + lines.extend(f"{name}: {placeholder_for(name)}" for name in names) + SECRETS_FILE.write_text("\n".join(lines) + "\n", encoding="utf-8") + + print( + f"generate_ci_secrets: wrote {len(names)} placeholder secrets to {SECRETS_FILE}" + ) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/tag_esphome_version.py b/scripts/tag_esphome_version.py index edbc4d3..46c6ccc 100644 --- a/scripts/tag_esphome_version.py +++ b/scripts/tag_esphome_version.py @@ -10,10 +10,10 @@ """ import json +from pathlib import Path import re import subprocess import sys -from pathlib import Path REPO_ROOT = Path(__file__).resolve().parent.parent DEVICES_DIR = REPO_ROOT / "devices" From dcc2ab7cca1d6f87fb6b261b9804c010f47cb357 Mon Sep 17 00:00:00 2001 From: Chris Nussbaum Date: Sat, 1 Aug 2026 21:48:47 -0500 Subject: [PATCH 2/3] Mark scripts executable so their shebangs are honoured ruff's EXE001 flags a shebang on a non-executable file. Both scripts are invoked as `python scripts/...`, so the shebangs were decorative, but marking the files executable matches upstream's script/ convention and keeps the shebang meaningful. The rule is skipped on Windows, where there is no executable bit, so this only ever fails in CI. Co-Authored-By: Claude Opus 5 --- scripts/generate_ci_secrets.py | 0 scripts/tag_esphome_version.py | 0 2 files changed, 0 insertions(+), 0 deletions(-) mode change 100644 => 100755 scripts/generate_ci_secrets.py mode change 100644 => 100755 scripts/tag_esphome_version.py diff --git a/scripts/generate_ci_secrets.py b/scripts/generate_ci_secrets.py old mode 100644 new mode 100755 diff --git a/scripts/tag_esphome_version.py b/scripts/tag_esphome_version.py old mode 100644 new mode 100755 From 5aadf905eeb8ec7f3c5e2d1cdf797a4ac0957834 Mon Sep 17 00:00:00 2001 From: Chris Nussbaum Date: Sat, 1 Aug 2026 21:57:28 -0500 Subject: [PATCH 3/3] Run CI on pull requests and master, not every branch push Pushing a feature branch with an open PR ran every job twice, once for the push event and once for the pull_request event, and the push run published a check for the skipped compile job under its unsubstituted matrix name (`compile ${{ matrix.device }}`), which is noise in the checks list and a hazard if status checks are ever made required. Branches are still fully covered from the moment their PR is opened, and master still gets a post-merge run. The trade-off is that a branch pushed without a PR now gets no CI at all. Co-Authored-By: Claude Opus 5 --- .github/workflows/ci.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f729d73..ebd3038 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -2,7 +2,12 @@ name: CI on: + # Feature branches are covered by the pull_request trigger. Running on their + # pushes as well would duplicate every job on any branch with an open PR, and + # publish a check for the skipped compile job under its unsubstituted matrix + # name. The trade-off is no CI on a branch until its PR is opened. push: + branches: [master] pull_request: concurrency: