Skip to content

Distrust stale current readings; measure runtime from flow - #86

Merged
nuttytree merged 1 commit into
masterfrom
f/current-staleness-watchdog
Jul 31, 2026
Merged

nuttytree merged 1 commit into
masterfrom
f/current-staleness-watchdog

Conversation

@nuttytree

Copy link
Copy Markdown
Owner

An ESPHome sensor keeps serving its last value indefinitely once its source stops responding, and nothing downstream can tell the difference between that and a real measurement. On two consecutive mornings the PZEM stopped returning valid Modbus frames for exactly the hour both pumps ran, 04:00–05:00, and because its readings froze rather than disappearing, every consumer believed them.

Pump No Current turned on at 04:00:01 and stayed on for the full hour while the pump was actually drawing ~1500 W. Runtime stopped accumulating for a pump that was running. When frames returned at 05:00:02 the current sensor stepped from a frozen value to a live one in a single sample, which spiked the spread estimate past the variance threshold and set Pump Anomaly Status chattering — 285 transitions, reason VARIANCE_SPIKE. The energy sensors appearing to dump a whole hour of consumption at once had the same cause: the meter had been counting internally the entire time, only the reads were dead.

binary_sensor.pool_status never left on and the recorder was healthy throughout, so this was neither an API drop nor a database problem. The only signal that could have distinguished a live reading from a frozen one was when it last arrived, and nothing was tracking that.

This does not fix the underlying dropout, which is still unexplained and most likely electrical — it recovers the instant the cleaner stops, so the cleaner's booster pump is the prime suspect. What this does is stop the controller drawing false conclusions from dead data, and make the outage visible as itself in history.

Changes:

  • Track the arrival time of every current_sensor publish and declare the reading stale after current_timeout (new, default 30s). Add current_stale_binary_sensor so a dead sensor is reported as a dead sensor rather than as a pump fault.
  • While stale: hold motor_running_ instead of recomputing it from a frozen value, suppress no_current (a sensor that stopped publishing supports no claim about the pump), and stop sampling for anomalies.
  • On recovery, clear the debounce streak and the spread estimate so the step change back to live data cannot trip the variance detector, and discard the inrush peak of any run whose startup window contained a gap.
  • Measure runtime from flow_sensor whenever one is configured, falling back to current_sensor and then to the output command. Moving water is what the schedule is actually buying, and flow stays valid through a current-sensor outage — in this incident the flow inputs are on the PCF8574 and were live all hour.
  • Split "a new run started" from "runtime is accumulating", which were tangled in track_runtime(). The anomaly startup window has to stay anchored to electrical turn-on, because flow lags the motor by long enough to miss the inrush peak the startup check exists to measure. turned_on_ms_ is now 0 until a sensor confirms the motor, which also prevents a stale recovery from restarting the startup window mid-run and recording steady-state current as if it were inrush.
  • Document the local ESP-IDF toolchain setup and its pitfalls in CLAUDE.md, and stop the commit_pr command delegating to the pr-workflow skill, which belongs to the upstream esphome/esphome checkout and would target the wrong repository.

No behaviour changes on upgrade beyond the above; nothing persisted is discarded. Both new options are optional with defaults, so existing configs need no edits.

Verified with esphome compile on an ESP32-S3 build of pool.yaml — clean, exit 0, no warnings from any touched file.

An ESPHome sensor keeps serving its last value indefinitely once its source
stops responding, and nothing downstream can tell the difference. On two
consecutive mornings the PZEM stopped returning valid Modbus frames for
exactly the hour both pumps ran (04:00-05:00), and because its readings
froze rather than disappearing, everything believed them: Pump No Current
reported a fault for a solid hour while the pump was drawing ~1500 W,
runtime stopped accumulating for a pump that was running, and when the
frames came back at 05:00:02 the current sensor jumped from a frozen value
to a live one in a single sample — spiking the spread estimate past the
variance threshold and setting the anomaly flag chattering 285 times. The
energy registers appeared to dump a whole hour at once for the same reason:
the meter had been counting internally all along, only the reads were dead.

The device never disconnected and the recorder was healthy throughout, so
the only signal that could have distinguished a live reading from a frozen
one was when it last arrived. Nothing was tracking that.

This does not fix the underlying dropout, which is still unexplained and
most likely electrical. It stops the controller drawing false conclusions
from dead data, and makes the outage visible as itself.

Changes:

- Track the arrival time of every current-sensor publish and declare the
  reading stale after current_timeout (default 30s). Add a
  current_stale_binary_sensor so the real fault is visible as itself.
- While stale: hold motor_running_ rather than recomputing it from a frozen
  value, suppress no_current (a silent sensor supports no claim about the
  pump), and stop sampling for anomalies.
- On recovery, clear the debounce streak and the spread estimate so the
  step change back to live data cannot trip the variance detector, and
  discard the inrush peak of any run whose startup window contained a gap.
- Measure runtime from the flow sensor whenever one is configured, falling
  back to current and then to the output command. Moving water is what the
  schedule is actually buying, and flow stays valid through a current-sensor
  outage.
- Split "a new run started" from "runtime is accumulating". The anomaly
  startup window must stay anchored to electrical turn-on, since flow lags
  the motor by long enough to miss the inrush peak entirely. turned_on_ms_
  is now 0 until a sensor confirms the motor, which also stops a stale
  recovery from restarting the window mid-run and recording steady-state
  current as inrush.
- Document the local ESP-IDF toolchain pitfalls in CLAUDE.md, and stop the
  commit_pr command pointing at the upstream esphome PR workflow.

Verified with esphome compile on an ESP32-S3 build of pool.yaml: clean, no
warnings from any touched file.

Co-Authored-By: Claude <noreply@anthropic.com>
@nuttytree
nuttytree merged commit c8ccc87 into master Jul 31, 2026
2 of 4 checks passed
@nuttytree
nuttytree deleted the f/current-staleness-watchdog branch July 31, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant