Repository navigation
Feature: PV Forecast function with 3 providers - #3782
Conversation
…dataclass_from_dict
…st/current in setdata
…e retained null payloads
…th EP architecture
… forecast/current topic
Sure I'll take that up on my ToDo for the next round and contact you if needed :) |
|
Hey @seaspotter, danke für deine unermüdliche Arbeit! Eine Frage bzw. einen Gedankenanstoß hätte ich. Die Eingabe der Daten zu WR-Leistung und Ausrichtung ist sehr statisch bzw. relativ ungenau und berücksichtigt beispielsweise keine saisonal bzw. monatsbedingt auftretende topografische oder Geländeabhängigen Verschattungen. Zum Vergleich anderer Systeme sehe ich Victron und HomeAsisstant, die jeweils einen Forecast anbieten. Bei Victron bekommt man einen PV-Forecast, ohne dass man überhaupt irgendwo seine Anlagengröße oder Modulausrichtung hinterlegen muss und die Vorhersage dazu noch überaus genau. Aus Nutzersicht ist das maximal komfortabel, denn eine Anmeldung bei einem externen Dienst entfällt und ungenaue statische Daten müssen ebenfalls nicht erfasst werden..nur eben den eigene Anlagenstandort. Ich habe kurz recherchiert und konnte dazu Folgendes herausfinden: Victron holt sich wohl die prognostizierten solaren Einstrahlungswerte in W/m2 und mappt diese mit der historischen tatsächlichen PV-Produktion und der damaligen Einstrahlung. Dadurch fließen Faktoren wie Anlagengröße, Ausrichtung/Neigung, Wirkungsgrad und Verschattung automatisch mit ein, ohne dass der Nutzer diese Daten selbst eingeben muss. Eventuell auch ein sinnvoller, zusätzlicher Weg für openWB? Allerdings muss ich auch gestehen, dass ich keine Ahnung habe wo openWB die Daten der Einstrahlung bekommen kann. VG Kai |
Hab dir mal hier in den Diskussionen geantwortet :) #3785 |
Wie hast Du den Build gemacht? Über das NPM-Skript "build-prod"? Bei der Entwicklung macht es Sinn, immer "dev", also den interaktiven Entwicklungsserver von Vite zu nutzen. Dein Patch macht aber durchaus Sinn. An die ACLs gehen wir erst, wenn der Rest soweit abgesegnet ist und die Topics fixiert sind. Weitere Diskussionen zum UI-Teil dann bitte in dem PR dazu. |
ich hab mit npm run build-prod direkt aufm raspi gearbeitet und hab mich irgendwie immer ausgesperrt gehabt und kam nicht mehr in die settings, mit den Änderungen gings dann. Aber wieder was gelernt :) Danke schonmal. |
…nose-Anbindung) Reine, ungekoppelte Fenster-/Pacing-Logik fuer BatControlMode.PEAK_SHAVING, unabhaengig von der noch nicht gemergten Prognose-Anbindung (PR openWB#3782) entwickelt und getestet. Enthaelt Mindest-Tagesprognose-Abschaltung und rampen-/dwell-begrenzte Ladeleistungs- aenderungen, um das Ladeleistungs-Register bei jedem (Sekundentakt-)Regelzyklus vor unnoetigen Schreibzugriffen zu schuetzen. Anbindung an bat_all.py/get_power_limit() folgt, sobald die Prognose-Daten verfuegbar sind.
…-modules # Conflicts: # data/config/mosquitto/public/default-dynamic-security.json # packages/helpermodules/logger.py
The /estimate/watts endpoint never returns pre-aggregated daily totals, so today_kwh/tomorrow_kwh always showed 0.00 despite valid hourly data. Ports the same variable-timestep integration already used by openmeteo/forecast.py as a fallback.
|
Hallo seaspotter, |
|
Hallo @seaspotter , im Backend gibt es noch ein paar Dinge im Backend die abweichend zu den anderen Modulen sind:
Kannst du das noch anpassen? |
…bei "kein Modul" (analog electricity_pricing)
@ndrsnhs ist jetzt angepasst und habs durchgetestet, wird jetzt sauber zurückgesetzt beim Wechsel auf nen anderen Provider und auch auf none. |
Konflikt in packages/control/optional_test.py mit openWB#3782 (PV-Prognose): beide Seiten ergänzen Importe und Tests, beide behalten. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add PV forecast providers and scheduled publishing * Add provider-aware PV forecast daily values and manual refresh trigger * Add multi-orientation forecast support and practical defaults * Remove unused PVNode scaling config fields * Fix PVNode forecast typing for Python 3.9 * Fix forecast typing for Python 3.9 startup * fix(core): initialize forecast module from provider topic * feat(forecast): add dedicated ramdisk logger * feat(forecast): add provider fetch success logs * fix(forecast): emit openmeteo completion log once * fix(forecast): always write info logs to forecast logger * fix(forecast): keep provider config object retained * fix(forecast): persist provider as object across state updates * fix(forecast): prevent provider mqtt reconfiguration loops * fix(forecast): avoid republishing unchanged provider state * fix(forecast): keep legacy provider type after restart * fix(security): add forecast ACL access topic defaults * fix(security): add forecast role ACLs and bump dynsec version * fix(security): allow wildcard subscription for settings access * fix(security): allow settings access wildcard subscribe pattern * fix(security): assign settings access to default user group * fix(security): simplify forecast acl defaults * fix(forecast): require strings and ignore legacy null provider resets * fix(forecastsolar): use unparameterized List type for strings to fix dataclass_from_dict * fix(openmeteo): use unparameterized List type for strings to fix dataclass_from_dict * fix(forecast): raise_for_status for 429 detection and register forecast/current in setdata * fix(forecast): skip null-type provider reset during startup to survive retained null payloads * fix(forecast): remove provider topic republish from setter - align with EP architecture * fix(forecast): correct Open-Meteo formula, clean configs, remove dead forecast/current topic * refactor(forecast): simplify subdata provider handler to match EP pattern * fix(forecastsolar): switch to /estimate/watts endpoint and handle flat-dict response format * feat(forecast): track and publish last_update_time after successful forecast fetch * fix(update_config): add forecast topics to valid_topic to prevent deletion on startup * fix(forecast): reset last_update_time when provider is removed * fix(security): revert unintended settings-access group assignment * fix(forecast): normalize config before comparison to stop runaway update loop; add optional api_key to forecastsolar * fix(security): restore compact formatting for groups * fix(forecast): restore next_query_time from MQTT state on provider init When forecast provider is re-initialized (e.g., config change, broker restart), next_query_time was reset to None, causing _is_update_due() to always return true. This led to forecast updates running every ~10 seconds (control_interval) instead of respecting the scheduled next_query_time. Now load next_query_time from the MQTT state (data.data.optional_data.data.forecast.get.next_query_time) when creating a new provider instance. This preserves scheduling across provider re-initialization while still allowing immediate first update if no prior time exists. * refactor(forecast): use self.get reference pattern like EP modules Store reference to data.data.optional_data.data.forecast.get in self.get and use it directly for next_query_time, fault_state, etc. This makes the forecast scheduling state automatically persistent across provider re-initialization, matching the pattern used in configurable_tariff.py (EP modules). - next_query_time state now lives in data layer, not instance variable - Automatically preserved on provider re-init without manual reload logic - Simpler, more consistent code pattern across optional modules * fix(forecast): defer first update on new provider init and handle config validation errors When a forecast provider is newly added (next_query_time = 0), defer the first update attempt by 2 minutes to allow the user to configure required fields and save before any API calls are made. Additionally, treat 'Missing required config field' errors specially: - Don't retry after 15 minutes (like other errors) - Instead wait until next scheduled update time - Display clear warning message about incomplete configuration - Prevents error spam in logs when config is still being filled in This addresses both the premature API calls on provider add and the repetitive retry behavior when configuration is incomplete. * feat(forecast): defer init only when config incomplete, support all providers Refine deferred first-update logic: only defer if the provider configuration is actually incomplete. This allows immediate updates for fully-configured providers while giving incomplete configs time to be finished. Config completeness checks: - PVNode: plant_id must be set and non-empty - Open-Meteo: must have at least one string (Dachfläche) configured - Forecast.Solar: must have at least one string (Dachfläche) configured This prevents unnecessary delays for users who add a pre-configured provider, while still giving time for new configs to be filled in. * refactor(forecast): move config validation to provider modules Make config validation generic and scalable: instead of hardcoding provider- specific checks in configurable_forecast.py, each provider module now defines its own is_configuration_complete() function. Benefits: - New providers can be added without modifying base ConfigurableForecast class - Validation logic stays close to the provider implementation - Easy to test per-provider validation rules independently Each provider validates: - PVNode: plant_id must be set and non-empty - Open-Meteo: at least one string (Dachfläche) must be configured - Forecast.Solar: at least one string (Dachfläche) must be configured ConfigurableForecast dynamically imports and calls the validation function, with graceful fallback for providers that don't implement it. * fix(forecast): block updates when config incomplete, match EP pattern Instead of deferring updates by 2 minutes when config is incomplete, now block them entirely by checking config completeness in _is_update_due(). This matches the electricity pricing (EP) module pattern exactly: - Provider selected (config incomplete) → no API call - User saves configuration → next update cycle triggers API call - After successful API call → schedule next update Key change: _is_update_due() returns False immediately if config is incomplete, preventing any API calls before required fields are saved. No artificial delays needed - the config check itself gates the updates. This is the correct solution: don't mask the problem with timeouts, just don't try to update until the configuration is actually complete. * debug(forecast): add debug logs to _is_update_due() * debug(forecast): change debug logs to INFO level * fix(forecast): use property for get state to avoid stale references Problem: self.get was storing a reference to forecast.get at init time. If the forecast.get object was recreated (e.g., provider removed/reset), self.get still pointed to the old instance, so it showed stale values. Solution: Make self.get a @Property that always returns the current instance from the data layer. This ensures we always read from the live state, even if the underlying object was recreated. This fixes the issue where next_query_time would be 0 in _is_update_due() even though it was just set to the next scheduled time. * fix(forecast): match EP pattern - accept get as parameter, use singleton OptionalData Problem: ConfigurableForecastProvider was creating new OptionalData() instances, which meant each forecast_module had a reference to a different forecast.get object. This caused next_query_time to be lost when the provider was re-initialized. Solution: Follow the exact same pattern as EP (ConfigurableTariff): 1. Accept 'get' as a parameter in ConfigurableForecast.__init__ (not a property) 2. Use the singleton data.data.optional_data.data.forecast.get in ConfigurableForecastProvider 3. This ensures all instances reference the same forecast.get object with persistent state This matches EP perfectly and solves the issue where next_query_time was reset to 0. * debug(forecast): add detailed debug log to update() to show state * fix(forecast): preserve next_query_time during provider re-initialization When modules are reloaded (e.g., during git updates), the provider is re-initialized with a new forecast.get object. This would reset next_query_time to 0, causing the scheduling to restart even though a valid scheduled time was already set. Now we preserve the old next_query_time (if > 0) during re-initialization to maintain scheduling continuity across module reloads. * fix(forecast): use @Property for get to always reference current singleton Instead of storing a stale reference to forecast.get in __init__, use a @Property that always returns the current singleton from data.data.optional_data.data.forecast.get. This prevents issues when MQTT updates modify the forecast.get object reference. Now each access to self.get returns the fresh singleton, ensuring next_query_time and other state values are always current. * fix(forecast): simplify singleton reference with @Property pattern Remove the preserve logic and the get parameter - use @Property instead. The @Property approach ensures self.get ALWAYS returns the current singleton from data.data.optional_data.data.forecast.get, preventing any stale references. This eliminates the 10-second update loop caused by forecast.get object references becoming stale when MQTT updates modify the data layer. * fix(forecast): point @Property to correct singleton instance The MQTT handlers update subdata.SubData.optional_data (global class variable), NOT data.data.optional_data (separate instance). The @Property was pointing to the wrong object, causing stale state. Now the @Property correctly returns: subdata.SubData.optional_data.data.forecast.get This is the ACTUAL singleton that MQTT messages update. * fix(forecast): initialize next_query_time on provider creation When a new forecast provider is created, initialize next_query_time to the next scheduled update time instead of leaving it at 0. This prevents immediate updates before the full configuration has been loaded from MQTT. Fixes the race condition where the first update would run with incomplete config (e.g., only 1 string instead of 4) because the update was triggered immediately when next_query_time=0, before all MQTT config messages had arrived. * refactor: consolidate forecast provider implementation - Remove redundant _log_forecast_solar_rate_limit() in Forecast.Solar (duplicate logging) - Add daily_kwh calculation to Open-Meteo provider for consistency across all providers - Standardize return type: all providers now return Tuple[Dict[Dict], Dict[Dict]] - Unify logging pattern: start + end logs with entry counts across all providers - Translate all docstrings and comments to German - Simplify store logic: remove _calculate_daily_kwh() (now done by providers) - Consistent implementation pattern for future providers * chore: translate remaining English docstrings and comments to German - Translate all docstrings in configurable_forecast.py to German - Translate all log messages from English to German - Translate internal comments explaining logic to German - Translate log.debug message in store/_forecast.py to German - Ensure consistency across all forecast modules * chore: fix flake8 style issues * chore: fix flake8 line length in loadvars.py * chore: add forecast topic to internal topics ACL test * chore: add forecast topic to valid_topic list * chore: add forecast topic to valid_topic list in correct alphabetical order * Derive daily_kwh from hourly values for forecastsolar The /estimate/watts endpoint never returns pre-aggregated daily totals, so today_kwh/tomorrow_kwh always showed 0.00 despite valid hourly data. Ports the same variable-timestep integration already used by openmeteo/forecast.py as a fallback. * review: general fixes * review: forecast solar * review: open meteo * review: pvnode * flake fixes * add forecast default topics * forecast: Zustand auch beim Anbieterwechsel zuruecksetzen, nicht nur bei "kein Modul" (analog electricity_pricing) --------- Co-authored-by: Lutz Bender <github@lutz-bender.de>
UI: openWB/openwb-ui-settings#1040
📋 Überblick
Implementiert ein PV-Prognose-Modul mit Unterstützung für 3 verschiedene Forecast-Provider. Weitere Verwendung über MQTT Topics für z.B. ECO-Laden, Zielladen, Speichersteuerung etc.
🎯 Features
Forecast.Solar: Cloud-API mit Anlagen-ID und API-Key
Open-Meteo: Kostenlose Meteorologie-API (Latitude/Longitude/Timezone)
PVNode: Community-Plattform mit Anlagen-ID und API-Key
Automatische Updates zu festen Zeiten: 5, 8, 11, 14, 17, 20 Uhr
Rate-Limit-Handling mit 15-Min Wiederholung bei HTTP 429
Konfiguration-Validierung (is_configuration_complete)
Umfassende Fehlerbehandlung mit Logging
MQTT-Integration für Zustandsmanagement
✅ Tests
@benderl ich bräuchte deine Hilfe mal bei durchsicht auf die ACLs/Security Themen, da hab ich ganz schön mit gekämpft um ehrlich zu sein. Besonders ohne die Änderung im UI in src/store/index.js bin ich nicht mehr in die settings nach dem Build gekommen, damit gings. Aber da bin ich mega unsicher was nun korrekt ist und was nicht.
Einbindung an eventuell Koala (wenn gewünscht) oder ans Colors (@cshagen) überlasse ich den Profis fürs UI Design, es hat mich schon ziemlich viel Zeit gekostet das Prognose Modul lauffähig zu bekommen. :) Aber es läuft und die Daten können natürlich auch für zukünftige Usecases verwendet werden, prognosebasierte Ladung etc :)
Ein paar UI Screenshots: