Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/strang-poa-pv-forecast.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
"ftw": minor
---

PV forecasts are now orientation-aware. When per-plane geometry is configured
(the Weather tab's PV arrays: tilt/azimuth/kWp), a radiation-bearing forecast
provider's global horizontal irradiance is projected onto each panel plane via
the physics `sunpos` model and summed, instead of the previous flat
`rated × (W/m² / 1000)` estimate that ignored panel orientation. Sites with no
arrays configured keep the existing behaviour, and providers that already return
site-calibrated watts (Forecast.Solar) are left untouched.

Providers that publish only global horizontal irradiance get an Erbs correlation
to split it into direct and diffuse components before projection, so a
south-facing 35° roof and a flat one no longer receive the same forecast.
46 changes: 46 additions & 0 deletions .changeset/strang-source-coverage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
"ftw": minor
---

SMHI STRÅNG becomes a first-class irradiance source, every external data source
now declares where in the world it works, and the location picker moves to
MapLibre GL JS.

- **STRÅNG as an irradiance source.** The client now covers the model's full
parameter set and knows its own domain. A nightly backfill scores measured
production against the DC energy the configured arrays should have produced
under that irradiance, exposed at `GET /api/pv/performance` and drawn as a
dashed "expected (STRÅNG)" overlay on the Produced tile. The resulting
performance ratio feeds back as a calibration factor on the forward forecast,
refused outright when it lands outside a plausible band — a site reading at
10% or 160% of nameplate is a configuration fault, and silently rescaling the
forecast would hide it.

- **Cloud cover, derived.** STRÅNG publishes no cloud-cover parameter; it is a
radiation model. It does publish sunshine duration (minutes per hour above the
WMO beam threshold), so cloudiness is recovered as `1 − minutes/60`. That is an
observed quantity rather than an inferred cloud field, but coarser: blind to
thin cirrus, and undefined at night. The API returns an explicit *unknown*
rather than defaulting to *clear*, because those lead to opposite decisions.

- **Coverage metadata.** New `GET /api/data-sources` reports every forecast,
irradiance and price source with its coverage area, country list, licence and
whether it reaches this specific site; the Weather tab renders it under the
map and flags sources that do not. This makes an existing silence explicit:
STRÅNG is Nordic-only and every price provider is European, so sites elsewhere
were getting empty results with no explanation. Bounds are advisory — a
rotated model grid means a lat/lon box can only be a superset — so `false` is
definitive and `true` means "worth trying". PV performance scoring now declines
to start outside the STRÅNG domain instead of retrying nightly forever.

- **MapLibre GL JS location picker.** The Weather tab's map is now MapLibre GL
JS 6 (BSD-3) instead of Leaflet, still lazy-loaded only when the tab opens. v6
is ESM-only and code-split, so it loads via a pinned dynamic import; the
stylesheet keeps its integrity hash, while the JS relies on version pinning
(an integrity hash on the entry point would not cover the shared chunk it
imports anyway). The style is built inline from the same OpenStreetMap raster
tiles as before, so neither the tile source nor the attribution changed. The
numeric latitude/longitude fields remain authoritative, so a CDN or WebGL
failure costs the picker and nothing else.

Nothing here touches the control tick, dispatch or the optimizer contract.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ metadata are the detailed reference.
- [Product roadmap](docs/roadmap.md)
- [Power sign convention](docs/site-convention.md)
- [Safety invariants](docs/safety.md)
- [Geographic coverage of external data](docs/data-coverage.md)
- [Operations and recovery](docs/operations.md)
- [Full backup and safe restore](docs/backup-and-restore.md)
- [Writing a driver](docs/writing-a-driver.md)
Expand Down
154 changes: 154 additions & 0 deletions docs/data-coverage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Geographic coverage of external data sources

FTW controls hardware anywhere, but it depends on external data for three
things: **spot prices**, **weather/PV forecasts** and **PV performance
scoring**. Those three have very different geographic reach, and the difference
decides how much of FTW is useful at a given site.

Short version:

- **Weather and PV forecasting works worldwide.**
- **Price-driven planning works in Europe only.**
- **PV performance scoring works in the Nordic region only.**
- **Roof geometry (planned) is Sweden only.**

A site outside Europe can still run FTW for monitoring, safety and control — but
the economic optimisation that motivates most of the planner has no price source
to work from.

`GET /api/data-sources` answers this per site: it returns every source with its
coverage area and, when the site location is known, whether that source reaches
it. The Weather settings tab renders the same data under the map. This file is
the prose; `go/internal/coverage` is the machine-readable source of truth, and
the two are meant to stay in step.

> **Coverage bounds are advisory.** Each bounded source declares a lat/lon box,
> but STRÅNG's model grid is rotated relative to lat/lon, so its box is a
> *superset* of the real domain — points near a corner pass the box test and
> still return nothing. Treat `covers: false` as definitive and `covers: true`
> as "worth trying". The upstream API is always the final word.

## Spot prices — Europe only

Configured under `price.provider`.

| Provider | Coverage | API key | Notes |
|---|---|---|---|
| `sourceful` | European day-ahead markets | No | Default. Sourceful's cached ENTSO-E API. |
| `elprisetjustnu` | **Sweden only** — zones SE1–SE4 | No | 15-minute PTU since late 2025. |
| `entsoe` | ENTSO-E member markets (most of Europe) | Yes | Direct from the Transparency Platform. |
| `none` | — | — | Disables price fetching entirely. |

There is **no provider for any market outside Europe**. North America (CAISO,
ERCOT, PJM, ISO-NE, NYISO, MISO, SPP, AESO, IESO), Australia (AEMO/NEM), Japan
(JEPX) and everywhere else are unsupported, and there is no manual or
fixed-tariff provider to stand in for them.

Two further Europe-centric assumptions live in the price layer: prices are
stored internally in **öre** (1 SEK = 100 öre), and ENTSO-E's EUR/MWh figures
are converted using **ECB** daily FX rates.

> The Tibber driver (`drivers/tibber.lua`) is telemetry only — it reports meter
> readings, not prices, so it is not a fourth price source.

## Weather and PV forecasts — worldwide

Configured under `weather.provider`. All four work at any latitude/longitude.

| Provider | Coverage | API key | Signal quality |
|---|---|---|---|
| `met_no` | Global | No | Cloud cover only — weakest PV signal. |
| `openweather` | Global | Yes | Cloud cover only. |
| `open_meteo` | Global | No | Shortwave radiation (GHI) — good. |
| `forecast_solar` | Global | No (free tier) | Site-calibrated watts from panel geometry — best. |

Accuracy varies by region because the underlying numerical weather models do,
but none of these are geographically gated. Outside the Nordics, prefer
`open_meteo` or `forecast_solar`: they carry an irradiance signal, which is what
the orientation-aware plane-of-array model needs.

## PV performance scoring — Nordic region only

The scorer (`GET /api/pv/performance`) compares measured production against a
physics baseline built from **SMHI STRÅNG** irradiance. STRÅNG is a mesoscale
analysis product covering the **Nordic region** hourly at ~2.5 km from 1999 to
roughly one day ago. It is free, keyless and CC BY 4.0.

Outside that domain STRÅNG returns no data, so scoring simply never produces
rows and the dashboard overlay stays hidden. Nothing fails loudly; the feature
is just unavailable.

Because the same scoring feeds the **forecast calibration factor**, sites
outside the STRÅNG domain also do not get measured calibration of their PV
forecast — they fall back to the uncalibrated physics estimate.

STRÅNG has **no forward horizon**. It is never used as a forecast provider; see
[architecture.md](architecture.md) for where it sits.

### What STRÅNG actually publishes

Probing the live API on 2026-07-31 (SMHI's own apidocs pages currently 404)
returned data for exactly seven parameters and 404 for everything else. Names
were confirmed from their magnitudes on a clear day rather than from docs:

| Code | Quantity | Unit | Noon value, Stockholm 2026-06-21 |
|---|---|---|---|
| 116 | CIE-weighted UV irradiance | mW/m² | 146.6 |
| 117 | **Global horizontal (GHI)** | W/m² | 810.5 |
| 118 | Direct normal (DNI) | W/m² | 917.2 |
| 119 | **Sunshine duration** | min/h | 60.0 |
| 120 | Photosynthetically active radiation | W/m² | 357.9 |
| 121 | Direct horizontal | W/m² | 723.0 |
| 122 | **Diffuse horizontal (DHI)** | W/m² | 87.5 |

Two checks confirm the identification: 121 + 122 = 723.0 + 87.5 = 810.5, exactly
parameter 117 (direct + diffuse = global), and 119 caps at exactly 60, i.e.
minutes within the hour.

**STRÅNG publishes no cloud cover.** It is a radiation model; cloudiness is not
among its outputs. It is however *derivable*: parameter 119 counts the minutes
in each hour during which direct beam irradiance exceeded the WMO sunshine
threshold, so `1 − minutes/60` is the fraction of the hour the sun spent
obscured. That is an observed quantity rather than an inferred cloud field, but
it is coarser than a forecast provider's cloud percentage — it cannot see thin
cirrus that dims without blocking. FTW exposes it via
`strang.IrradianceHour.CloudCover(lat, lon)`, which returns an explicit
"unknown" rather than defaulting to "clear".

The location argument is not decoration. Sunshine duration is zero at night for
the trivial reason that there is no sun, and zero again near sunrise and sunset
because the beam crosses ten or more air masses and cannot reach the 120 W/m²
threshold even under a spotless sky. Both would read as "100% overcast" if taken
at face value. `CloudCover` therefore declines to answer unless the sun clears
**5° of elevation** at some point in the hour, sampling the hour's start,
midpoint and end so the hour in which the sun crosses that line is still counted.

Live data from 2026-06-21 at Stockholm shows the distinction:

| Hour (UTC) | GHI W/m² | Sunshine | Cloud cover |
|---|---|---|---|
| 00:00 | 0.0 | 0 min | *unknown* — sun below horizon |
| 04:00 | 163.2 | 60 min | 0% |
| 12:00 | 810.5 | 60 min | 0% |
| 20:00 | 2.5 | 0 min | *unknown* — sun minutes from setting |

## Roof geometry — Sweden only (planned)

The roof-derivation module proposed in
[RFC #717](https://github.com/srcfl/ftw/discussions/717) reads **Lantmäteriet**
building footprints and LiDAR, which exist for **Sweden only** and require a
Geotorget account. Everywhere else, panel tilt/azimuth/kWp stays a manual entry
in the Weather settings tab — which is the fallback by design, not a
degraded mode.

## What a non-European site loses

| Capability | Works outside Europe? |
|---|---|
| Device control, safety, dispatch | Yes |
| Telemetry, history, dashboard | Yes |
| Weather + PV forecasting | Yes |
| Self-learning PV twin | Yes |
| Price-driven planning / optimisation | **No** — no price source |
| PV performance scoring + calibration | **No** — outside STRÅNG's domain |
| Automatic roof geometry | **No** — Sweden only |
22 changes: 22 additions & 0 deletions go/cmd/ftw/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ import (
"github.com/srcfl/ftw/go/internal/prices"
"github.com/srcfl/ftw/go/internal/proxy"
"github.com/srcfl/ftw/go/internal/pvmodel"
"github.com/srcfl/ftw/go/internal/pvperf"
"github.com/srcfl/ftw/go/internal/selftune"
"github.com/srcfl/ftw/go/internal/selfupdate"
"github.com/srcfl/ftw/go/internal/state"
Expand Down Expand Up @@ -992,6 +993,26 @@ func main() {
"lat", forecastSvc.Lat, "lon", forecastSvc.Lon, "rated_pv_w", ratedPVW)
}

// ---- Start PV performance scoring (optional) ----
// Nightly backfill of SMHI STRÅNG historical irradiance + expected-vs-actual
// PV scoring. Nil when the site has no PV geometry to score against. This is
// read-only with respect to control: it only fetches weather data and writes
// the irradiance_history + pv_performance_daily tables.
pvPerfSvc := pvperf.FromConfig(cfg.Weather, ratedPVW, st,
"ftw/"+Version+" github.com/srcfl/ftw")
if pvPerfSvc != nil {
pvPerfSvc.Start(ctx)
defer pvPerfSvc.Stop()
// Close the loop: measured performance calibrates the forward
// forecast. The hook is read at fetch time, so it starts correcting
// as soon as enough days are scored — no restart needed.
if forecastSvc != nil {
forecastSvc.Calibration = pvPerfSvc.CalibrationFactor
}
slog.Info("pv performance scoring started",
"lat", pvPerfSvc.Lat, "lon", pvPerfSvc.Lon, "arrays", len(pvPerfSvc.Arrays))
}

// ---- Start PV digital twin (optional, requires weather config) ----
// pvSvc is pre-declared above so the reload Applier can update it.
if cfg.Weather != nil && cfg.Weather.Provider != "" && cfg.Weather.Provider != "none" {
Expand Down Expand Up @@ -2081,6 +2102,7 @@ func main() {
SnapshotDir: filepath.Join(filepath.Dir(statePath), "snapshots"),
Prices: priceSvc,
Forecast: forecastSvc,
PVPerf: pvPerfSvc,
MPC: mpcSvc,
PVModel: pvSvc,
LoadModel: loadSvc,
Expand Down
Loading