feat(roofmodel): optional shadow-aware irradiance via vostok - #736
Draft
HuggeK wants to merge 6 commits into
Draft
feat(roofmodel): optional shadow-aware irradiance via vostok#736HuggeK wants to merge 6 commits into
HuggeK wants to merge 6 commits into
Conversation
Radiation-bearing providers previously produced one flat estimate, rated x (W/m2 / 1000), which ignored panel orientation entirely: a south-facing 35 deg roof and a flat one got the same forecast. When per-plane geometry is configured (Weather tab: tilt/azimuth/kWp), global horizontal irradiance is now projected onto each plane via the existing sunpos physics and summed. Providers publishing only GHI get an Erbs correlation to split direct from diffuse first. Sites with no arrays keep the previous behaviour, and Forecast.Solar - which already returns site-calibrated watts - is deliberately left untouched so its numbers are not scaled twice. Split out of the original srcfl#718 per review: this half is user-visible and improves every radiation-bearing provider today. The STRANG client and performance scoring that shared that branch move to a follow-up PR. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
…coverage STRANG becomes a real irradiance source rather than an unreferenced client, every external data source now says where in the world it works, and the location picker moves off Leaflet. STRANG's parameter set was mapped against the live API because SMHI's apidocs pages 404: exactly 116-122 exist. Identification was confirmed by physics rather than by guessing - at solar noon 121 + 122 = 723.0 + 87.5 = 810.5, exactly parameter 117, and 119 caps at 60, i.e. minutes within the hour. STRANG publishes no cloud cover; it is a radiation model. Cloudiness is instead derived from sunshine duration as 1 - minutes/60, which is observed rather than inferred but coarser: blind to thin cirrus, undefined at night. CloudCover() therefore returns an explicit unknown instead of defaulting to clear, because those two lead to opposite decisions. The new coverage registry makes an existing silence explicit. STRANG is Nordic-only and every price provider is European, so sites elsewhere were getting empty results with no explanation (srcfl#726). GET /api/data-sources now reports area, countries, licence and whether each source reaches this site, and the Weather tab renders it under the map. Bounds are advisory: STRANG's grid is rotated, so a lat/lon box can only ever be a superset - all four in-box corners were probed and returned no data. False is definitive, true means worth trying. Scoring now declines to start outside the domain instead of retrying nightly forever. Stacked on srcfl#718, which carries the plane-of-array wiring this builds on. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
A Swedish site can stop typing panel angles in by hand. The new roofmodel/ module reads Lantmateriet LiDAR around the site and recovers each roof face's tilt, azimuth and usable area, which pre-fill weather.pv_arrays. Structured as a separate Python package alongside optimizer/, reached at arm's length: core spawns it, passes coordinates and the operator's own Geotorget credentials, and reads one versioned roof_model.json from stdout. Segmentation drags in a compiled point-cloud stack and runs for minutes, so a time-boxed subprocess keeps it off the control tick and lets it be absent entirely - the normal case, since the data exists only for Sweden. The pipeline follows the SPAN method: iterative RANSAC pulls one surface at a time out of the cloud, then DBSCAN splits faces sharing a plane equation but not a location, because two wings of a building fit the same plane and are not the same roof. Method only - no code is taken from SPAN's GPL QGIS plugin, and the dependencies (numpy, scikit-learn, requests) are all BSD. SWEREF 99 TM is implemented directly rather than via pyproj: it is one projection with fixed parameters, and a full PROJ build is disproportionate for that. Verified against the projection's exact analytic properties - easting on the central meridian is exactly 500000 at every latitude - and by sub-millimetre round trips from Smygehuk to Treriksroeset. Applying derived arrays to config is deliberately a separate act. Derivation is a best guess from a point cloud that may be years old, and silently rewriting an operator's panel config is a change they should make knowingly. Also generalises the __pycache__ ignore rule, which was optimizer-specific and would otherwise need repeating per module. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
The roofmodel tests passed on Windows and failed on Linux, which is where CI
runs them. Two separate bugs, both caused by stubbing the module with a shell
script:
- The stub handed back "sh" as the command, and the service invokes it as
`<cmd> -m ftw_roofmodel ...`. dash reads that as "run the script named
ftw_roofmodel", cannot open it, and exits 2. The stub never ran at all, so
five tests failed against an error that had nothing to do with what they
were asserting.
- The module-error stub echoed an unquoted JSON document. dash strips the
double quotes, so `{"error":"..."}` reached the parser as `{error:...}` and
could not be unmarshalled. cmd.exe keeps them, which is the only reason
that test ever passed.
The stub is now this test binary re-executed with an environment variable
telling it which behaviour to play. Nothing goes through a shell, so there is
no quoting to get wrong and no per-platform behaviour to diverge. The timeout
test consequently runs everywhere instead of being skipped on Windows.
Two assertions the shell stub could not make: that the site, credentials and
radius actually survive the process boundary, and that PYTHONPATH is set --
without it the module is only importable if installed system-wide, which on a
Pi it is not. Also that --vostok stays absent unless configured, so a GPL tool
is never invoked by default.
The Python module's tests did not run in CI at all: the workflow runs
`pytest -q optimizer/tests` and nothing else, so 75 tests covering the plane
fitting every derived tilt and azimuth depends on were never executed upstream.
Added a roofmodel job mirroring the optimizer's, wired into the required-check
gate so a failure blocks rather than being reported and ignored. It installs
without the `geo` extra on purpose: LAZ decoding pulls a compiled backend, and
everything except the point-cloud read is exercised without it.
Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
HuggeK
force-pushed
the
vostok-shading
branch
2 times, most recently
from
July 31, 2026 13:41
8e07ecb to
2f890d6
Compare
Settings -> Weather gains a Geotorget credential form and a building picker. Press "Find buildings here" and the footprints near the marker are drawn on the map and listed beside it; click yours, press "Read roof from LiDAR", and the PV arrays fill in with one entry per usable roof face. The form is filled but nothing is saved. The operator sees the numbers, corrects what is wrong and presses Save. FTW does not rewrite a panel configuration on its own: the derivation is a guess from a scan that may be years old, and only the operator knows whether that face has panels on it. Picking a building is not cosmetic. Without a footprint the module segments whatever stands inside its search radius, and the plane fitting is global -- a fitted plane is infinite, so a roof at azimuth 180 is z = f(y) with no x term and extends across the whole tile. A second building sharing that ridge orientation lands inside its inlier band however far away it is, and the two lose returns to each other. Measured on a synthetic pair: a detached garage recovered 93% of its true area and split into two fragments while coplanar with the house, against 100% and one clean face once clipped to its own footprint. The clip buffers the outline by a metre first, because roofs overhang their walls and the eaves carry the lowest returns. Frames are detected rather than assumed. GeoJSON mandates WGS84 but Lantmateriet publishes this catalogue in SWEREF 99 TM and its STAC search takes a SWEREF bbox, so ring coordinates are classified by magnitude -- six- and seven-figure numbers are projected metres, degrees never are. New GET /api/roofmodel/buildings lists footprints as GeoJSON, honouring an explicit lat/lon so the picker can search where the marker is rather than where the last save put it. POST /api/roofmodel/derive accepts a building_id. GET /api/roofmodel reports has_credentials so the UI can stop asking. The Geotorget token now masks and restores like every other secret. It never appears in an API response, and saving an unrelated setting no longer wipes it -- the settings form returns the blank it was given, which without PreserveMaskedSecrets would have deleted the stored credential. docs/roof-geometry.md covers ordering the two Geotorget products, what the derived kWp does and does not mean (an upper bound on what fits, not what is installed), and what each failure message is telling you. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
Roof segmentation gives each face a tilt and an azimuth, which predicts yield for an unobstructed roof and says nothing about the spruce to the south. Those can cost a real installation far more than a few degrees of azimuth ever will. vostok computes per-point solar potential against voxelised occlusion geometry, so pointing it at the same LiDAR tile the roof came from supplies exactly the missing number. Each face is run twice - once against the real geometry, once with shadowing off - and the ratio becomes the shading factor. Taking a ratio rather than an absolute makes the result independent of vostok's units, sky model and chosen year, since all three cancel. vostok is GPL-3.0 and FTW is not, so it is invoked at arm's length: a separate process talking through files and a command line, never linked, no code copied either way. Two rules keep that boundary intact and neither is negotiable - it is never bundled or redistributed with FTW, and FTW never installs it. The operator installs it and sets roofmodel.vostok_binary. Unset or missing, shading is not evaluated and shading_factor is absent rather than 1.0. "We did not look" and "we looked and it is unobstructed" are different claims, and only the second justifies trusting the yield estimate. A missing or failing binary never fails a derive; the geometry is still good without a shading number. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> Signed-off-by: Hugo Karlsson <48095810+HuggeK@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gives each derived roof face a shading factor from neighbouring buildings and
trees.
Segmentation gives a face its tilt and azimuth, which predicts yield for an
unobstructed roof and says nothing about the spruce to the south — and that can
cost a real installation far more than a few degrees of azimuth ever will.
vostok ("Voxel Octree Solar
Toolkit", 3DGeo Heidelberg) computes per-point solar potential against voxelised
occlusion geometry. Pointing it at the same LiDAR tile the roof was segmented
from supplies exactly the missing number: the trees and neighbouring roofs that
cast the shadows are already in the data.
The licensing boundary, first
vostok is GPL-3.0. FTW is not. It is therefore invoked at arm's length: a
separate process communicating through files and a command line, never linked,
with no code copied in either direction. Under the FSF's own mere-aggregation
reading that leaves FTW's licence unaffected.
Two rules keep it that way, and the module documents both as non-negotiable:
it. Adding it to an image, an installer or a dependency list would turn
aggregation into distribution.
manager invocation.
I'd suggest a maintainer sanity-checks that reading before this ships. It is the
standard interpretation, not legal advice.
How the factor is computed
Each face is run twice — once against the real occlusion geometry, once with
shadowing off — and the factor is the ratio.
That is deliberate. An absolute Wh/m²/day figure would depend on vostok's sky
model, its units and the year chosen; a ratio cancels all three, so the number
means "fraction of open-sky irradiation this face actually receives" regardless
of how the tool is configured. It costs two runs per face, which is affordable
because this is onboarding-time work, not runtime.
The distinction between "not evaluated" and "unobstructed"
When vostok is absent,
shading_factoris omitted, not set to 1.0.Those are different claims. 1.0 says we looked and the roof is clear, which
justifies trusting the yield estimate at face value. Absent says we did not look.
Collapsing them would silently upgrade an unmeasured roof to a verified one — and
since vostok is usually absent, that would be the common case rather than the
edge case.
The same reasoning applies to failures: a non-zero exit, a missing output file or
a timeout all leave the model unevaluated with a stated reason, and never fail
the derive. Roof geometry is perfectly good without a shading number.
Testing
15 shading tests, driving a stub that implements vostok's documented contract
— read a
.sol, write the query points back with an appended Wh/m²/day column.Covered: the ratio (600 against 1000 → 0.6), unobstructed → 1.0, fully shaded →
0.0, clamping when a ratio exceeds 1 (shadowing can only remove irradiance, so
a value above 1 means the runs are not comparable), non-zero exit, missing output
file, a plane with too few points being skipped rather than fatal, and every
absence path.
Two tests assert the
.solcontract itself: that both runs are emitted withshadowingtrue and false, that the site location and voxel size are carriedthrough, and that every query point has a normal — vostok refuses without one.
One test closes the loop: the normal handed to vostok is fed back through the
segmenter's
_tilt_azimuthand must return the original tilt and azimuth.Without it a sign error would silently compute shading for a different surface
than the one being derated.
Not covered: a real vostok binary. It is GPL C++ requiring a local build, and
FTW's rule is that it is never auto-installed — so the adapter is written to the
documented interface and exercised against a stub. The
.solkey names should bechecked against your build before this is relied on in the field; that is the one
part I could not verify against the real tool.
Part of the roadmap in discussion #717.