Forecast capping if calibrate forecast is too far from the uncalibrated by Kat-90 · Pull Request #2449 · metoppv/improver

Kat-90 · 2026-09-04T20:23:16Z

Addresses #GitHubissuenum

This change adds a forecast cap, if set, that caps the calibrated forecast to an amount away from the original uncalibrated forecast. It acts as a risk mitigation for erroneous observations e.g. Fiji in the global calibration.
This is an optional argument that doesn't need to affect calibration of other diagnostics unless set in the suite.

Testing:

Ran tests and they passed OK
Added new tests for the new feature(s)

CLA

If a new developer, signed up to CLA

uncalibrated. Also adds a test.

…ons, so that these values can be used as bounds to clip the calibrated forecast.

gavinevans

Thanks @Kat-90 👍

I've added a comment, as I don't think the original forecast that you're trying to compare to, is quite right, in terms of identifying the original forecast that we want to compare to. I've therefore created this PR (Kat-90#25) to your PR to identify the min and max within the original forecast and use these as the lower and bound when clipping the calibrated forecast.

gavinevans · 2026-09-15T10:59:11Z

+            original_forecast = forecast_df.loc[
+                forecast_df[representation_name]
+                == forecast_df[representation_name].iloc[0],
+                self.target_name,
+            ].values


There's an assumption in this pandas selection that we only want to use the zeroth item in forecast_df[representation_name] as the original forecast. This could be the 5th percentile, for example, or a single realization, and we don't really want to be clipping based on a single realization or the 5th percentile. I've therefore created a PR to this PR (Kat-90#25) with the aim of finding the min and max across realization or percentile, and using these values as the lower and upper bounds for clipping.

Thanks Gavin, I've reviewed and merged in your changes.

Correction to "Forecast capping if calibrate forecast is too far from the uncalibrated"

mo-kbogue

Limited experience on the actual application of QRF in the suite but from what I can see it looks good. Just a few notes on possibly rewording some comments, and on the testing,

mo-kbogue · 2026-09-16T10:24:33Z

                provided, the forecast period found in the first forecast cube
                will be used.
+            max_allowed_difference (float, optional):
+                The maximum allowed difference for the forecast calibration. If not


minor note but I'd prefer the below;

Suggested change

The maximum allowed difference for the forecast calibration. If not

The maximum allowed difference between the uncalibrated and calibrated forecast. If not

mo-kbogue · 2026-09-16T10:25:45Z

            unique_site_id_keys: The names of the coordinates that uniquely identify
                each site, e.g. "wmo_id" or ["latitude", "longitude"].
+            max_allowed_difference (float, optional):
+                The maximum allowed difference for the forecast calibration. If not


Same as above, personally I'd go for;

Suggested change

The maximum allowed difference for the forecast calibration. If not

The maximum allowed difference between the uncalibrated and calibrated forecast. If not

mo-kbogue · 2026-09-16T10:30:28Z

            provided, the first forecast period found in the forecast cube
            will be used.
+        max_allowed_difference (float, optional):
+            The maximum allowed difference for the forecast calibration. If not


Same again

Suggested change

The maximum allowed difference for the forecast calibration. If not

The maximum allowed difference between the uncalibrated and calibrated forecast. If not

mo-kbogue · 2026-09-16T11:00:57Z

+            [0.5],
+            np.array([6], dtype=np.float32),
+            np.array([17.0, 0.0], dtype=np.float32),
+            5.0,
+            np.array([11.0, 3.0], dtype=np.float32),


If I'm reading this correctly, we're inputting one single uncalibrated forecast value, 6, but our fake QRF calibrated forecast returns two values, 17 and 0? This test would maybe make more sense to me if the fake QRF calibrated forecast was 17 (rather than 17 and 0) and the expected output was 11 (rather than 11 and 3). Or equivalently, if the input forecast was [6.0, 6.0].
Some extra comments here might also just be useful, even if only for this first example, e.g the below;

Suggested change

[0.5],

np.array([6], dtype=np.float32),

np.array([17.0, 0.0], dtype=np.float32),

5.0,

np.array([11.0, 3.0], dtype=np.float32),

[0.5], #50th percentile, i.e median

np.array([6.0], [6.0], dtype=np.float32), #input uncalibrated forecast

np.array([17.0, 0.0], dtype=np.float32), #qrf calibrated forecast

5.0, #maximum difference

np.array([11.0, 3.0], dtype=np.float32), #expected output

mo-kbogue · 2026-09-16T11:02:45Z

+        ),
+        (
+            [0.1, 0.5, 0.9],
+            np.array([6, 12, 18], dtype=np.float32),


As in the previous point, the uncalibrated and calibrated examples here not being the same length threw me off - might be better if they have the same format

mo-kbogue · 2026-09-16T11:02:53Z

+        ),
+        (
+            [0.1, 0.5, 0.9],
+            np.array([6, 12, 18], dtype=np.float32),


Same as above

mo-kbogue · 2026-09-16T11:04:59Z

+            # 18 + 5 = 23 is the upper bound for the first site, while 6 - 5 = 3 is
+            # the lower bound for the second site. Values beyond these limits are
+            # clipped, while points already inside the allowed range remain unchanged.
+            np.array([[23.0, 23.0, 22.0], [3.0, 3.0, 9.0]], dtype=np.float32),


If we're inputting the 10th, 50th and 90th percentile as specified above, would we not expect the result to be in ascending order? Maybe that goes beyond the scope of this PR? Could be worth including as a warning perhaps

mo-kbogue

Thanks for the changes made - conversation offline helped me understand how this code works.

mo-kbogue

Thanks for the changes made - conversation offline helped me understand how this code works.

…sion * upstream/master: Spatial morphing (metoppv#2441) Stochastic noise extensions (metoppv#2442) Forecast capping if calibrate forecast is too far from the uncalibrated (metoppv#2449) Vicinity metadata from the threshold plugin (metoppv#2439) Update cluster sources bookkeeping, so that forecast periods are only stored in one forecast source. (metoppv#2437) Correct realization numbering bug in temporal interpolation (metoppv#2436) Modify precedence ordering when clustering (metoppv#2435)

* master: Spatial Morphing - refinement of Google FILM output (#2451) Spatial morphing (#2441) Stochastic noise extensions (#2442) Forecast capping if calibrate forecast is too far from the uncalibrated (#2449) Vicinity metadata from the threshold plugin (#2439) Update cluster sources bookkeeping, so that forecast periods are only stored in one forecast source. (#2437)

Kat-90 self-assigned this Sep 4, 2026

Kat-90 added the Type:Enhancement label Sep 4, 2026

Adds forecast capping if calibrated forecast is too far from the

54ea772

uncalibrated. Also adds a test.

Kat-90 force-pushed the forecast_capping branch from 322e157 to 54ea772 Compare September 15, 2026 09:36

Modifications to find the min and max across percentiles or realizati…

54795e6

…ons, so that these values can be used as bounds to clip the calibrated forecast.

gavinevans mentioned this pull request Sep 15, 2026

Correction to "Forecast capping if calibrate forecast is too far from the uncalibrated" Kat-90/improver#25

Merged

2 tasks done

gavinevans requested changes Sep 15, 2026

View reviewed changes

Merge pull request #25 from gavinevans/forecast_capping2

9ec4d2f

Correction to "Forecast capping if calibrate forecast is too far from the uncalibrated"

gavinevans previously approved these changes Sep 15, 2026

View reviewed changes

Adding my user to the mailmap file

184d885

Kat-90 dismissed gavinevans’s stale review via 184d885 September 16, 2026 07:31

Adding user to CONTRIBUTING.md file

05b3299

gavinevans previously approved these changes Sep 16, 2026

View reviewed changes

mo-kbogue requested changes Sep 16, 2026

View reviewed changes

Changes as per review comments.

bda819f

Kat-90 dismissed gavinevans’s stale review via bda819f September 16, 2026 18:37

Katharine Hurst added 3 commits September 16, 2026 19:49

CI checks

d9c2a9d

Updated comments to make it more readable.

9e6543a

CI check change

fc753ee

mo-kbogue previously approved these changes Sep 17, 2026

View reviewed changes

Kat-90 dismissed mo-kbogue’s stale review via 97a86e2 September 17, 2026 12:08

Kat-90 force-pushed the forecast_capping branch from fa97707 to fc753ee Compare September 17, 2026 12:15

Merge branch 'master' into forecast_capping

e83c581

mo-kbogue reviewed Sep 17, 2026

View reviewed changes

mo-kbogue approved these changes Sep 17, 2026

View reviewed changes

Kat-90 merged commit e68a8d4 into metoppv:master Sep 17, 2026
7 checks passed

	The maximum allowed difference for the forecast calibration. If not
	The maximum allowed difference between the uncalibrated and calibrated forecast. If not

-            [0.5],
-            np.array([6], dtype=np.float32),
-            np.array([17.0, 0.0], dtype=np.float32),
-.0,
-            np.array([11.0, 3.0], dtype=np.float32),
+            [0.5], #50th percentile, i.e median
+            np.array([6.0], [6.0], dtype=np.float32), #input uncalibrated forecast
+            np.array([17.0, 0.0], dtype=np.float32), #qrf calibrated forecast
+.0, #maximum difference
+            np.array([11.0, 3.0], dtype=np.float32), #expected output

Conversation

Kat-90 commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gavinevans left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mo-kbogue left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mo-kbogue left a comment

Choose a reason for hiding this comment

Uh oh!

mo-kbogue left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Kat-90 commented Sep 4, 2026 •

edited

Loading