When processing the arielle at Neumayer data, @ulysses78 and me realized that the cloud screening has some issues in the case of saturation.
Example for 24 Jan 2023:

However, the 532 and 1064 total signals are saturated as correctly indicated by data_cube.flagSaturation (slightly decreased data_cube.polly_config_dict['saturate_thresh'] = 80)
In fact the saturation flag is not considered in profilesegment.segment at all
flagValPrf = data_cube.flagCloudFree & (~data_cube.retrievals_highres['depCalMask']) \
& (~data_cube.retrievals_highres['shutterOnMask']) & (~data_cube.retrievals_highres['fogMask'])
A quick monkey patch to the fog mask for testing resasons gave promising results (not saying that we should go for that flag in the operational setup):
original_fog_mask = data_cube.retrievals_highres['fogMask'].copy()
mask_saturation = np.sum(
np.squeeze(data_cube.flagSaturation[:, :, data_cube.gf(1064, 'total', 'FR')]),
axis=1)
data_cube.retrievals_highres['fogMask'] = original_fog_mask | (mask_saturation > 5)
data_cube.cloudScreen()
data_cube.cloudFreeSeg()
....
Before we consider changes, the proposed changes for #76 and branch 76-cloudy_aggregation, especially the newly introduced genFlagValPrf might be a good option.
Maybe it's good to have a flexible approach where the saturation flag depending on campaign and system (or usecase)? Any opinions? @HavardStridBuholdt @HolgerPollyNet
When processing the arielle at Neumayer data, @ulysses78 and me realized that the cloud screening has some issues in the case of saturation.
Example for 24 Jan 2023:

However, the 532 and 1064 total signals are saturated as correctly indicated by
data_cube.flagSaturation(slightly decreaseddata_cube.polly_config_dict['saturate_thresh'] = 80)In fact the saturation flag is not considered in
profilesegment.segmentat allA quick monkey patch to the fog mask for testing resasons gave promising results (not saying that we should go for that flag in the operational setup):
Before we consider changes, the proposed changes for #76 and branch 76-cloudy_aggregation, especially the newly introduced genFlagValPrf might be a good option.
Maybe it's good to have a flexible approach where the saturation flag depending on campaign and system (or usecase)? Any opinions? @HavardStridBuholdt @HolgerPollyNet