Adding LensShadingCorrection maps and ToneCurve to controls metadata by kunzmi · Pull Request #356 · raspberrypi/libcamera

kunzmi · 2026-08-18T17:09:17Z

Having the LensShadingCorrection maps and ToneCurves available in frame metadata allows creating a DNG file with all necessary information so that the DNG matches the JPEG image without colour casts.

(This is a necessary change for another PR in rpicam-apps)

naushir · 2026-08-19T06:47:40Z

I think this is a great addition to have! Given that this does introduce new controls to the core libcamera code, would you be able to post your patch onto the libcamera-devel mailing list please? The core libcamera team can then review your proposal and once merged, we can backport it to the Raspberry Pi libcamera source tree.

kunzmi · 2026-08-19T10:20:01Z

Sure, patch is sent - waiting for approval as I didn't register to the mailing list...

davidplowman · 2026-08-20T08:45:36Z

Hi, thanks very much for doing this. I'd like to see this merged. Obviously it will need to go through the mailing list review process, which always involves a bit to to-ing and fro-ing, but I'll start by adding a few comments here on what I'd like to see. But as I said, thanks very much for doing this, I'll be very happy to submit support to PiDNG for this once the data formats and names in libcamera are settled.

kbingham · 2026-08-20T08:52:57Z

Sure, patch is sent - waiting for approval as I didn't register to the mailing list...

Sorry - both me and Laurent are the moderators but we've both been on holiday at the same time, I'm back today - so I've just let this through.

davidplowman

As I said, thanks very much and I would very much like to see this merged to mainline. Am happy to take care of the PiDNG end of things myself once this is in.

davidplowman · 2026-08-20T08:46:22Z

+      description: |
+        A map giving the lens shading correction factors for blue channel.
+      size: [n]
+


My personal preference is probably for a single metadata control for this, maybe LensShadingCorrectionTables, or LensShadingTables if the former is too long!

I agree that one single table would makes things less verbose. Question on this subject: How are monochrome camereas handled (I don't have any to test...)? From what I've seen they just fill RGB with the same values but maintain the RGB channels. Is this correct?

davidplowman · 2026-08-20T08:46:57Z

+
+  - LensShadingCorrectionMapBlue:
+      type: float
+      direction: inout


I think everything here needs to be out rather than inout if we haven't implementing setting the control, rather than just finding out what was used.

davidplowman · 2026-08-20T08:48:12Z

+      direction: inout
+      description: |
+        The size of the lens shading correction maps in pixels.
+      size: [2]


With one table this would go up to 3 items. Probably we should store the gains in planar form rather than interleaved, as I think that matches what's in the DNG files. So for Pi 5, I guess this would report [3, 32, 32].

DNG is actually interleaved, so [32, 32, 3]

davidplowman · 2026-08-20T08:49:49Z

+			iterator++;
+			*iterator = static_cast<float>(y);
+			iterator++;
+		});


The iterator feels a bit verbose here. Could we leave it out and do

contrast->push_back(x); contrast->push_back(y);

instead?

davidplowman · 2026-08-20T08:55:37Z

-		{ &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }
-	} },
+const std::map<const std::string, ControlInfoMap::Map> platformControls{
+	{ "pisp", { { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) } } },


I don't super-love the formatting that it forces on you here. Could we do something like

const ControlInfoMap::Map pispControls = { { { &controls::rpi::ScalerCrops, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) } } }; const std::map<const std::string, ControlInfoMap::Map> platformControls{ { "pisp", pispControls }, };

It looks slightly less hideous, and would be extensible if we had vc4-only controls.

clang-format was changing a few more things that I reverted before committing, it seems I missed this one :)

davidplowman · 2026-08-20T08:57:55Z

+			g[i] = static_cast<float>(alscStatus->g[i]);
+			b[i] = static_cast<float>(alscStatus->b[i]);
+		}
+		uint32_t sizeTable[] = { alscStatus->cols, alscStatus->rows };


Wondering if we could call std::copy three times instead?

I was also wondering if we should write the table as doubles, too? Is there some specific reason for the higher precision or are floats just fine?

davidplowman · 2026-08-20T09:00:08Z

+        The size of the lens shading correction maps in pixels.
+      size: [2]
+
+  - ToneCurve:


ToneCurve? GammaCurve? I expect folks will want to debate the name...

I'd argue that GammaCurves are a subset of ToneCurves :)

kunzmi · 2026-08-20T10:44:59Z

Thanks for the comments! I'll wait for comments on the mailing list and will update then...
One thing that I couldn't figure out was how to get the noise profile data converted to DNG format. Does anyone know how to read the numbers? In that case it would be nice to add them.
The DNG spec specifies a standard deviation equal to sqrt(s * pixelValue + o), in the PI5 ISP spec I found a definition of s * sqrt(pixelValue) + o, where I have some doubts about its correctness, and what is the internal libcamera convention? Further, given that the one pixel value range is from 0..65535 and the others 0..1, I have no idea how to do the conversion.

Another question: The colour temperatures for the different CCM calibrations, I suppose these are the estimates from AWB and not some reference lights like D65 or StandardA? Only if the CCMs are calibrated with standard illuminants it would make sense to add ColorMatrix1 and ColorMatrix2 to the DNG so that one could do "correct" white balancing adjustments in photoshop/darktable later on. But if these are just estimates, it is better to maintain the single ColorMatrix as it is right now.

davidplowman · 2026-08-20T11:08:52Z

Hi again, just a few little things...

How would you store the LS tables in a DNG? What I've mostly seen, I think, is an OpcodeList2 followed by a number of gain tables, 1 per colour. I believed that's "most" common, though I realise there's variation in how things are done. Can you say if some other format would be better?
Re. the noise values. I agree sqrt(s * pixelValue + offset) is better, but that's not we have. It's not a libcamera convention or anything. In practice, for all our sensors we've only ever measured an offset of zero or nearly zero, so it doesn't really matter.
The CCMs are measured in an Image Engineering light box, though we have some misgivings about the colour accuracy of its tubes at this point. There's no existing way for libcamera to get this information. I suppose it could be yet more metadata, though it's per sensor, not per image. So it might better belong somewhere else? Probably a thing to talk about with the libcamera folks.

kunzmi · 2026-08-20T11:31:27Z

we have two options: OpcodeList2 which is before debayering, one then needs an individual table for each bayer-sub-pixel with the corresponding col/row offsets. Or we put it in OpcodeList3 which is after debayering, then it is one single table with 3 channels (interleaved, or single channel table applied to all channels in the image). As of now, I went for latter as I didn't want to mess around with the different bayer patterns.
this means we could in theory just take the scaling divide by 65535 (and some square?) to get the DNG scale value? If so, I would add the controls for this, too
I was thinking of adding a control for the first and last ccm of the provided list, under the assumption that these correspond to StandardA/D65. Alternatively one could also use the two closest ccms with the corresponding colour temperature, but I think that is a relatively new feature of DNG 1.6 or even 1.7, so likely no one supports it yet. Maybe just keep the single CCM as it is right now...

jmondi · 2026-08-20T12:22:35Z

        The nominal range is [-180, 180], where 0° leaves hues unchanged and the
        range wraps around continuously, with 180° == -180°.

+  - LensShadingCorrectionMapRed:


Isn't reporting tables per every frame unconditionally quite some data to pass around ? Are you planning for a control to enable/disable tables reporting ?

This is an option if we want it. All in all, it's around 12kb, so not a huge amount.

Your suggestion would be something similar to AeEnable at the very top of this file? And report the tables only if set to true - sounds reasonable to me

See also:

libcamera/src/libcamera/control_ids_rpi.yaml

Lines 10 to 18 in 6c1dd9d

- StatsOutputEnable:

type: bool

direction: inout

description: |

Toggles the Raspberry Pi IPA to output the hardware generated statistics.

When this control is set to true, the IPA outputs a binary dump of the

hardware generated statistics through the Request metadata in the

Bcm2835StatsOutput control.

Maybe not a huge amount (but not super tiny either). It's anyway data that have to be copied for each frame, and I guess it's fair to be able to enable/disable that.

A boolean control (like AeEnable and many other should do). We already have LensShadingCorrectionEnable to enable/disable the algorithms, myabe LensShadingTablesEnable or something similar ?)

For reference, Android has a similar control: https://developer.android.com/reference/android/hardware/camera2/CaptureRequest#STATISTICS_LENS_SHADING_MAP_MODE

However, maybe it's a discussion to be had on the mailing list ?

davidplowman · 2026-08-20T14:24:42Z

we have two options: OpcodeList2 which is before debayering, one then needs an individual table for each bayer-sub-pixel with the corresponding col/row offsets. Or we put it in OpcodeList3 which is after debayering, then it is one single table with 3 channels (interleaved, or single channel table applied to all channels in the image). As of now, I went for latter as I didn't want to mess around with the different bayer patterns.

We do LSC before demosaic, but so long as everything is linear it doesn't particularly matter. I think a DNG file gets to say what the order of the tables is, so there's no particular problem there. As I said, my impression is that OpcodeList2 is a bit more widely supported, but I'm not an expert on this and could be wrong there.

this means we could in theory just take the scaling divide by 65535 (and some square?) to get the DNG scale value? If so, I would add the controls for this, too

I was thinking of adding a control for the first and last ccm of the provided list, under the assumption that these correspond to StandardA/D65. Alternatively one could also use the two closest ccms with the corresponding colour temperature, but I think that is a relatively new feature of DNG 1.6 or even 1.7, so likely no one supports it yet. Maybe just keep the single CCM as it is right now...

Works for me!

…s metadata Having the LensShadingCorrection maps and ToneCurves available in frame metadata allows creating a DNG file with all necessary information so that the DNG matches the JPEG image without colour casts. I opened a PR in raspberrypi/rpicam-apps (raspberrypi/rpicam-apps#928) improving the colour accuracy of said DNG files created by these apps. The information needed is currently not provided by libcamera in the frame metadata. This patch intends to add LensShadingCorrection maps and ToneCurve to the controls metadata in libcamera, so that DNG files can be written with correct colours. Signed-off-by: Michael Kunz <mkunz@articimaging.eu>

kunzmi · 2026-08-20T21:00:50Z

Updated the PR according to the comments, the main changes are:

the LensShadingCorrection maps are no more 3 seperate tables but one in
planar configuration.
Due to the size of the maps, the output has to be activated with the new
control EnableLensShadingCorrectionMapOutput
The controls for output are set to 'out'
Added NoiseProfile control

The PR raspberrypi/rpicam-apps#928 is also updated to take the changes into account. I also sent an update to the mailing list, so everything should be in sync :)

naushir · 2026-08-24T07:20:01Z

@kbingham do you know if this patch is still waiting to be approved on the mailing list? I haven't seen it yet...

kbingham · 2026-08-24T08:10:06Z

@kbingham do you know if this patch is still waiting to be approved on the mailing list? I haven't seen it yet...

Hrm..

I definitely wrote this after I'd (believed I had) hit the approve button:

Sure, patch is sent - waiting for approval as I didn't register to the mailing list...

Sorry - both me and Laurent are the moderators but we've both been on holiday at the same time, I'm back today - so I've just let this through.

And yet - nothing is in my inbox either after the moderation email. I can only fear I might have pushed the wrong button and discarded by mistake ?

@kunzmi I'm sorry to ask, but could you resend please ?

kunzmi · 2026-08-24T14:37:37Z

Hi, sure I can send the patch again once I'm back to my Pi5 tomorrow. But before I do this and spam you the mailing list, I can see both patches (original and update) in the August archive, so they are there somehow?

kunzmi · 2026-08-25T13:40:42Z

@kbingham Hi again, I just sent a v3 to the mailing list with the same changes as the v2. Did you receive anything?

kbingham · 2026-08-25T14:23:12Z

@kbingham Hi again, I just sent a v3 to the mailing list with the same changes as the v2. Did you receive anything?

Thank you for resending, but I'm afraid it's still not getting through. I can see you are listed in the 'automatically accept' list, So I suspect there's something else going on with the list-server.

You weren't moderated (you're marked as an approved sender), and it's been received in the archives (https://lists.libcamera.org/pipermail/libcamera-devel/2026-August/061628.html) - but not sent out?

I know there were some DKIM/SPF changes lately so I wonder if it's related to that. Do you have a custom email server by any chance ? Are you sending mails which would fail DKIM/SPF challenges ?

kunzmi mentioned this pull request Aug 18, 2026

Improve DNG output quality using new frame metadata from libcamera raspberrypi/rpicam-apps#928

Open

davidplowman reviewed Aug 20, 2026

View reviewed changes

jmondi reviewed Aug 20, 2026

View reviewed changes

kunzmi force-pushed the improveDNGOutput branch from eb6e4ed to e6db90d Compare August 20, 2026 20:36

	- StatsOutputEnable:
	type: bool
	direction: inout
	description: \|
	Toggles the Raspberry Pi IPA to output the hardware generated statistics.

	When this control is set to true, the IPA outputs a binary dump of the
	hardware generated statistics through the Request metadata in the
	Bcm2835StatsOutput control.

Uh oh!

Conversation

kunzmi commented Aug 18, 2026

Uh oh!

naushir commented Aug 19, 2026

Uh oh!

kunzmi commented Aug 19, 2026

Uh oh!

davidplowman commented Aug 20, 2026

Uh oh!

kbingham commented Aug 20, 2026

Uh oh!

davidplowman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kunzmi commented Aug 20, 2026

Uh oh!

davidplowman commented Aug 20, 2026

Uh oh!

kunzmi commented Aug 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

naushir Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmondi Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davidplowman commented Aug 20, 2026

Uh oh!

kunzmi commented Aug 20, 2026

Uh oh!

naushir commented Aug 24, 2026

Uh oh!

kbingham commented Aug 24, 2026

Uh oh!

kunzmi commented Aug 24, 2026

Uh oh!

kunzmi commented Aug 25, 2026

Uh oh!

kbingham commented Aug 25, 2026

Uh oh!

Reviewers

naushir Aug 20, 2026 •

edited

Loading

jmondi Aug 20, 2026 •

edited

Loading