Skip to content

media: apple: isp: report the frame gain and exposure - #689

Merged
RyanTheTide merged 1 commit into
aurora-wipfrom
feat/isp-frame-gain
Oct 31, 2026
Merged

RyanTheTide merged 1 commit into
aurora-wipfrom
feat/isp-frame-gain

Conversation

@RyanTheTide

Copy link
Copy Markdown
Contributor

The camera firmware writes each frame's exposure into the metadata buffer it hands back with the frame. The driver maps that buffer but never reads it, so userspace cannot tell how hard the sensor is working. The camera denoise filter (aurora-silicon/camera-denoise#1) needs that to turn on at the same gain levels as macOS.

Change

  • The metadata buffer opens with a table of section offsets. On H17 firmware (T8140, T6040) the fourth section starts with the integration time in microseconds and the total sensor gain in 1/256 steps (256 is unity).
  • Read it, through the table and bounded by the buffer size, when a metadata buffer comes back, and report the last frame's values as read-only, volatile V4L2_CID_GAIN and V4L2_CID_EXPOSURE_ABSOLUTE (100 us units).
  • Only hardware whose match data sets meta_exposure (T8140 and T6040) gets the controls. No parameters or new knobs.

How the layout was found (Linux-side measurement only): kprobes on ipc_bt_handle with the module's BTF dumped the metadata buffers of the running fleet12 driver, read-only. On a fixed scene, auto exposure starts at 8.3 ms and gain 260, raises the integration time to the frame period (33.3 ms at 30 fps, 66.6 ms at 15 fps) and then climbs the gain field 260, 372, 990, 1758, 2489, 3253 until it settles. Settled values were 3253 to 4168 on the dark J700 scene and 600 to 888 on the brighter J616s one. Both machines have the same section table, with the section at 0x1870.

Testing

  • KUnit (arm64 QEMU, apple-isp*): 31/31, including the new isp_test_meta_exposure (table lookup, out-of-bounds section, unwritten section, hardware without the report).
  • defconfig + asahi.config, LLVM, W=1 KCFLAGS=-Werror on the ISP objects: 0 warnings.
  • checkpatch --strict: clean.
  • Not yet booted. I did not install kernels on the test machines, so the controls themselves need a fleet boot. Check with v4l2-ctl -d /dev/video1 -C gain,exposure_time_absolute while streaming. The values should match what the kprobe dump showed.

🤖 Generated with Claude Code

Userspace has no way to tell how hard the sensor is working. Gain is the
signal a camera pipeline needs to decide when the picture is noisy enough
to denoise, and the firmware already writes it into every frame's
metadata buffer, which the driver maps but never reads.

The metadata buffer opens with a table of section offsets. On H17
firmware the fourth section starts with the frame's exposure: the
integration time in microseconds, then the total sensor gain in 1/256
steps with 256 as unity. This was established on T6040 and T8140 by
watching auto exposure converge on a fixed scene, where the gain climbed
from 260 to its final value once the integration time hit the frame
period, and by stepping the frame rate, which capped the integration
time at the frame period.

Read the section when a metadata buffer comes back, through the table and
bounded by the buffer size, and report the last values as read-only,
volatile V4L2_CID_GAIN and V4L2_CID_EXPOSURE_ABSOLUTE controls. Only
hardware whose match data says its metadata carries the report gets the
controls.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Tested in fleet12d (aurora-wip cd217c2 + #661 #633 #689 #683) on the J616s, the M1 Pro and the Neo, with no regression: kernel -p3 0 and aurora-diag check --expect 0 fail on each.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant