Camera mask: drawn areas are skipped; framing shows its outline (#170)

README.md

@@ -154,7 +154,7 @@ Detected surfaces appear as numbered outlines over the scan. In the editor:
 | Move a corner | Drag a handle. On a curved run, the neighboring points follow with a smooth falloff, so the curve bends instead of kinking. |
 | Add a corner | Double-click an edge of the selected surface. |
 | Remove a corner | Alt-click a handle. A surface always keeps at least three corners. |
-| Skip part of the room when scanning | Camera section: **Mask area**, then click corners on the camera preview around what to scan, double-click to finish; add more areas as needed. Windows, TVs and shiny things outside the mask are ignored by the next scan. **Hole fill** sets how much small gaps in the scan are filled (more: fuller surfaces; less: truer edges). **HDR** captures each pattern at 2 or 3 exposures and keeps the best-lit reading per pixel, for scenes with dark and bright surfaces (slower). All are kept per camera. |
+| Skip part of the room when scanning | Camera section: **Skip an area**, then click corners on the camera preview around a window, a TV or shiny things, double-click to finish; add more areas as needed. Skipped areas are shaded on the preview and left out of the next scan; click one to stop skipping it, or **Skip nothing** to clear them all. **Hole fill** sets how much small gaps in the scan are filled (more: fuller surfaces; less: truer edges). **HDR** captures each pattern at 2 or 3 exposures and keeps the best-lit reading per pixel, for scenes with dark and bright surfaces (slower). All are kept per camera. |
 | Zoom in to place points precisely | Pinch on the trackpad (or Cmd/Ctrl + scroll) over the scan to zoom, up to 16×; two-finger scroll moves around. **−**, **+** and **Fit** sit under the scan. Handles and labels keep their size, so you can place points much more precisely. |
 | Less on screen | Click a sidebar heading to fold or open that section; the Editor remembers it. Set-up-once sections (Schedule, MIDI, Hardware, Camera, Test frame) start folded, and a section with a problem (no projector, sound blocked) opens by itself. |
 | Delete surfaces | Click a surface and press **Delete** or **Backspace** (or the panel's Delete). Shift-click more surfaces to delete them all at once; one Undo brings them back. |

docs/http-api.md

@@ -23,7 +23,7 @@ Most write routes respond with the updated show, and the show is broadcast over
 | GET | `/api/camera/preview.jpg` | | A JPEG frame, downscaled to 1280 px wide. 409 if no camera is selected or a scan is running |
 | POST | `/api/camera/release` | | Closes the camera. 409 while scanning |
 | GET | `/api/camera/scan-settings` | | The selected camera's scan settings: `{"hole_fill": 9, "hdr": 1, "mask": null}`. 404 with no camera |
-| POST | `/api/camera/scan-settings` | `{"hole_fill"?: 0..31, "hdr"?: 1..3, "mask"?: [[[x, y], ...], ...], "clear_mask"?: true}` | Changes them for the selected camera (saved with its calibration in settings.json); the next scan uses them. `hole_fill`: how wide a gap between decoded pixels is filled (0 = none). `mask`: areas of the camera image to scan (up to 16 shapes of 3+ points, 0..1 camera coordinates); everything outside is ignored. `clear_mask`: scan the whole image again. `hdr`: 1 (off), 2 or 3 exposures per pattern, the calibrated one plus longer ones (x4; x3 and x9) capped at the camera's limit, merged per pixel |
+| POST | `/api/camera/scan-settings` | `{"hole_fill"?: 0..31, "hdr"?: 1..3, "mask"?: [[[x, y], ...], ...], "clear_mask"?: true}` | Changes them for the selected camera (saved with its calibration in settings.json); the next scan uses them. `hole_fill`: how wide a gap between decoded pixels is filled (0 = none). `mask`: areas of the camera image to skip, e.g. a window, a TV or shiny things (up to 16 shapes of 3+ points, 0..1 camera coordinates); everything else is scanned. `clear_mask`: skip nothing again. `hdr`: 1 (off), 2 or 3 exposures per pattern, the calibrated one plus longer ones (x4; x3 and x9) capped at the camera's limit, merged per pixel |
 | POST | `/api/camera/calibrate` | | Shows white on the output and runs the exposure search; returns `{"exposure", "gain", "p99"}`. 409 without a USB webcam or output window, 422 if calibration fails |
 | POST | `/api/camera/framing` | | Shows white then black on the output (acknowledged, like scan patterns) and returns how the projection fills the camera's view: `{"span": 0..1, "cut_off": bool, "outline": [[x, y], ...] (0..1), "advice": str or null}`. The projector goes back to the show after. 409 without a camera or output window, or while scanning |
 

engine/messages.py

@@ -108,6 +108,7 @@ class OscRequest(BaseModel):
 class ScanSettingsRequest(BaseModel):
     # What each means, its default and bounds: engine/scan_settings.py.
     hole_fill: Annotated[int, Field(ge=0, le=HOLE_FILL_MAX)] | None = None
+    # Areas of the camera image to skip (0..1 coordinates), drawn on the preview (#170).
     mask: Annotated[list[Annotated[list[UnitPoint], Field(min_length=3, max_length=MASK_MAX_POINTS)]],
                     Field(min_length=1, max_length=MASK_MAX_AREAS)] | None = None
     clear_mask: bool = False  # back to scanning the whole camera image

engine/scan.py

@@ -206,19 +206,18 @@ def result(self) -> DecodeResult:
 
 
 def apply_camera_mask(r: DecodeResult, polygons: list[list[list[float]]] | None) -> DecodeResult:
-    """Keeps only camera pixels inside the mask (#66): polygons in 0..1 camera coordinates, e.g.
-    drawn on the preview to skip a window or a TV. None or [] keeps everything."""
+    """Drops camera pixels inside the skipped areas (#66, #170): polygons in 0..1 camera coordinates,
+    drawn on the preview around a window, a TV or shiny things. None or [] skips nothing."""
     import cv2
 
     if not polygons:
         return r
     h, w = r.valid.shape
-    keep = np.zeros((h, w), np.uint8)
+    skip = np.zeros((h, w), np.uint8)
     for poly in polygons:
         pts = np.round(np.array(poly, np.float64) * [w, h]).astype(np.int32)
-        cv2.fillPoly(keep, [pts], 1)
-    inside = keep.astype(bool)
-    r.valid = r.valid & inside
+        cv2.fillPoly(skip, [pts], 1)
+    r.valid = r.valid & ~skip.astype(bool)
     r.proj_x = np.where(r.valid, r.proj_x, -1)
     r.proj_y = np.where(r.valid, r.proj_y, -1)
     return r

engine/scan_settings.py

@@ -17,7 +17,7 @@
 class ScanSettings:
     hole_fill: int = 9  # px gap between decoded pixels the scan fills in (0 = none)
     hdr: int = 1  # exposures per pattern: 1 = off
-    mask: list | None = None  # areas of the camera image to scan (0..1 coordinates); None = all of it
+    mask: list | None = None  # areas of the camera image to skip (0..1 coordinates); None = scan it all
     aperture: str = "8"  # still cameras: the f-number to scan at (f/8: deep focus), or "camera": as set
 
     @classmethod

frontend/index.html

@@ -88,7 +88,8 @@
       #framing-overlay { pointer-events: none; }
       #framing-overlay polygon { fill: none; stroke: #38bdf8; stroke-width: 2; stroke-dasharray: 6 4; vector-effect: non-scaling-stroke; }
       #framing-note.warn { color: #fbbf24; }
-      #mask-overlay polygon { fill: rgba(74, 222, 128, 0.15); stroke: #4ade80; stroke-width: 2; vector-effect: non-scaling-stroke; }
+      /* Skipped areas: shaded, so the preview shows what the scan will leave out (#170); click one to remove it. */
+      #mask-overlay polygon { fill: rgba(239, 68, 68, 0.35); stroke: #ef4444; stroke-width: 2; vector-effect: non-scaling-stroke; cursor: pointer; }
       #mask-overlay polyline { fill: none; stroke: #fbbf24; stroke-width: 2; vector-effect: non-scaling-stroke; }
       #preview-box.drawing { cursor: crosshair; }
       details.fold-inline { margin: 0 0 12px; }
@@ -333,9 +334,9 @@ <h2 id="surface-title">Surface</h2>
           <svg id="framing-overlay" viewBox="0 0 1 1" preserveAspectRatio="none" xmlns="http://www.w3.org/2000/svg"><polygon /></svg>
           <svg id="mask-overlay" viewBox="0 0 1 1" preserveAspectRatio="none" xmlns="http://www.w3.org/2000/svg"></svg>
         </div>
-        <div class="row" title="Scan only part of what the camera sees: skip a window, a TV or shiny things. Click corners on the preview, double-click or Enter to finish; add as many areas as you like. Applies to the next scan.">
-          <button id="mask-draw">Mask area</button>
-          <button id="mask-clear" hidden>Clear mask</button>
+        <div class="row" title="Skip part of what the camera sees: draw around a window, a TV or shiny things, and the next scan leaves them out. Click corners on the preview, double-click or Enter to finish; add as many areas as you like. Click a shaded area to stop skipping it.">
+          <button id="mask-draw">Skip an area</button>
+          <button id="mask-clear" hidden>Skip nothing</button>
           <span id="mask-note" class="muted"></span>
         </div>
         </details>

frontend/src/editor/cameraPanel.browser.test.ts

@@ -148,4 +148,46 @@ describe("the Camera section", () => {
     await vi.waitFor(() => expect(notices).toEqual([`Exposure check failed: ${detail}`]));
     expect($("calibrate").textContent).toBe("Check exposure");
   });
+
+  it("says drawn areas are skipped (#170)", async () => {
+    const twoAreas = [[[0, 0], [0.2, 0], [0.2, 0.2]], [[0.5, 0.5], [0.7, 0.5], [0.7, 0.7]]];
+    const { engine } = fakeEngine({ scanSettings: async () => json({ hole_fill: 9, hdr: 1, mask: twoAreas, aperture: "8" }) });
+    const panel = mountCameraPanel({ engine, notice: () => {} });
+    expect($("mask-draw").textContent).toBe("Skip an area");
+    await panel.loadScanSettings();
+    expect($("mask-note").textContent).toBe("Skipping 2 areas");
+    expect($("mask-draw").title + $("mask-draw").closest(".row")!.getAttribute("title")).toMatch(/skip/i);
+  });
+
+  it("removes one skipped area when it's clicked, keeping the others", async () => {
+    const twoAreas = [[[0, 0], [0.2, 0], [0.2, 0.2]], [[0.5, 0.5], [0.7, 0.5], [0.7, 0.7]]];
+    const { engine, calls } = fakeEngine({ scanSettings: async () => json({ hole_fill: 9, hdr: 1, mask: twoAreas, aperture: "8" }) });
+    const panel = mountCameraPanel({ engine, notice: () => {} });
+    await panel.loadScanSettings();
+    document.querySelectorAll("#mask-overlay polygon")[0].dispatchEvent(new MouseEvent("click", { bubbles: true }));
+    expect(calls).toContain(`setScanSettings ${JSON.stringify({ mask: [twoAreas[1]] })}`);
+  });
+
+  it("shows the preview after a framing check, so its outline can be seen (#170)", async () => {
+    const { engine } = fakeEngine();
+    const panel = mountCameraPanel({ engine, notice: () => {}, fetchPreview: async () => new Response(new Blob(["x"])) });
+    panel.update(status(AC410), false);
+    expect($("preview-box").hidden).toBe(true);
+    $("check-framing").click();
+    await vi.waitFor(() => expect($("framing-note").hidden).toBe(false));
+    expect($("preview-box").hidden).toBe(false);
+    panel.stopPreview();
+  });
+
+  it("forgets the framing result when another camera is chosen", async () => {
+    const { engine } = fakeEngine();
+    const panel = mountCameraPanel({ engine, notice: () => {} });
+    panel.update(status(AC410), false);
+    $("check-framing").click();
+    await vi.waitFor(() => expect($("framing-note").hidden).toBe(false));
+    panel.update(status(A6600), false);
+    expect($("framing-note").hidden).toBe(true);
+    expect(document.querySelector("#framing-overlay polygon")!.getAttribute("points")).toBe("");
+  });
 });
+

frontend/src/editor/cameraPanel.ts

@@ -57,16 +57,23 @@ export function mountCameraPanel(deps: CameraPanelDeps): CameraPanel {
   holeFill.addEventListener("input", () => { holeFillReadout.textContent = `${holeFill.value} px`; });
   holeFill.addEventListener("change", () => void engine.setScanSettings({ hole_fill: Number(holeFill.value) }));
 
-  // ---- Camera mask (#66): click corners on the preview to mark areas to scan; the rest is ignored.
+  // ---- Skipped areas (#66, #170): click corners on the preview around what the scan should leave out
+  // (a window, a TV, shiny things); click a shaded area to stop skipping it.
   const previewBox = $("preview-box");
   const maskOverlay = $("mask-overlay");
   let maskDraw = idleDraw; // in preview pixels, so the double-click echo rule works; saved as 0..1
   function renderMask() {
     const { width, height } = previewBox.getBoundingClientRect();
     const toUnit = (p: number[]) => `${p[0] / Math.max(1, width)},${p[1] / Math.max(1, height)}`;
-    const shapes: SVGElement[] = (cameraMask ?? []).map((poly) => {
+    const shapes: SVGElement[] = (cameraMask ?? []).map((poly, i) => {
       const el = document.createElementNS(SVG_NS, "polygon");
       el.setAttribute("points", poly.map(([x, y]) => `${x},${y}`).join(" "));
+      el.addEventListener("click", (ev) => {
+        if (maskDraw.active) return; // drawing: the click places a corner
+        ev.stopPropagation();
+        const rest = (cameraMask ?? []).filter((_, j) => j !== i);
+        void engine.setScanSettings(rest.length ? { mask: rest } : { clear_mask: true }).then(() => loadScanSettings());
+      });
       return el;
     });
     if (maskDraw.active && maskDraw.points.length) {
@@ -76,9 +83,10 @@ export function mountCameraPanel(deps: CameraPanelDeps): CameraPanel {
     }
     maskOverlay.replaceChildren(...shapes);
     previewBox.classList.toggle("drawing", maskDraw.active);
-    $("mask-draw").textContent = maskDraw.active ? "Double-click to finish" : "Mask area";
+    $("mask-draw").textContent = maskDraw.active ? "Double-click to finish" : "Skip an area";
     $("mask-clear").hidden = !cameraMask;
-    $("mask-note").textContent = cameraMask ? `Scanning ${cameraMask.length} area${cameraMask.length > 1 ? "s" : ""} only` : "";
+    $("mask-note").textContent = maskDraw.active ? "Click corners on the preview around what to skip"
+      : cameraMask ? `Skipping ${cameraMask.length} area${cameraMask.length > 1 ? "s" : ""}` : "";
   }
   function maskStep(ev: DrawEvent) {
     maskDraw = drawStep(maskDraw, ev);
@@ -183,6 +191,7 @@ export function mountCameraPanel(deps: CameraPanelDeps): CameraPanel {
     framingNote.classList.toggle("warn", view.warn);
     framingNote.hidden = false;
     document.querySelector("#framing-overlay polygon")!.setAttribute("points", view.points);
+    if (!previewRunning()) togglePreview(); // the outline is drawn on the preview (#170)
   }));
 
   // ---- What status says about the camera.
@@ -206,6 +215,10 @@ export function mountCameraPanel(deps: CameraPanelDeps): CameraPanel {
 
   return {
     update(next, isScanning) {
+      if (next?.camera.selected !== status?.camera.selected) { // another camera: its framing is unknown
+        framingNote.hidden = true;
+        document.querySelector("#framing-overlay polygon")!.setAttribute("points", "");
+      }
       status = next;
       scanning = isScanning;
       render();

frontend/src/shared/messages.ts

@@ -267,7 +267,7 @@ export function parseServerMessage(raw: string): ServerMessage | null {
 export interface ScanSettingsResponse {
   hole_fill: number; // px gap between decoded pixels the scan fills in
   hdr: number; // exposures per pattern: 1 = off
-  mask: number[][][] | null; // areas of the camera image to scan (0..1); null = all of it
+  mask: number[][][] | null; // areas of the camera image to skip (0..1); null = skip nothing
   aperture: string; // still cameras: f-number ("8" = f/8), or "camera"
 }
 

tests/test_scan_api.py

@@ -211,7 +211,7 @@ def scan_covered(client, ed, out):
     assert settings["scan_settings"][AC410["unique_id"]]["hole_fill"] == 25  # by camera, like calibration
 
 
-def test_a_camera_mask_limits_what_the_scan_finds(rig, tmp_path):
+def test_skipped_camera_areas_leave_out_what_they_cover(rig, tmp_path):
     scene, uvc, hw, cams = rig
 
     def scan(client, ed, out):
@@ -225,8 +225,8 @@ def scan(client, ed, out):
     with engine(hw, data_dir=tmp_path, camera_factory=cams, uvc_factory=lambda a: uvc) as client, \
             editor(client) as ed, output(client, W, H) as out:
         full = scan(client, ed, out)
-        left = [[[0, 0], [0.5, 0], [0.5, 1], [0, 1]]]
-        assert client.post("/api/camera/scan-settings", json={"mask": left}).json()["mask"] == left
+        right = [[[0.5, 0], [1, 0], [1, 1], [0.5, 1]]]  # skip the right half, where the box is (#170)
+        assert client.post("/api/camera/scan-settings", json={"mask": right}).json()["mask"] == right
         masked = scan(client, ed, out)
         assert masked["coverage"] < full["coverage"]
         assert len(masked["surfaces"]) < len(full["surfaces"])

tests/test_scan_decode.py

@@ -122,18 +122,19 @@ def test_hole_fill_closes_more_gaps_the_higher_it_is():
     assert projector_space_image(r)[1].sum() == projector_space_image(r, hole_fill_px=9)[1].sum()  # today's default
 
 
-def test_a_camera_mask_drops_everything_outside_it():
-    """#66: only the masked area of the camera image is scanned; the rest yields nothing."""
+def test_skipped_areas_of_the_camera_image_yield_nothing():
+    """#170: drawn areas are skipped (a window, a TV, shiny things); everything else is scanned.
+    (Before, the drawn area was the only part kept, the opposite of what its button suggested.)"""
     from engine.scan import apply_camera_mask, projector_space_image
     from engine.surfaces import detect_surfaces
 
     scene = Scene()
     full = decode(scene)
-    # Keep the left half of the camera image (0..1 coordinates), so the box (u 190..250) is outside.
-    left = [[[0, 0], [0.5, 0], [0.5, 1], [0, 1]]]
-    masked = apply_camera_mask(decode(scene), left)
-    assert not masked.valid[:, 170:].any()  # nothing decoded right of the mask's edge (160 px)
-    assert masked.valid[:, :150].sum() == full.valid[:, :150].sum()  # inside: untouched
+    # Skip the right half of the camera image (0..1 coordinates), so the box (u 190..250) is in it.
+    right = [[[0.5, 0], [1, 0], [1, 1], [0.5, 1]]]
+    masked = apply_camera_mask(decode(scene), right)
+    assert not masked.valid[:, 170:].any()  # nothing decoded in the skipped area (from 160 px)
+    assert masked.valid[:, :150].sum() == full.valid[:, :150].sum()  # outside it: untouched
     assert (masked.proj_x[~masked.valid] == -1).all()
     surfaces = detect_surfaces(masked, projector_space_image(masked))
     assert len(surfaces) < len(detect_surfaces(full, projector_space_image(full)))  # the box is gone