Cut the single-core time out of mapping, and make the Ceres backend configurable

[kevintsq]

Summary

Mapping spent most of its wall clock on one core. Sampling process-tree CPU once a
second across a 617-image run showed 740s of 1218s (61%) running on roughly a single
core, almost none of it inside the native solvers. This branch removes the bulk of that
serial work, makes the Ceres solver backend configurable, fixes point colors, and repairs
the native build.

The same scene now finishes in 681s with 250s (37%) on one core.

	before	after
wall clock	1218s	681s
single-core time	740s (61%)	250s (37%)
mean cores (of 32)	3.66	5.66
global positioning, pass 1	123s @ 6.9 cores	105s @ 7.7 cores
global positioning, pass 2	151s @ 6.8 cores	126s @ 7.8 cores
BA problem construction, per round	52–62s @ 1.0 core	6–8s @ 1.0 core
mean reprojection error	1.16164	1.16021
registered images / points	617 / 242576	617 / 242656

What was slow

Every bundle-adjustment round spent 52–62s building the problem in Python before handing
a 35s solve to Ceres — preparation cost more than the solve itself, eight rounds in a row.

The dominant term was small_triangulation_angle_mask, called once per image inside the
per-image loop, each call materializing all 380k track ids into a Python list and then a
set. That work is loop-invariant, and the underlying filter is per-track independent, so
one pass over the problem answers every per-image query.

Two smaller serial costs sat alongside it: per-observation pybind lookups for point
coordinates and track lengths (~1.25M calls per round), and a full rebuild of every
pycolmap point after each positioning pass, which only moves coordinates.

Changes

Fix the native extension build against the installed COLMAP. Rebuilding
vidmap_native currently yields a module that cannot be imported: COLMAP's static
libraries reference Abseil's logging internals without listing Abseil in their exported
configuration, leaving undefined absl::log_internal symbols, and a shared Ceres outside
the loader's search path is not found at import time. Links Abseil when available and
keeps link-time library directories in the installed RPATH.

Extract point colors before writing a reconstruction. Nothing on the mapping path
sampled color, so every points3D.bin carried RGB (0, 0, 0) and COLMAP rendered the
cloud black — only the HTML visualization called extract_colors_for_all_images, which
is why the same run looked colored there and blank in COLMAP. Color is cosmetic, so a
missing image root degrades to a warning rather than discarding a finished
reconstruction.

Cut the single-threaded Python work between native solves. Hoists the small-angle id
computation out of the per-image loop; snapshots point coordinates and track lengths into
one Point3DTable per solve so per-image lookups are a binary search. Rows stay in
reconstruction order because that order reaches Ceres through variable_point3D_ids.
Positioning passes now publish values instead of rebuilding all points, and the replay
track snapshot is only materialized when replay is enabled.

Make the Ceres linear solver and thread counts configurable. The global solves
hard-coded a direct sparse Schur factorization, whose factorization step is
single-threaded; rotation averaging was pinned to one thread with no way to change it.
Adds linear_solver, preconditioner and use_cuda for bundle adjustment and global
positioning, plus num_threads for rotation averaging.

python -m vidmap.map ... \
  mapper.ba.solver_backend.linear_solver=iterative_schur \
  mapper.gp.solver_backend.use_cuda=true \
  mapper.ra.num_threads=8

Every default reproduces current behavior: the direct sparse solve with CUDA off, and one
thread for rotation averaging, which keeps that stage byte-identical as its pinned random
seed intends. Requesting CUDA is rejected with a clear message when Ceres was built
without it. Also drops global positioning's CLUSTER_TRIDIAGONAL preconditioner, which
SPARSE_SCHUR ignores — timing it explicitly confirmed no difference.

Verification

Multi-threaded Ceres is not bit-reproducible here, so results are compared against the
pipeline's own run-to-run spread after removing the similarity gauge:

• 89-image scene: baseline varies by 0.10% of scene radius between two runs of
  unmodified code; before-versus-after differs by 0.05%. Reprojection error 0.94155
  vs 0.94173, same 89 registered images.
• 617-image scene: self-noise 0.41%, registered images and reprojection error
  unchanged.

Notes for reviewers

• iterative_schur and use_cuda are wired but off by default and not yet recommended.
  On the 89-image scene iterative_schur was slower (3m43 vs 1m30) and converged to a
  materially different solution (scale off by 2x), so it needs tuning before it is
  useful.
• Two Ceres installs can be present, and CMake may prefer a shared build with
  EigenSparse only over the SuiteSparse build COLMAP was compiled against. That choice
  is worth 2x on global positioning (212s @ 3.8 cores vs 105s @ 7.7 cores), so watch
  the -- Found Ceres version: configure line and pass Ceres_DIR if it picks the wrong
  one.
• This branch also carries the earlier Expose full BA diagnostics and tidy descriptor loading commit.

🤖 Generated with Claude Code

[kevintsq]

Superseded by #6