Skip to content

Image: CUDA -base instead of -runtime, plus libcurand (-1.45 GB download per image) - #22

Merged
pgodlews merged 1 commit into
mainfrom
image/slim-cuda-base
Sep 29, 2026
Merged

pgodlews merged 1 commit into
mainfrom
image/slim-cuda-base

Conversation

@pgodlews

Copy link
Copy Markdown
Owner

Part of #16 (option 1). The runtime stage now starts from nvidia/cuda:13.0.2-base-ubuntu24.04 plus libcurand-13-0 (the same 10.4.0.35 that -runtime shipped), instead of -runtime. The build stage is unchanged, so this rebuilds nothing heavy.

Why

Of the system CUDA libraries, LichtFeld requires only libcudart and libcurand (plus libcuda from the driver), and pycolmap requires libcurand. torch, torchvision and gsplat load CUDA 13 from their own wheels in venv_gs, and did so on -runtime too: /proc/self/maps shows the wheel's libcublas.so.13 in both. -runtime's cuBLAS, cuFFT, cuSOLVER, cuSPARSE, NVRTC, nvJitLink and NPP were 1.51 GB of every image's download, all unused.

Measured

0.2.0-rc2's contents, repacked onto the new base with no compiling. Sizes are compressed download / on disk:

Image 0.2.0-rc2 on -base
all 6.92 / 21.8 GB 5.48 / 18.3 GB
prep 5.46 / 15.7 GB 4.01 / 12.2 GB
train 3.51 / 11.9 GB 2.06 / 8.4 GB

Checked on nosacz, RTX 3090 (GPU0, 250 W limit), each original image against its repack:

  • train: 2,000 steps from the 0005 handoff bundle. PSNR 18.9935 / SSIM 0.6057 / LPIPS 0.2697 on the original, 18.9932 / 0.6059 / 0.2700 on the repack. Both reached 3.0M splats and exported.
  • prep, clip 0198 up to masks: frames (ffmpeg -hwaccel cuda) byte-identical (455 files). All 304 mask files byte-identical (70 of 76 frames had detections).
  • Mask R-CNN on 20 images from the 0005 bundle (181 person detections): identical scores, boxes and mask probabilities (max difference 0.0).
  • Host benchmark: fp32 17.05 vs 17.14 TFLOPS, fp16 48.98 vs 49.47, gsplat 62.1 vs 62.3 it/s, no errors.
  • The PR's own base stage builds (342 MB compressed); /usr/local/cuda/lib64 holds only cudart and curand.

Not checked here

  • The new LichtFeld pin. These tests ran LichtFeld e654717e (0.2.0-rc2), not 3067e9e0 from pycolmap 4.2.1, LichtFeld 3067e9e0, opt-in sfm.skip_redundant_points #21. A new system-library dependency needed at startup would fail the Dockerfile's ldd check at build time. One opened later with dlopen would only show in the rc's test run. e654717e opens none, and the 5 commits in the range change no CMake files.
  • cuda-compat stays, since -base includes it: datacenter GPUs on older drivers need it.

Other #16 options (LichtFeld leftovers, 3-architecture builds, venv_gs trims) are left for a later rc. liblfs_mcp.so can't be removed: LichtFeld-Studio requires it at startup, and it holds the training kernels.

…oad per image)

LichtFeld links only libcudart and libcurand from the system, pycolmap
libcurand, and torch loads the CUDA 13 libraries from its own wheels in
venv_gs. The -runtime image's cuBLAS, cuFFT, cuSOLVER, cuSPARSE, NVRTC,
nvJitLink and NPP (1.51 GB compressed in each of the three images) were
unused. Measured on 0.2.0-rc2's contents repacked onto -base: all 6.92 ->
5.48 GB, prep 5.46 -> 4.01, train 3.51 -> 2.06 (download); 2,000-step
training from the 0005 bundle on a 3090 gives the same PSNR/SSIM/LPIPS.

Part of #16.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant