Repository navigation
Conversation
The queue now runs on a Mac as a prep install: a job with run_until "sfm"
goes through frames, select, mask and sfm and writes a handoff bundle.
- config.PREP_BACKEND ("cuda", or "apple" on macOS; QUEUE_PREP_BACKEND
overrides, which the tests use to model a CUDA host wherever they run).
On apple the queue schedules on one device without nvidia-smi and is a
prep install (IMAGE_VARIANT defaults to "prep").
- JobConfig.prep_backend, set by the host that creates the job. An apple job
carries PREP_APPLE in every cache key; cuda adds no term, so existing keys
and the pins in test_stages.py are unchanged. The term travels in the
bundle's config, so a CUDA box computes the same keys on import.
- A host refuses to build a prep stage of a job made for the other backend.
- Stitched input is refused on a Mac (30_run_sfm.py needs CUDA).
- queue/run_mac.sh: the launcher, deploy.sh without systemd.
- Telemetry fills the CPU model and memory size on macOS.
- queue/test_prep_backend.py, added to CI.
Checked on clip 0141 through the queue on an M5 Max: 28.2 min, 473/473 rig
frames, 510,745 points, 0.807 px, a 2.27 GB bundle; a second instance set to
cuda imported it under the same keys. That bundle was not trained. Images
without the field refuse a Mac's bundle as an unknown config field.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The stages before training (frames, select, mask, sfm) run on an Apple silicon Mac, and the queue runs there as a prep install that writes a handoff bundle for a CUDA box. Training stays on CUDA.
What changes
sift_backend.pypicks the SIFT backend (cuda, metal, cpu); the stitch and Mask R-CNN run on MPS; frames decode with VideoToolbox.30_run_sfm.py(stitched input) refuses to run without CUDA.scripts/setup_mac.shbuilds pycolmap frompgodlews/colmapbranchmetal-sift-lanxinger, pinned by commit, and avenv_gswith torch 2.14.1.PREP_BACKEND(cuda, orappleon macOS). On a Mac the queue schedules on one device withoutnvidia-smi, requiresrun_until, and refuses stitched input.queue/run_mac.shstarts it.JobConfig.prep_backend, set by the host. Apple jobs carryPREP_APPLEin every key; CUDA keys are unchanged (the pins intest_stages.pypass untouched). A host refuses to build a prep stage of a job made for the other backend.Measured
Trained on an RTX 3090 (30,000 iterations, 3M splats, 0.2.0-rc3 train image), one run per arm, against a CUDA prep of the same job:
Through the queue on an M5 Max, 0141 preps in 28.2 min and writes a 2.27 GB bundle, which a second queue instance set to
cudaimported under the same keys.Not done
prep_backend), so training one needs a train image built from this.Details and all figures:
docs/how-it-works.md, "Prep on Apple silicon".