What happened
code_version folds in lightcone.engine.container.image_identity, a content hash of the Containerfile's
COPY sources. With the scaffolded COPY . . this includes everything in the project not excluded by
directory name (.git/.venv/results/.lightcone/.snakemake/pycache), so a new SLURM log file in logs/,
a README edit or a notebook changed the identity between two runs: Snakemake reported "Params have
changed", deleted a finished 10-minute fit and re-ran it, and lc status flipped all outputs to stale.
Nothing in the output tells the user which file changed the hash.
Reproduction
- Scaffolded project with the default
COPY . . Containerfile, runtime: none.
lc run output_a (ok).
touch logs/anything.out (or edit README.md).
lc status -> output_a is stale; lc run output_a re-runs it.
Workaround used: COPY only src/ config/ scripts/ and keep job logs under results/. A default
.dockerignore-style exclusion of logs/docs, or a status line naming the changed COPY source, would help.
Environment
- ASTRA: 0.2.11
- lightcone-cli: 0.4.2
- Python: 3.12.8
- OS: Linux 4.18 (RHEL 8, SLURM cluster, container runtime: none)
What happened
code_versionfolds inlightcone.engine.container.image_identity, a content hash of the Containerfile'sCOPY sources. With the scaffolded
COPY . .this includes everything in the project not excluded bydirectory name (.git/.venv/results/.lightcone/.snakemake/pycache), so a new SLURM log file in
logs/,a README edit or a notebook changed the identity between two runs: Snakemake reported "Params have
changed", deleted a finished 10-minute fit and re-ran it, and
lc statusflipped all outputs tostale.Nothing in the output tells the user which file changed the hash.
Reproduction
COPY . .Containerfile,runtime: none.lc run output_a(ok).touch logs/anything.out(or edit README.md).lc status-> output_a isstale;lc run output_are-runs it.Workaround used: COPY only
src/ config/ scripts/and keep job logs underresults/. A default.dockerignore-style exclusion of logs/docs, or a status line naming the changed COPY source, would help.Environment