Skip to content

DM-54951 : Modularize ap_daytime script for CM-Service - #277

Open
tcjennings wants to merge 5 commits into
mainfrom
tickets/DM-54951
Open

tcjennings wants to merge 5 commits into
mainfrom
tickets/DM-54951

Conversation

@tcjennings

@tcjennings tcjennings commented Jun 12, 2026 •

Copy link
Copy Markdown
  • adds a CM Service campaign template file for ap_daytime
  • CM template files can be directly consumed as campaign definition files for scheduled campaigns in CM service, or imported with the CM campaign edit tool.
  • These templates are fully-hydrated and vsc-managed template files with dynamic/variable components.
  • A CM Service scheduled campaign will read the raw file from a github link (with full ref and repo path); CM Service does not check out this repo or perform other git operations.

@tcjennings
tcjennings force-pushed the tickets/DM-54951 branch 3 times, most recently from 37db766 to ed5bc27 Compare June 16, 2026 15:41
@tcjennings
tcjennings force-pushed the tickets/DM-54951 branch 6 times, most recently from 8cd71f5 to 6b16bc8 Compare June 25, 2026 19:30
@tcjennings
tcjennings force-pushed the tickets/DM-54951 branch 3 times, most recently from 95dbca6 to 1726e50 Compare July 8, 2026 19:21
@tcjennings
tcjennings marked this pull request as ready for review July 8, 2026 19:24
@tcjennings
tcjennings force-pushed the tickets/DM-54951 branch 4 times, most recently from 0283889 to 102942e Compare July 13, 2026 18:46
@tcjennings
tcjennings force-pushed the tickets/DM-54951 branch 2 times, most recently from fb6a4b3 to 58f8d76 Compare August 10, 2026 20:14
@tcjennings
tcjennings force-pushed the tickets/DM-54951 branch 6 times, most recently from 78f424a to 71d2b3e Compare October 16, 2026 17:04
@tcjennings
tcjennings force-pushed the tickets/DM-54951 branch 3 times, most recently from 6be1807 to 6d74ba4 Compare October 27, 2026 16:39
@tcjennings
tcjennings force-pushed the tickets/DM-54951 branch 17 times, most recently from 14f0730 to cd0c153 Compare October 30, 2026 21:36
@tcjennings
tcjennings requested a review from hsinfang October 30, 2026 21:36

@hsinfang hsinfang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; a question about --qgraph-datastore-records.

Should we remove submit_ap_daytime.sh and other now-obsolete files too, now this is run in cm-service?

- "-c parameters:release_id=1"
- "-c parameters:apdb_config=s3://embargo@rubin-summit-users/apdb_config/cassandra/pp_apdb_lsstcam.yaml"
- "-c associateApdb:doRunForcedMeasurement=False"
- "--dataset-query-constraint off"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing this to ap_pipe/scripts/LSSTCam/submit_ap_daytime.sh, --qgraph-datastore-record is left out. Was it intentional?
(https://rubinobs.atlassian.net/browse/DM-55426 may be relevant)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(though post_isr_image datasets have been generated... )

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The legacy script uses pipetask directly for the qgraph generation, which is why that flag was needed. BPS includes --qgraph-datastore-records in its default set of arguments for pipetask.

CM has never used pipetask for qgraph generation; instead, it used bps acquire to generate the qgraph prior to pruning, so the argument is included by default.

Now CM doesn't separate the acquire/submit stage at all and uses the --prune-... argument to affect the same outcome with an ordinary bps submit, and still relies on the bps defaults to supply --qgraph-datastore-records where needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants