Repository navigation
Track-CRV association branches (<trk>crv, <trk>crvxings, <trk>crvcands and their truth) - #441
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #441
Reviewed at head 5eb9a7ec004c8c8ea14172f1423dd9c8313f72ab. First pass.
Decision
- 🟢 approve. The six branches fill consistently on every sample I ran, from data, extracted MC, on-spill mock data and off-spill MC. Every fcl that does not set
trkCrvMatchTagswrites the same ntuple asmain. One finding concerns what happens once Production#595 is in the input files (finding 1), and one is a small item (finding 2). Neither blocks.
Scope understood
- A fit branch with
trkCrvMatchTags(and optionallytrkCrvMatchMCTags) gets<trk>crv,<trk>crvxings,<trk>crvcandsand the three MC branches, from the OfflineTrkCrvMatchandTrkCrvMatchMCrecords found byKalSeedPtr. - The prolog adds
TrkCrvMatcherandTrkCrvMatchMCMakerinstances that run in the ntuple job.from_mcs-extracted,from_rec-combinedand the newfrom_mcs-mockdata_trkcrvuse them. Version 6.17.0.
Findings
-
🟡 [S2] Once input files carry Production#595's association, the ntuple jobs recompute it under the same module labels, and the ntuple stores the poorer recomputed records.
- Evidence:
from_mcs-extracted.fclandfrom_mcs-mockdata_trkcrv.fcladdTrkCrvMatchNoFieldMCPathandTrkCrvMatchAllMCPath. These produceTrkCrvMatchLine,TrkCrvMatchDeand so on, the labels Production#595 uses. A tag with no process name then resolves to the ntuple job's own product. - I made off-spill reco with Production#595 (3,000 events) and ran
from_mcs-OffSpillwithtrkCrvMatchTags : ["TrkCrvMatchCHmu"]twice, once reading the file's records and once withTrkCrvMatchOffSpillMCPathadded:- The candidates are identical: 1,178 candidates, 664 geometric.
- The recomputed records have
probe2 instead of 3. On this sample the layer probe still finds the same 1,535 pulses. - The recomputed truth has no deposits:
hasdeposits0 of 1,176 true crossings instead of 1,187 of 1,187. The 11 crossings found only through their deposits are gone, andedepsums to 0 instead of 19.5 GeV. The reco-stage maker read the digiCrvSteps, which the reco output drops (TrkCrvMatchMCNtuplesetsRequireCrvSteps : falsefor that reason).
- Suggested fix: on files made after Production#595, read the stored records instead of recomputing. For example, keep the recompute sequences in variants meant for older files, or give the ntuple-stage producers their own labels, so a job never silently picks one record over the other.
- Evidence:
-
⚪ [S3]
DtRefCenter(five values) andSigmaV : [ 13 ]are copied from ProductionJobConfig/reco/prolog.fcl, as the comment atfcl/prolog.fcl:206says. Per-fit tables in Offline'sCRVRecoprolog, used by both repositories, would keep the two from drifting when the Run 1 tuning arrives.
Checked, no action needed
-
🟢 No change for fcls without
trkCrvMatchTags.from_mcs-mockdata(200 events),from_mcs-primary(200) andfrom_mcs-OffSpill(3,000), atmainand at this head: the same entries, the same branches in the same order, and the same total bytes in every branch. Only the version histogram changes, from 6.16 to 6.17. -
🟢 The new branches are consistent on every run:
- one entry per track;
ncandsandnxingsequal the vector sizes;bestgeomandbesttimeare valid indices, andbestgeompoints at a geometric candidate;- the
bestflag sits on the indicated candidate; ngeommatches the flags;- the MC branches are parallel to their reco branches;
- every
crvcoincindexes intocrvcoincs.
The runs:
job events tracks candidates (geometric) from_mcs-extracted, MDC2025bb300 205 86 (64) from_rec-combined, PassN#28 pass 1 on KPP run 1254721,000 476 230 (156), probe3from_mcs-mockdata_trkcrv,ensembleMDS3dOnSpill200 321 91 time-only off-spill reco with Production#595, validation setting 9,314 4,876 3,242 (2,800) -
🟢 The new branches add 0.2% (on-spill mock data) to 2.2% (off-spill) of the ntuple size.
-
🟢
utils/trkcrv_example.pyruns underpyenv anaon all four ntuples and writes its plot. On the off-spill MC, the best geometric candidate is the track's own cluster for every track, and the best time candidate for 99.4%. -
🟢 The dictionaries load: I read every new leaf with PyROOT after loading
libeventntuple_dict.
Validation check
- Build/tests run: yes. This head and
mainf745427, each with ArtAnalysis491fb22, on the CI build of Offlinemainca7b6580(Production#595 and Offline#2107 in the head workspace). Both built with exit status 0 and no warnings. All the jobs above exited with status 0. The PR merges cleanly withmain. - Config contract check: pass. Mismatched
trkCrvMatchMCTagsthrow at construction, and a missing collection throws throughgetValidHandle. - Cross-repo consistency: needs Offline#2106 (merged); it does not need Production#595. See finding 1 for after Production#595 merges.
Residual risk
from_rec-combinedwas run on 1,000 events of one KPP subrun. The CRV-tracker time offset on data is whatever the matcher's configuration gives; I did not check it against data.
Author follow-ups
- Finding 1: read the stored association on files that have it.
- Optional: one home for the per-fit tuning (finding 2).
|
Thanks @oksuzian for the review. 1. Recomputing under the reco labels. Fixed in 8d269c1:
Tests, on the rebuilt area:
2. One home for the per-fit tuning. Agreed. I'll make per-fit |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #441
Reviewed at head 8d269c11442a3853948e80e2c836863e7b81f2ba. Re-review of 5eb9a7e.
Decision
- 🟢 approve. The new commit fixes my finding 1: the ntuple-stage matchers now have their own labels, so they can no longer shadow the association that Production#595 writes into reco files. Every fcl that uses them resolves to the new labels, and none still names the old ones.
What changed since 5eb9a7e
- One commit,
8d269c1"Address review comments." (fcl/prolog.fcl,from_mcs-extracted.fcl,from_rec-combined.fcl; +30 −26). Not a rebase. - Every producer in
TrkCrvMatchProducersand every path and tag list gains the suffixNtuple(TrkCrvMatchLineNtuple,TrkCrvMatchDeNtupleMC, ...). New listsTrkCrvMatchRecoTagsAllandTrkCrvMatchRecoMCTagsAllname the reco-stage labels, for files whose reco already ran the association.
Prior findings (from my review at 5eb9a7ec)
- 🟢 [was 🟡 S2] The ntuple jobs recomputed the association under the reco labels, so on files carrying Production#595's products the poorer recomputed records won silently. FIXED, verified with
fhicl-dumpat this head:from_mcs-extracted,from_rec-combinedandfrom_mcs-mockdata_trkcrvschedule only*Ntupleproducers, and theirtrkCrvMatchTags/trkCrvMatchMCTagsname exactly those (TrkCrvMatchLineNtuple(MC) in the first two; the four LoopHelix*Ntuple(MC) in the third).git grepfinds the reco-stage labels only in the two newRecotag lists. A job now reads either its own records or the stored ones, by name, never one in place of the other.- On a file made after Production#595,
from_mcs-extractedstill recomputes the association, as its comment says. Reading the stored records, which keep their true-crossing deposits, means setting the tags to theRecolists. A one-line note indoc/branches.mdsaying so would help analysers pick the right one.
- ⚪ [S3, open] The per-fit
DtRefCenterandSigmaVtuning is still copied from Production's prolog, now with the matching comment. Optional, as before.
Validation check
- Build/tests run:
fhicl-dumpoffrom_mcs-extracted,from_rec-combined,from_mcs-mockdata_trkcrvandfrom_mcs-OffSpillat this head, with Production#59534d293eb, ArtAnalysis491fb22and Offlinemainca7b6580: exit status 0 for all four. No jobs were rerun: the commit only renames labels, and my runs at5eb9a7eccover the branch filling. - Config contract check: pass.
- Cross-repo consistency: pass with Production#595 in either merge order.
|
Thanks! I added the note on the reco tag lists to doc/branches.md (and the ntuplehelper text it is generated from). |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #441
Reviewed at head 54eb677825918d481acfc8087871c709e3712292. Re-review of 8d269c1.
Decision
- 🟢 approve. The new commit adds the note on choosing stored or recomputed track-CRV records, which my last review suggested. I checked it against the fcl and the data product.
What changed since 8d269c1
- One commit,
54eb677"Add documentation for new branches." (doc/branches.md+2,helper/ntuplehelper.py+1). Not a rebase. The same paragraph goes into both files, so the generated branch list matches the doc.
Prior findings (from my review at 8d269c11)
- 🟢 [suggestion] A note in
doc/branches.mdon reading the stored records. DONE, and checked against the code:TrkCrvMatchRecoTagsAllandTrkCrvMatchRecoMCTagsAll(fcl/prolog.fcl:237-238) list the four LoopHelix fits under their reco labels.TrkCrvMatchTagsAll(:234) lists the*Ntuplelabels, as the note says.- "probe 3" is
TrkCrvMatch::probeAllPulses(Offline RecoDataProducts/inc/TrkCrvMatch.hh:89).
- ⚪ [S3, open, optional] The
DtRefCenterandSigmaVtuning is still copied from Production's prolog.
Validation check
- Build/tests run: no. The commit changes only documentation and a printed string.
- Config contract check: not applicable.
- Cross-repo consistency: pass.
Summary
This PR stores the Offline track-CRV association (
TrkCrvMatch, andTrkCrvMatchMCon MC; Mu2e/Offline#2106)in the ntuple, per track branch, so analyses use one association instead of matching
crvcoincsto tracksthemselves. It adds:
utils/trkcrv_example.py;ntuplehelperentries,doc/branches.md, and the version bump to 6.17.0.Track branches without
trkCrvMatchTagsget no new branches, so every other fcl writes the same ntuple as before.Branches
For a track branch
<trk>(e.g.trk):<trk>crvTrkCrvInfobestgeom/besttimeindices into<trk>crvcands, the reference time and its surface,probe(which pulses the layer probe saw: 0 off, 1 none, 2 clustered only, 3 all)<trk>crvxingsTrkCrvXingInfosigu,sigv, correlationrhouv), edge distances, and the 4-layer probe<trk>crvcandsTrkCrvCandInfocrvcoinc(index intocrvcoincs),dtref,dtxing,du,dv, their errors and correlationrhouv, χ²,nshare, and the flags as bools<trk>crvcandsmcTrkCrvCandInfoMCcrvcandsrel: theMCRelationshipof the cluster to the track's particle (−1 none, 0 same, 1 daughter, 2 mother, 3 sibling, 4-6 theu*relations),rem,hasmc, energy from the particle, from related particles and in total<trk>crvxingsmcTrkCrvXingInfoMCcrvxings<trk>crvtruexingsmcTrkCrvTrueXingInfoMCrel,rem: 0 same, 2 mother, 5 umother, with the generation asrem), with the particle's own cluster;hasdepositsis false where the input had no CrvSteps (reco files)crvcands.crvcoincis the cluster's key in the collectioncrvcoincsis filled from; it is −1 when the matcher usedanother cluster collection (checked by ProductID), so the join cannot silently point at the wrong cluster.
Configuration
trk.fits[i].trkCrvMatchTags: theTrkCrvMatchcollections covering the KalSeeds behind that fit branch, one perKalSeed collection; empty (default) means no association branches. Tracks are matched to their record by
art::Ptr<KalSeed>, so merged and selected track collections work.trk.fits[i].trkCrvMatchMCTags: the parallelTrkCrvMatchMCcollections, for themcbranches.trk.fillTrkCrv: global switch (default true).fcl/prolog.fcl:TrkCrvMatchProducers(one OfflineTrkCrvMatcherper KalSeed collection, and itsTrkCrvMatchMCMaker), pathsTrkCrvMatch{NoField,OffSpill,All}[MC]Path, andTrkCrvMatchTagsAll/TrkCrvMatchMCTagsAll. These run the association on existing reco files. LoopHelix fits do not reach the CRV, sotheir matchers are time only, each with its own
DtRefCenter(the Production values).TrkCrvMatchCHmuruns onthe Run1 geometries, whose T1 and T2 are both CRV sector type 1, so it sets
SigmaV : [ 13 ]as Production does;TrkCrvMatchLinekeeps the Offline per-sector-type list of the extracted geometry. Reco output keeps onlyclustered CRV pulses and no CrvSteps, so these instances set
ProbeAllPulses : falseandRequireCrvSteps : false,and the records say so (
probe = 2,hasdepositsfalse).Once the reco paths schedule the matchers (Production companion PR, Mu2e/Production#595), the ntuple can
read those products by setting
trkCrvMatchTagsalone, without running the producers.fcls:
from_mcs-extracted.fcl: the line fits, with truth.from_rec-combined.fcl: the line fits on pass-1 data, on the pass-1 clusters and all pulses, so the probe is complete.from_mcs-mockdata_trkcrv.fcl(new):from_mcs-mockdata.fclplus the time-only association of thetrkbranch(KKDe, KKUe, KKDmu, KKUmu), with truth.
Tests
validation/test_fcls.pywithdatasets_MDC2025.txt: 18 of 22 pass, 3 skip for datasets marked N/A ormissing (crv_kpp, Run1B), and the RooUtil macro call fails in the harness on
maintoo:test_fcls.pypasses theshell-escaped command in
commands.txtto ROOT without a shell. The same macro run directly succeeds on the samentuple.
from_mcs-extracted.fclon MDC2025bb cosmic reco, 5,000 events: the branches are filled with truth, and everycandidate's
crvcoincresolves.from_rec-combined.fclon CRT run 125472 pass-1 reco, 5,000 events: the branches are filled, the probe included.from_mcs-mockdata_trkcrv.fclon one MDC2025au OnSpill mock-data file, 9,905 events: 31,003 tracks, every onewith its record; the four matchers' tracks all have truth.
create_val_file_rooutil.Cruns on the new ntuples (RooUtil ignores the new branches).utils/trkcrv_example.pyon all three ntuples (rerun on the build with the review changes):dv0.7 mm, bestdtref13.5 ns; probe finds a pulse in 67 % of predicted layersdtref27.0 nsdv−12.7 mm, bestdtref13.2 ns; probe 59 %validation/ntuplehelper-test.py: the new leaves resolve (the test's final error case raises a KeyError onmainas well).Not in this PR
Event,Track, output ntuple, validation histograms), and pyutilsselection helpers. pyutils already reads the branches with its standard importer.
valCompare) against amainbuild: the stock fcls are unchanged and set notrkCrvMatchTags, sotheir branches should be identical.
🤖 Generated with Claude Code