Skip to content

Track-CRV association branches (<trk>crv, <trk>crvxings, <trk>crvcands and their truth) - #441

Merged
sophiemiddleton merged 7 commits into
Mu2e:mainfrom
RobMina:trkcrv-match
Nov 11, 2026
Merged

sophiemiddleton merged 7 commits into
Mu2e:mainfrom
RobMina:trkcrv-match

Conversation

@RobMina

@RobMina RobMina commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR stores the Offline track-CRV association (TrkCrvMatch, and TrkCrvMatchMC on MC; Mu2e/Offline#2106)
in the ntuple, per track branch, so analyses use one association instead of matching crvcoincs to tracks
themselves. It adds:

  • six branches per configured track branch;
  • the configuration to fill them, and the Offline matcher modules to run on existing reco files;
  • the association in the extracted-cosmic and combined-data fcls, and a new mock-data fcl;
  • a Python example, utils/trkcrv_example.py;
  • ntuplehelper entries, doc/branches.md, and the version bump to 6.17.0.

Track branches without trkCrvMatchTags get no new branches, so every other fcl writes the same ntuple as before.

Branches

For a track branch <trk> (e.g. trk):

Branch Struct Shape Content
<trk>crv TrkCrvInfo per track counts, bestgeom / besttime indices into <trk>crvcands, the reference time and its surface, probe (which pulses the layer probe saw: 0 off, 1 none, 2 clustered only, 3 all)
<trk>crvxings TrkCrvXingInfo per track, per CRV sector crossing crossing position, direction, time, momentum, track errors in the sector plane (sigu, sigv, correlation rhouv), edge distances, and the 4-layer probe
<trk>crvcands TrkCrvCandInfo per track, per associated cluster crvcoinc (index into crvcoincs), dtref, dtxing, du, dv, their errors and correlation rhouv, χ², nshare, and the flags as bools
<trk>crvcandsmc TrkCrvCandInfoMC parallel to crvcands rel: the MCRelationship of the cluster to the track's particle (−1 none, 0 same, 1 daughter, 2 mother, 3 sibling, 4-6 the u* relations), rem, hasmc, energy from the particle, from related particles and in total
<trk>crvxingsmc TrkCrvXingInfoMC parallel to crvxings the true crossing of the same sector, or of a coplanar neighbour for a fake
<trk>crvtruexingsmc TrkCrvTrueXingInfoMC per track, per true crossing true crossings of the track's particle and its ancestors (rel, rem: 0 same, 2 mother, 5 umother, with the generation as rem), with the particle's own cluster; hasdeposits is false where the input had no CrvSteps (reco files)

crvcands.crvcoinc is the cluster's key in the collection crvcoincs is filled from; it is −1 when the matcher used
another cluster collection (checked by ProductID), so the join cannot silently point at the wrong cluster.

Configuration

  • trk.fits[i].trkCrvMatchTags: the TrkCrvMatch collections covering the KalSeeds behind that fit branch, one per
    KalSeed collection; empty (default) means no association branches. Tracks are matched to their record by
    art::Ptr<KalSeed>, so merged and selected track collections work.
  • trk.fits[i].trkCrvMatchMCTags: the parallel TrkCrvMatchMC collections, for the mc branches.
  • trk.fillTrkCrv: global switch (default true).
  • fcl/prolog.fcl: TrkCrvMatchProducers (one Offline TrkCrvMatcher per KalSeed collection, and its
    TrkCrvMatchMCMaker), paths TrkCrvMatch{NoField,OffSpill,All}[MC]Path, and TrkCrvMatchTagsAll /
    TrkCrvMatchMCTagsAll. These run the association on existing reco files. LoopHelix fits do not reach the CRV, so
    their matchers are time only, each with its own DtRefCenter (the Production values). TrkCrvMatchCHmu runs on
    the Run1 geometries, whose T1 and T2 are both CRV sector type 1, so it sets SigmaV : [ 13 ] as Production does;
    TrkCrvMatchLine keeps the Offline per-sector-type list of the extracted geometry. Reco output keeps only
    clustered CRV pulses and no CrvSteps, so these instances set ProbeAllPulses : false and RequireCrvSteps : false,
    and the records say so (probe = 2, hasdeposits false).

Once the reco paths schedule the matchers (Production companion PR, Mu2e/Production#595), the ntuple can
read those products by setting trkCrvMatchTags alone, without running the producers.

fcls:

  • from_mcs-extracted.fcl: the line fits, with truth.
  • from_rec-combined.fcl: the line fits on pass-1 data, on the pass-1 clusters and all pulses, so the probe is complete.
  • from_mcs-mockdata_trkcrv.fcl (new): from_mcs-mockdata.fcl plus the time-only association of the trk branch
    (KKDe, KKUe, KKDmu, KKUmu), with truth.

Tests

  • validation/test_fcls.py with datasets_MDC2025.txt: 18 of 22 pass, 3 skip for datasets marked N/A or
    missing (crv_kpp, Run1B), and the RooUtil macro call fails in the harness on main too: test_fcls.py passes the
    shell-escaped command in commands.txt to ROOT without a shell. The same macro run directly succeeds on the same
    ntuple.

  • from_mcs-extracted.fcl on MDC2025bb cosmic reco, 5,000 events: the branches are filled with truth, and every
    candidate's crvcoinc resolves.

  • from_rec-combined.fcl on CRT run 125472 pass-1 reco, 5,000 events: the branches are filled, the probe included.

  • from_mcs-mockdata_trkcrv.fcl on one MDC2025au OnSpill mock-data file, 9,905 events: 31,003 tracks, every one
    with its record; the four matchers' tracks all have truth.

  • create_val_file_rooutil.C runs on the new ntuples (RooUtil ignores the new branches).

  • utils/trkcrv_example.py on all three ntuples (rerun on the build with the review changes):

    Ntuple best geometric candidate is the track's particle best time candidate is the track's particle other
    extracted MC (KKLine) 96.5 % 84.9 % median dv 0.7 mm, best dtref 13.5 ns; probe finds a pulse in 67 % of predicted layers
    OnSpill mock data (LoopHelix, time only) — 62.3 % (36.7 % its parent's or a further ancestor's cluster, 0.1 % unrelated) median best dtref 27.0 ns
    CRT data (KKLine) — — median dv −12.7 mm, best dtref 13.2 ns; probe 59 %
  • validation/ntuplehelper-test.py: the new leaves resolve (the test's final error case raises a KeyError on
    main as well).

Not in this PR

  • RooUtil support for the new branches (Event, Track, output ntuple, validation histograms), and pyutils
    selection helpers. pyutils already reads the branches with its standard importer.
  • Validation (valCompare) against a main build: the stock fcls are unchanged and set no trkCrvMatchTags, so
    their branches should be identical.

🤖 Generated with Claude Code

@RobMina
RobMina marked this pull request as ready for review November 10, 2026 14:49

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #441

Reviewed at head 5eb9a7ec004c8c8ea14172f1423dd9c8313f72ab. First pass.

Decision

  • 🟢 approve. The six branches fill consistently on every sample I ran, from data, extracted MC, on-spill mock data and off-spill MC. Every fcl that does not set trkCrvMatchTags writes the same ntuple as main. One finding concerns what happens once Production#595 is in the input files (finding 1), and one is a small item (finding 2). Neither blocks.

Scope understood

  • A fit branch with trkCrvMatchTags (and optionally trkCrvMatchMCTags) gets <trk>crv, <trk>crvxings, <trk>crvcands and the three MC branches, from the Offline TrkCrvMatch and TrkCrvMatchMC records found by KalSeed Ptr.
  • The prolog adds TrkCrvMatcher and TrkCrvMatchMCMaker instances that run in the ntuple job. from_mcs-extracted, from_rec-combined and the new from_mcs-mockdata_trkcrv use them. Version 6.17.0.

Findings

  1. 🟡 [S2] Once input files carry Production#595's association, the ntuple jobs recompute it under the same module labels, and the ntuple stores the poorer recomputed records.

    • Evidence: from_mcs-extracted.fcl and from_mcs-mockdata_trkcrv.fcl add TrkCrvMatchNoFieldMCPath and TrkCrvMatchAllMCPath. These produce TrkCrvMatchLine, TrkCrvMatchDe and so on, the labels Production#595 uses. A tag with no process name then resolves to the ntuple job's own product.
    • I made off-spill reco with Production#595 (3,000 events) and ran from_mcs-OffSpill with trkCrvMatchTags : ["TrkCrvMatchCHmu"] twice, once reading the file's records and once with TrkCrvMatchOffSpillMCPath added:
      • The candidates are identical: 1,178 candidates, 664 geometric.
      • The recomputed records have probe 2 instead of 3. On this sample the layer probe still finds the same 1,535 pulses.
      • The recomputed truth has no deposits: hasdeposits 0 of 1,176 true crossings instead of 1,187 of 1,187. The 11 crossings found only through their deposits are gone, and edep sums to 0 instead of 19.5 GeV. The reco-stage maker read the digi CrvSteps, which the reco output drops (TrkCrvMatchMCNtuple sets RequireCrvSteps : false for that reason).
    • Suggested fix: on files made after Production#595, read the stored records instead of recomputing. For example, keep the recompute sequences in variants meant for older files, or give the ntuple-stage producers their own labels, so a job never silently picks one record over the other.
  2. ⚪ [S3] DtRefCenter (five values) and SigmaV : [ 13 ] are copied from Production JobConfig/reco/prolog.fcl, as the comment at fcl/prolog.fcl:206 says. Per-fit tables in Offline's CRVReco prolog, used by both repositories, would keep the two from drifting when the Run 1 tuning arrives.

Checked, no action needed

  • 🟢 No change for fcls without trkCrvMatchTags. from_mcs-mockdata (200 events), from_mcs-primary (200) and from_mcs-OffSpill (3,000), at main and at this head: the same entries, the same branches in the same order, and the same total bytes in every branch. Only the version histogram changes, from 6.16 to 6.17.

  • 🟢 The new branches are consistent on every run:

    • one entry per track;
    • ncands and nxings equal the vector sizes;
    • bestgeom and besttime are valid indices, and bestgeom points at a geometric candidate;
    • the best flag sits on the indicated candidate;
    • ngeom matches the flags;
    • the MC branches are parallel to their reco branches;
    • every crvcoinc indexes into crvcoincs.

    The runs:

    job events tracks candidates (geometric)
    from_mcs-extracted, MDC2025bb 300 205 86 (64)
    from_rec-combined, PassN#28 pass 1 on KPP run 125472 1,000 476 230 (156), probe 3
    from_mcs-mockdata_trkcrv, ensembleMDS3dOnSpill 200 321 91 time-only
    off-spill reco with Production#595, validation setting 9,314 4,876 3,242 (2,800)
  • 🟢 The new branches add 0.2% (on-spill mock data) to 2.2% (off-spill) of the ntuple size.

  • 🟢 utils/trkcrv_example.py runs under pyenv ana on all four ntuples and writes its plot. On the off-spill MC, the best geometric candidate is the track's own cluster for every track, and the best time candidate for 99.4%.

  • 🟢 The dictionaries load: I read every new leaf with PyROOT after loading libeventntuple_dict.

Validation check

  • Build/tests run: yes. This head and main f745427, each with ArtAnalysis 491fb22, on the CI build of Offline main ca7b6580 (Production#595 and Offline#2107 in the head workspace). Both built with exit status 0 and no warnings. All the jobs above exited with status 0. The PR merges cleanly with main.
  • Config contract check: pass. Mismatched trkCrvMatchMCTags throw at construction, and a missing collection throws through getValidHandle.
  • Cross-repo consistency: needs Offline#2106 (merged); it does not need Production#595. See finding 1 for after Production#595 merges.

Residual risk

  • from_rec-combined was run on 1,000 events of one KPP subrun. The CRV-tracker time offset on data is whatever the matcher's configuration gives; I did not check it against data.

Author follow-ups

  1. Finding 1: read the stored association on files that have it.
  2. Optional: one home for the per-fit tuning (finding 2).

@RobMina

RobMina commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks @oksuzian for the review.

1. Recomputing under the reco labels. Fixed in 8d269c1:

  • The ntuple-stage producers are now TrkCrvMatch<fit>Ntuple, with truth TrkCrvMatch<fit>NtupleMC. This covers the
    paths, TrkCrvMatchTagsAll / TrkCrvMatchMCTagsAll, from_mcs-extracted.fcl, from_rec-combined.fcl and
    from_mcs-mockdata_trkcrv.fcl. A job can no longer pick up its own recomputed records in place of the stored ones.
  • On files that already carry the association, a fit branch sets trkCrvMatchTags to the reco labels and needs no
    producer. TrkCrvMatchRecoTagsAll / TrkCrvMatchRecoMCTagsAll give them for the LoopHelix fits.

Tests, on the rebuilt area:

  • from_mcs-extracted, from_rec-combined and from_mcs-mockdata_trkcrv with the new labels reproduce every number
    in the PR's Tests section (5,000, 5,000 and 9,905 events; trkcrv_example.py output identical).

  • Off-spill recoMC with Production#595 (2,000 events of the nightly input, CHFilter passing every event), then
    from_mcs-OffSpill with TrkCrvMatchOffSpillMCPath scheduled in both jobs:

    trkCrvMatchTags tracks probe candidates true crossings with deposits
    TrkCrvMatchCHmu (stored) 259 3 for all 105 104 104
    TrkCrvMatchCHmuNtuple (recomputed in the job) 259 2 for all 105 103 0

    With both in the same job, the stored records are the ones read. The recomputed ones differ as you found: the probe
    saw only the clustered pulses, the truth has no deposits, and the one crossing found only through its deposits is
    missing.

2. One home for the per-fit tuning. Agreed. I'll make per-fit TrkCrvMatcher tables in Offline's CRVReco
prolog, for both repositories, part of the Offline follow-up that also makes the matcher refuse a geometry whose
sector types its resolution lists don't describe (Production#595 discussion).

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #441

Reviewed at head 8d269c11442a3853948e80e2c836863e7b81f2ba. Re-review of 5eb9a7e.

Decision

  • 🟢 approve. The new commit fixes my finding 1: the ntuple-stage matchers now have their own labels, so they can no longer shadow the association that Production#595 writes into reco files. Every fcl that uses them resolves to the new labels, and none still names the old ones.

What changed since 5eb9a7e

  • One commit, 8d269c1 "Address review comments." (fcl/prolog.fcl, from_mcs-extracted.fcl, from_rec-combined.fcl; +30 −26). Not a rebase.
  • Every producer in TrkCrvMatchProducers and every path and tag list gains the suffix Ntuple (TrkCrvMatchLineNtuple, TrkCrvMatchDeNtupleMC, ...). New lists TrkCrvMatchRecoTagsAll and TrkCrvMatchRecoMCTagsAll name the reco-stage labels, for files whose reco already ran the association.

Prior findings (from my review at 5eb9a7ec)

  • 🟢 [was 🟡 S2] The ntuple jobs recomputed the association under the reco labels, so on files carrying Production#595's products the poorer recomputed records won silently. FIXED, verified with fhicl-dump at this head:
    • from_mcs-extracted, from_rec-combined and from_mcs-mockdata_trkcrv schedule only *Ntuple producers, and their trkCrvMatchTags / trkCrvMatchMCTags name exactly those (TrkCrvMatchLineNtuple(MC) in the first two; the four LoopHelix *Ntuple(MC) in the third).
    • git grep finds the reco-stage labels only in the two new Reco tag lists. A job now reads either its own records or the stored ones, by name, never one in place of the other.
    • On a file made after Production#595, from_mcs-extracted still recomputes the association, as its comment says. Reading the stored records, which keep their true-crossing deposits, means setting the tags to the Reco lists. A one-line note in doc/branches.md saying so would help analysers pick the right one.
  • ⚪ [S3, open] The per-fit DtRefCenter and SigmaV tuning is still copied from Production's prolog, now with the matching comment. Optional, as before.

Validation check

  • Build/tests run: fhicl-dump of from_mcs-extracted, from_rec-combined, from_mcs-mockdata_trkcrv and from_mcs-OffSpill at this head, with Production#595 34d293eb, ArtAnalysis 491fb22 and Offline main ca7b6580: exit status 0 for all four. No jobs were rerun: the commit only renames labels, and my runs at 5eb9a7ec cover the branch filling.
  • Config contract check: pass.
  • Cross-repo consistency: pass with Production#595 in either merge order.

@RobMina

RobMina commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Thanks! I added the note on the reco tag lists to doc/branches.md (and the ntuplehelper text it is generated from).

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #441

Reviewed at head 54eb677825918d481acfc8087871c709e3712292. Re-review of 8d269c1.

Decision

  • 🟢 approve. The new commit adds the note on choosing stored or recomputed track-CRV records, which my last review suggested. I checked it against the fcl and the data product.

What changed since 8d269c1

  • One commit, 54eb677 "Add documentation for new branches." (doc/branches.md +2, helper/ntuplehelper.py +1). Not a rebase. The same paragraph goes into both files, so the generated branch list matches the doc.

Prior findings (from my review at 8d269c11)

  • 🟢 [suggestion] A note in doc/branches.md on reading the stored records. DONE, and checked against the code:
    • TrkCrvMatchRecoTagsAll and TrkCrvMatchRecoMCTagsAll (fcl/prolog.fcl:237-238) list the four LoopHelix fits under their reco labels. TrkCrvMatchTagsAll (:234) lists the *Ntuple labels, as the note says.
    • "probe 3" is TrkCrvMatch::probeAllPulses (Offline RecoDataProducts/inc/TrkCrvMatch.hh:89).
  • ⚪ [S3, open, optional] The DtRefCenter and SigmaV tuning is still copied from Production's prolog.

Validation check

  • Build/tests run: no. The commit changes only documentation and a printed string.
  • Config contract check: not applicable.
  • Cross-repo consistency: pass.

@sophiemiddleton
sophiemiddleton merged commit 428345c into Mu2e:main Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants