Skip to content

Validation histograms for the track-CRV association (ValTrkCrvMatch, ValTrkCrvMatchMC) - #2107

Merged
oksuzian merged 5 commits into
Mu2e:mainfrom
RobMina:trkcrv-match-val
Nov 11, 2026
Merged

oksuzian merged 5 commits into
Mu2e:mainfrom
RobMina:trkcrv-match-val

Conversation

@RobMina

@RobMina RobMina commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds validation histograms for the TrkCrvMatch and TrkCrvMatchMC products of the track-CRV association
PR (#2106), so that the nightly validation and the PassN Validation/val step monitor the association. It
depends on that PR and changes nothing else.

The Validation module fills one histogram set per product instance it finds in the event, like every other
Val* class, so no configuration is needed: any reco job that runs TrkCrvMatcher (and TrkCrvMatchMCMaker on MC)
gets the histograms.

What is added

File Content
Validation/inc/ValTrkCrvMatch.hh, Validation/src/ValTrkCrvMatch.cc reco histograms
Validation/inc/ValTrkCrvMatchMC.hh, Validation/src/ValTrkCrvMatchMC.cc truth histograms
Validation/src/Validation_module.cc the two classes added to the product loop
Validation/CMakeLists.txt, Validation/src/SConscript the two sources, and the CRV geometry library (shield names)

ValTrkCrvMatch (reco):

  • counts per track: tracks, candidates, geometric candidates, CRV crossings; the nShare distribution and the
    candidate flag bits;
  • time: dtRef for all candidates and for the best one, and dtRefToF;
  • per shield, for the best candidate in each sector: dtXing, du and dv; and its χ²/ndof;
  • per shield, the profile of dv against the across-bar track slope. A slope-dependent dv is a normal-direction
    offset between tracker and CRV; the mean dv is the in-plane offset. These double as an alignment monitor;
  • crossings per shield, and those with a geometric candidate in their shield;
  • the shield of the reference crossing against the shield of the best geometric candidate's cluster;
  • the layer probe: predicted bar, pulse found and pulse in a cluster per layer, the bar offset of the selected
    pulse, its time difference, PEs, and the two-end position against the prediction.

ValTrkCrvMatchMC (truth):

  • the truth relation (same particle, same family, same primary, unrelated, background, no MC) of the best geometric
    candidate, the best time candidate, and all geometric and time-only candidates;
  • per shield: the track particle's own cluster, whether it is a geometric candidate and whether it is the best in
    its sector;
  • per shield: reco crossings and those with a true crossing; true crossings and those the extrapolation found;
  • reco − true crossing distance and time per shield.

Tests

On the build of both branches:

  • MC: MDC2025bb cosmic reco (KKLine, extracted v06 geometry), 30,311 events: both classes fill. The truth
    histograms reproduce the association study (best geometric candidate is the track's own particle in 97.45 % of
    tracks).
  • Data: CRT combined run 125472, file 12, 27,142 events: ValTrkCrvMatch fills, layer probe included.

Notes

@FNALbuild

Copy link
Copy Markdown
Collaborator

Hi @RobMina,
You have proposed changes to files in these packages:

  • CRVReco
  • RecoDataProducts
  • MCDataProducts
  • CRVResponse
  • Validation

which require these tests: build.

@Mu2e/write, @Mu2e/fnalbuild-users have access to CI actions on main.

⌛ The following tests have been triggered for 468f855: build (Build queue - API unavailable)

About FNALbuild. Code review on Mu2e/Offline.

@FNALbuild

Copy link
Copy Markdown
Collaborator

☀️ The build tests passed at 468f855.

Test Result Details
test with ✅ Command did not list any other PRs to include
merge ✅ Merged 468f855 at b24d416
build (prof) ✅ Log file. Build time: 04 min 35 sec
ceSimReco ✅ Log file.
g4test_03MT ✅ Log file.
transportOnly ✅ Log file.
POT ✅ Log file.
g4study ✅ Log file.
cosmicSimReco ✅ Log file.
cosmicOffSpill ✅ Log file.
ceSteps ✅ Log file.
ceDigi ✅ Log file.
muDauSteps ✅ Log file.
ceMix ✅ Log file.
rootOverlaps ✅ Log file.
g4surfaceCheck ✅ Log file.
trigger ✅ Log file.
check_cmake ✅ Log file.
FIXME, TODO ✅ TODO (0) FIXME (0) in 12 files
clang-tidy ➡️ 6 errors 222 warnings
whitespace check ✅ no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 468f855 after being merged into the base branch at b24d416.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

@RobMina
RobMina force-pushed the trkcrv-match-val branch 2 times, most recently from b891442 to 683c78d Compare November 6, 2026 21:40
@brownd1978

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

1 similar comment
@brownd1978

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

@FNALbuild

Copy link
Copy Markdown
Collaborator

⌛ The following tests have been triggered for 6b0bc14: build (Build queue - API unavailable)

@FNALbuild

Copy link
Copy Markdown
Collaborator

☀️ The build tests passed at 6b0bc14.

Test Result Details
test with ✅ Command did not list any other PRs to include
merge ✅ Merged 6b0bc14 at b6462a0
build (prof) ✅ Log file. Build time: 04 min 40 sec
ceSimReco ✅ Log file.
g4test_03MT ✅ Log file.
transportOnly ✅ Log file.
POT ✅ Log file.
g4study ✅ Log file.
cosmicSimReco ✅ Log file.
cosmicOffSpill ✅ Log file.
ceSteps ✅ Log file.
ceDigi ✅ Log file.
muDauSteps ✅ Log file.
ceMix ✅ Log file.
rootOverlaps ✅ Log file.
g4surfaceCheck ✅ Log file.
trigger ✅ Log file.
check_cmake ✅ Log file.
FIXME, TODO ✅ TODO (0) FIXME (0) in 5 files
clang-tidy ➡️ 3 errors 112 warnings
whitespace check ✅ no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 6b0bc14 after being merged into the base branch at b6462a0.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

@RobMina
RobMina marked this pull request as ready for review November 10, 2026 14:04

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #2107

Reviewed at head 91c901679ee06a6110937f0f6aebe2099e6c89e0. First pass.

Decision

  • 🟡 comment now, approve when green. I would approve this code. The two classes fill sensible histograms from the products of #2106, and on files without those products nothing changes. One finding: the per-shield histograms take their shield list and directions from the geometry of the validation job, not of the job that made the product, so val.fcl on an extracted-geometry file labels and drops shields wrongly (finding 1). It does not block. FNALbuild has not built this head (mu2e/buildtest has no status at 91c90167, checked 15:42 UTC). Build 3572 passed at 6b0bc140, and the new head only merges main: git diff 6b0bc140 91c90167 -- Validation is empty. I am triggering a build and will approve once it is green.

Scope understood

  • ValTrkCrvMatch and ValTrkCrvMatchMC histogram every TrkCrvMatch and TrkCrvMatchMC instance in the event, through the existing analyzeProduct loop. No configuration is needed; the only build change is the CosmicRayShieldGeom dependency.

Findings

  1. 🟡 [S2] The per-shield histograms use the validation job's geometry, so they are wrong for any file made with another geometry.
    • Evidence: ValTrkCrvMatch::declare (ValTrkCrvMatch.cc:9-23) and ValTrkCrvMatchMC::declare (:7-8) size the shield axes, name the dvVsSlope_<shield> profiles and take the across-bar and normal directions from GeomHandle<CosmicRayShield> and GeomHandle<KinKalGeom> of the job running Validation. The product's sector_ is the shield index of the geometry that made it. Offline/Validation/fcl/val.fcl uses the default geometry (Run 1A, 2 shields), and codetools/bin/valJobNode.sh:266 runs it with no geometry override.
    • I wrote TrkCrvMatchLine for 300 events of an MDC2025bb extracted file (11 shields) and ran val.fcl on it twice:
      • with the default geometry, XingShield has 2 bins, 19 of its 86 entries land in the overflow, dv keeps 53 of 64 entries, and the EX sector's crossings fill dvVsSlope_CRV_T1, projected with the Run 1A T1 directions;
      • with geom_cosmic_extracted.txt, all 11 shields are binned and named correctly.
    • Impact: Production's extracted nightly configuration (Validation/nightly/extracted_*.fcl, recoMC/Extracted.fcl) keeps TrkCrvMatchLine once Production#595 merges. If its validation step runs val.fcl as valJobNode.sh does, its per-shield histograms will be mislabelled and truncated, with no message.
    • Suggested fix: make the histograms independent of the validation job's geometry, for example by binning on the product's own sector index up to a fixed maximum, with the directions taken from the product. Alternatively, fail when a sector_ does not exist in the job geometry instead of dropping it silently.

Checked, no action needed

  • 🟢 The histograms are filled and sensible on Production#595 off-spill reco output (3,000 events, production setting), with val.fcl and the matching Run 1A geometry. TrkCrvMatchCHmu: 1,733 tracks, 1,178 candidates, 664 geometric. The best-in-sector dv has mean −5.8 mm and dtXing RMS 3.1 ns. PredMatched is diagonal (661 and 2, with 1 off-diagonal). TrkCrvMatchCHmuMC: BestGeomRel is "same" for all 664, and XingDt has RMS 2.1 ns. The four LoopHelix instances fill their time-only histograms. Filled the same way on the on-spill ceMix reco output.
  • 🟢 Files without the products get no new directories: val.fcl at this head on the output of the committed nightly off-spill configuration has no TrkCrv directory.
  • 🟢 The header guards match the Val* siblings, and the GeomHandle use has precedent in ValHelixSeed and ValStrawHit.

Validation check

  • Build/tests run: yes. A muse build of this head on the CI build of main ca7b6580 (with Production#595 and EventNtuple#441) exited with status 0 and no warnings. The jobs above all exited with status 0. FNALbuild build 3572 (at 6b0bc140, merged on b6462a04) passed every test.
  • Config contract check: not applicable, no fcl changes.
  • Cross-repo consistency: the histograms appear only where reco output keeps the products. As committed, Production#595's off-spill nightly output does not keep them; I raised that on #595.

Residual risk

  • Not checked: the PassN Validation/val step.

Author follow-ups

  1. Finding 1: make the shield binning independent of the validation job's geometry, or fail loudly on a mismatch.

@oksuzian

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

@FNALbuild

Copy link
Copy Markdown
Collaborator

⌛ The following tests have been triggered for 91c9016: build (Build queue - API unavailable)

@FNALbuild

Copy link
Copy Markdown
Collaborator

☀️ The build tests passed at 91c9016.

Test Result Details
test with ✅ Command did not list any other PRs to include
merge ✅ Merged 91c9016 at 152833a
build (prof) ✅ Log file. Build time: 04 min 37 sec
ceSimReco ✅ Log file.
g4test_03MT ✅ Log file.
transportOnly ✅ Log file.
POT ✅ Log file.
g4study ✅ Log file.
cosmicSimReco ✅ Log file.
cosmicOffSpill ✅ Log file.
ceSteps ✅ Log file.
ceDigi ✅ Log file.
muDauSteps ✅ Log file.
ceMix ✅ Log file.
rootOverlaps ✅ Log file.
g4surfaceCheck ✅ Log file.
trigger ❓ Log file. Return Code 1.
check_cmake ✅ Log file.
FIXME, TODO ✅ TODO (0) FIXME (0) in 5 files
clang-tidy ➡️ 3 errors 112 warnings
whitespace check ✅ no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 91c9016 after being merged into the base branch at 152833a.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

@RobMina

RobMina commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @oksuzian, good catch. Fixed in 1886f04: the per-sector histograms no longer take their axes from the
validation job's geometry.

  • One fixed axis for every per-sector histogram (Validation/inc/ValTrkCrvSectorAxis.hh). There is one labelled
    bin per CRV sector SurfaceId (CRV_EX ... CRV_M8), from SurfaceIdDetail::names(), and a last bin, unknown.
    • Crossings are binned by their own sid_. Clusters are binned by the SurfaceId of the product's crossings with
      the same shield index, so both come from the job that made the product.
    • ValTrkCrvMatchMC needs no geometry at all now. CosmicRayShieldGeom is gone from the build files.
  • dvVsSlope_<sector> needs the sector's across-bar direction and normal, which only a geometry gives. There is
    one profile per CRV sector of the validation job's geometry.
    • A candidate is filled only when that geometry's plane of the reference crossing's sector holds the crossing, to
      within 1 mm.
    • Otherwise it goes to a new SlopeNoGeom histogram, by sector, so a geometry mismatch shows there and nothing is
      dropped or projected with the wrong directions.

Test: recoMC/Extracted.fcl on 300 events of MDC2025bb extracted digis (v06 geometry), then val.fcl on the output
twice:

default geometry (Run 1A) geom_common_extracted_v06.txt
XingShield 104: EX 66, T1 16, T2 16, M2 1, M5 2, M6 1, M8 2 identical
dv (best in sector) 92: EX 60, T1 12, T2 15, M5 4, unknown 1 identical
PredMatched, MC RecoXing, TrueXing, TrueClu identical
dvVsSlope_* T1 and T2 only, both empty EX 60, T1 12, T2 15, M5 4
SlopeNoGeom 92: EX 60, T1 12, T2 15, M5 4, unknown 1 1: unknown

So the binning is the same whichever geometry the validation job uses. With the default geometry, T1 and T2 exist but
sit elsewhere, so the plane check sends their candidates to SlopeNoGeom. The one unknown candidate is a cluster in
a shield that no crossing of the product has crossed yet.

The off-spill nightly output now keeps the products (Production#595), and val.fcl on it fills the TrkCrv
directories: on 300 events of the nightly off-spill input, all five matchers and their truth, with KKCHmu's 8 crossings in
CRV_T1 (7) and CRV_T2 (1), its 4 best-in-sector candidates in dvVsSlope_CRV_T1, and SlopeNoGeom empty.

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #2107

Reviewed at head 1886f04beedaa5e4bf8b719687f8ed00ff9aa204. Re-review of 91c9016.

Decision

  • 🟡 comment now, approve when green. The one finding from my first pass is fixed, and I checked the fix by rerunning the test that found it. One nit is left, about the PR description. FNALbuild has not built this head: mu2e/buildtest has no status at 1886f04b (checked 21:32 UTC). A build will be triggered, and I will approve once it is green.

Scope understood

  • Commit 1886f04b replaces the shield axes of the per-sector histograms with one fixed, labelled axis (ValTrkCrvSectorAxis). It has one bin per CRV sector SurfaceId (CRV_EX to CRV_M8, 34 bins) and a last bin, unknown. Crossings are binned by their own sid_. Clusters are binned through the crossings with the same shield index.
  • dvVsSlope_<sector> still needs the validation job's geometry for its directions. A candidate is filled only if that geometry's plane holds the reference crossing to within 1 mm. Otherwise it goes to a new SlopeNoGeom histogram.
  • CosmicRayShieldGeom is removed from both CMakeLists.txt and SConscript. Against main, the PR now only adds the two sources to CMakeLists.txt and does not touch SConscript. Nothing in Validation/ uses CosmicRayShield any more.

Findings

  1. ⚪ [S3] The PR description is out of date. Its file table still says the build files add "the CRV geometry library (shield names)", and it describes the histograms as "per shield". Neither is true at 1886f04b. Please update it, since it becomes the record of the merge.

Checked, no action needed

  • 🟢 [was S2] The per-sector histograms used the validation job's geometry — FIXED in 1886f04b, verified. I reran val.fcl on the same input as in my first pass: TrkCrvMatchLine written for 300 events of an MDC2025bb extracted-geometry file (11 sectors).
    • With the default geometry (Run 1A, as valJobNode.sh runs it), XingShield has all 86 crossings in named bins and none in overflow (before: 19 of 86 in overflow). The bins are CRV_EX 54, CRV_T1 13, CRV_T2 13, CRV_M1 2, CRV_M3 1, CRV_M7 1 and CRV_M8 2. dv keeps 64 of 64 entries (before: 53). The CRV_EX crossings are no longer filled as CRV_T1.
    • With geom_cosmic_extracted.txt, every per-sector histogram is identical to the default-geometry run. Only dvVsSlope_* and SlopeNoGeom differ, as intended.
    • With the default geometry, the Run 1A CRV_T1 and CRV_T2 planes do not hold the extracted crossings. So both profiles stay empty and all 64 candidates go to SlopeNoGeom. Nothing is projected with the wrong directions. With the extracted geometry, the profiles hold CRV_EX 43, CRV_T1 10, CRV_T2 9 and CRV_M8 1.
    • Against the first-pass output made with the extracted geometry, 45 of 49 histograms match bin for bin, after mapping shield indices to sector names. The other 4 differ by one candidate: a cluster in CRV_M2, a sector no crossing in the file reaches. It now goes to unknown (in dv, dtXing, PredMatched) and to SlopeNoGeom, as your reply says.
  • 🟢 A sector outside the axis is not dropped silently. It is counted in the labelled unknown bin, which every per-sector histogram has. A geometry mismatch for the profiles shows up in SlopeNoGeom.
  • 🟢 Run 1A off-spill output is unchanged apart from the relabelling. On the Production#595 off-spill reco output from my first pass (3,000 events, five matcher instances and their truth), all 200 histograms of the first pass are present. 125 are identical and 75 are identical after mapping shield 0 to CRV_T1 and shield 1 to CRV_T2. None differ. The five new SlopeNoGeom histograms are empty, and no entry is in unknown. For example, TrkCrvMatchCHmu XingShield is CRV_T1 840, CRV_T2 6.

Validation check

  • Build/tests run: yes. I merged 1886f04b into main 7fb240fc (clean) and ran muse build on the CI build main/7fb240fc_ec057b9e_5999ffa2. It exited with status 0 and no warnings. I did not build with CMake. The three val.fcl jobs above exited with status 0. FNALbuild build 3593 passed at the previous head 91c90167 (merged at 152833a9), except trigger, which is ":question: Return Code 1" on every build against current main since Offline#2129, so not caused by this PR.
  • Config contract check: not applicable, no fcl changes.
  • Cross-repo consistency: pass. No other repo needs a change.

Residual risk

  • A cluster's sector is learned from the crossings seen so far in the job. So a cluster in a sector no crossing has reached goes to unknown (1 of 64 here). A job that reads files made with different geometries could also carry the shield-to-sector mapping of one file into the next. I did not test that case; the nightly and PassN validation jobs read one configuration each.
  • Not checked: the PassN Validation/val step.

Author follow-ups

  1. Update the PR description (finding 1).

@oksuzian

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

@FNALbuild

Copy link
Copy Markdown
Collaborator

⌛ The following tests have been triggered for 1886f04: build (Build queue - API unavailable)

@FNALbuild

Copy link
Copy Markdown
Collaborator

☀️ The build tests passed at 1886f04.

Test Result Details
test with ✅ Command did not list any other PRs to include
merge ✅ Merged 1886f04 at 7fb240f
build (prof) ✅ Log file. Build time: 04 min 36 sec
ceSimReco ✅ Log file.
g4test_03MT ✅ Log file.
transportOnly ✅ Log file.
POT ✅ Log file.
g4study ✅ Log file.
cosmicSimReco ✅ Log file.
cosmicOffSpill ✅ Log file.
ceSteps ✅ Log file.
ceDigi ✅ Log file.
muDauSteps ✅ Log file.
ceMix ✅ Log file.
rootOverlaps ✅ Log file.
g4surfaceCheck ✅ Log file.
trigger ✅ Log file.
check_cmake ✅ Log file.
FIXME, TODO ✅ TODO (0) FIXME (0) in 6 files
clang-tidy ➡️ 2 errors 96 warnings
whitespace check ✅ no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 1886f04 after being merged into the base branch at 7fb240f.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #2107

Reviewed at head 1886f04beedaa5e4bf8b719687f8ed00ff9aa204. Follow-up to my comment review at the same head.

Decision

  • 🟢 approve. FNALbuild build 3600 has now built this head, merged on main 7fb240fc, and every test passed: the prof build, all eleven validation jobs, rootOverlaps, g4surfaceCheck, trigger, check_cmake and the whitespace check. The FIXME/TODO count and clang-tidy rows are informational; both clang-tidy errors are in cetlib's LibraryManager.h, not in this PR. This is the approval my comment review promised once the build was green. Finding 1, the out-of-date PR description, still stands and does not block.

@oksuzian
oksuzian merged commit 30088f3 into Mu2e:main Nov 11, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants