Repository navigation
Validation histograms for the track-CRV association (ValTrkCrvMatch, ValTrkCrvMatchMC) - #2107
Conversation
|
Hi @RobMina,
which require these tests: build. @Mu2e/write, @Mu2e/fnalbuild-users have access to CI actions on main. ⌛ The following tests have been triggered for 468f855: build (Build queue - API unavailable) |
|
☀️ The build tests passed at 468f855.
N.B. These results were obtained from a build of this Pull Request at 468f855 after being merged into the base branch at b24d416. For more information, please check the job page here. |
b891442 to
683c78d
Compare
683c78d to
6b0bc14
Compare
|
@FNALbuild run build test |
1 similar comment
|
@FNALbuild run build test |
|
⌛ The following tests have been triggered for 6b0bc14: build (Build queue - API unavailable) |
|
☀️ The build tests passed at 6b0bc14.
N.B. These results were obtained from a build of this Pull Request at 6b0bc14 after being merged into the base branch at b6462a0. For more information, please check the job page here. |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #2107
Reviewed at head 91c901679ee06a6110937f0f6aebe2099e6c89e0. First pass.
Decision
- 🟡 comment now, approve when green. I would approve this code. The two classes fill sensible histograms from the products of #2106, and on files without those products nothing changes. One finding: the per-shield histograms take their shield list and directions from the geometry of the validation job, not of the job that made the product, so
val.fclon an extracted-geometry file labels and drops shields wrongly (finding 1). It does not block. FNALbuild has not built this head (mu2e/buildtesthas no status at91c90167, checked 15:42 UTC). Build 3572 passed at6b0bc140, and the new head only mergesmain:git diff 6b0bc140 91c90167 -- Validationis empty. I am triggering a build and will approve once it is green.
Scope understood
ValTrkCrvMatchandValTrkCrvMatchMChistogram everyTrkCrvMatchandTrkCrvMatchMCinstance in the event, through the existinganalyzeProductloop. No configuration is needed; the only build change is theCosmicRayShieldGeomdependency.
Findings
- 🟡 [S2] The per-shield histograms use the validation job's geometry, so they are wrong for any file made with another geometry.
- Evidence:
ValTrkCrvMatch::declare(ValTrkCrvMatch.cc:9-23) andValTrkCrvMatchMC::declare(:7-8) size the shield axes, name thedvVsSlope_<shield>profiles and take the across-bar and normal directions fromGeomHandle<CosmicRayShield>andGeomHandle<KinKalGeom>of the job runningValidation. The product'ssector_is the shield index of the geometry that made it.Offline/Validation/fcl/val.fcluses the default geometry (Run 1A, 2 shields), andcodetools/bin/valJobNode.sh:266runs it with no geometry override. - I wrote
TrkCrvMatchLinefor 300 events of an MDC2025bb extracted file (11 shields) and ranval.fclon it twice:- with the default geometry,
XingShieldhas 2 bins, 19 of its 86 entries land in the overflow,dvkeeps 53 of 64 entries, and the EX sector's crossings filldvVsSlope_CRV_T1, projected with the Run 1A T1 directions; - with
geom_cosmic_extracted.txt, all 11 shields are binned and named correctly.
- with the default geometry,
- Impact: Production's extracted nightly configuration (
Validation/nightly/extracted_*.fcl,recoMC/Extracted.fcl) keepsTrkCrvMatchLineonce Production#595 merges. If its validation step runsval.fclasvalJobNode.shdoes, its per-shield histograms will be mislabelled and truncated, with no message. - Suggested fix: make the histograms independent of the validation job's geometry, for example by binning on the product's own sector index up to a fixed maximum, with the directions taken from the product. Alternatively, fail when a
sector_does not exist in the job geometry instead of dropping it silently.
- Evidence:
Checked, no action needed
- 🟢 The histograms are filled and sensible on Production#595 off-spill reco output (3,000 events, production setting), with
val.fcland the matching Run 1A geometry.TrkCrvMatchCHmu: 1,733 tracks, 1,178 candidates, 664 geometric. The best-in-sectordvhas mean −5.8 mm anddtXingRMS 3.1 ns.PredMatchedis diagonal (661 and 2, with 1 off-diagonal).TrkCrvMatchCHmuMC:BestGeomRelis "same" for all 664, andXingDthas RMS 2.1 ns. The four LoopHelix instances fill their time-only histograms. Filled the same way on the on-spillceMixreco output. - 🟢 Files without the products get no new directories:
val.fclat this head on the output of the committed nightly off-spill configuration has noTrkCrvdirectory. - 🟢 The header guards match the
Val*siblings, and theGeomHandleuse has precedent inValHelixSeedandValStrawHit.
Validation check
- Build/tests run: yes. A
muse buildof this head on the CI build ofmainca7b6580(with Production#595 and EventNtuple#441) exited with status 0 and no warnings. The jobs above all exited with status 0. FNALbuild build 3572 (at6b0bc140, merged onb6462a04) passed every test. - Config contract check: not applicable, no fcl changes.
- Cross-repo consistency: the histograms appear only where reco output keeps the products. As committed, Production#595's off-spill nightly output does not keep them; I raised that on #595.
Residual risk
- Not checked: the PassN
Validation/valstep.
Author follow-ups
- Finding 1: make the shield binning independent of the validation job's geometry, or fail loudly on a mismatch.
|
@FNALbuild run build test |
|
⌛ The following tests have been triggered for 91c9016: build (Build queue - API unavailable) |
|
☀️ The build tests passed at 91c9016.
N.B. These results were obtained from a build of this Pull Request at 91c9016 after being merged into the base branch at 152833a. For more information, please check the job page here. |
|
Thanks @oksuzian, good catch. Fixed in 1886f04: the per-sector histograms no longer take their axes from the
Test:
So the binning is the same whichever geometry the validation job uses. With the default geometry, T1 and T2 exist but The off-spill nightly output now keeps the products (Production#595), and |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #2107
Reviewed at head 1886f04beedaa5e4bf8b719687f8ed00ff9aa204. Re-review of 91c9016.
Decision
- 🟡 comment now, approve when green. The one finding from my first pass is fixed, and I checked the fix by rerunning the test that found it. One nit is left, about the PR description. FNALbuild has not built this head:
mu2e/buildtesthas no status at1886f04b(checked 21:32 UTC). A build will be triggered, and I will approve once it is green.
Scope understood
- Commit
1886f04breplaces the shield axes of the per-sector histograms with one fixed, labelled axis (ValTrkCrvSectorAxis). It has one bin per CRV sectorSurfaceId(CRV_EXtoCRV_M8, 34 bins) and a last bin,unknown. Crossings are binned by their ownsid_. Clusters are binned through the crossings with the same shield index. dvVsSlope_<sector>still needs the validation job's geometry for its directions. A candidate is filled only if that geometry's plane holds the reference crossing to within 1 mm. Otherwise it goes to a newSlopeNoGeomhistogram.CosmicRayShieldGeomis removed from bothCMakeLists.txtandSConscript. Againstmain, the PR now only adds the two sources toCMakeLists.txtand does not touchSConscript. Nothing inValidation/usesCosmicRayShieldany more.
Findings
- ⚪ [S3] The PR description is out of date. Its file table still says the build files add "the CRV geometry library (shield names)", and it describes the histograms as "per shield". Neither is true at
1886f04b. Please update it, since it becomes the record of the merge.
Checked, no action needed
- 🟢 [was S2] The per-sector histograms used the validation job's geometry — FIXED in
1886f04b, verified. I reranval.fclon the same input as in my first pass:TrkCrvMatchLinewritten for 300 events of an MDC2025bb extracted-geometry file (11 sectors).- With the default geometry (Run 1A, as
valJobNode.shruns it),XingShieldhas all 86 crossings in named bins and none in overflow (before: 19 of 86 in overflow). The bins areCRV_EX54,CRV_T113,CRV_T213,CRV_M12,CRV_M31,CRV_M71 andCRV_M82.dvkeeps 64 of 64 entries (before: 53). TheCRV_EXcrossings are no longer filled asCRV_T1. - With
geom_cosmic_extracted.txt, every per-sector histogram is identical to the default-geometry run. OnlydvVsSlope_*andSlopeNoGeomdiffer, as intended. - With the default geometry, the Run 1A
CRV_T1andCRV_T2planes do not hold the extracted crossings. So both profiles stay empty and all 64 candidates go toSlopeNoGeom. Nothing is projected with the wrong directions. With the extracted geometry, the profiles holdCRV_EX43,CRV_T110,CRV_T29 andCRV_M81. - Against the first-pass output made with the extracted geometry, 45 of 49 histograms match bin for bin, after mapping shield indices to sector names. The other 4 differ by one candidate: a cluster in
CRV_M2, a sector no crossing in the file reaches. It now goes tounknown(indv,dtXing,PredMatched) and toSlopeNoGeom, as your reply says.
- With the default geometry (Run 1A, as
- 🟢 A sector outside the axis is not dropped silently. It is counted in the labelled
unknownbin, which every per-sector histogram has. A geometry mismatch for the profiles shows up inSlopeNoGeom. - 🟢 Run 1A off-spill output is unchanged apart from the relabelling. On the Production#595 off-spill reco output from my first pass (3,000 events, five matcher instances and their truth), all 200 histograms of the first pass are present. 125 are identical and 75 are identical after mapping shield 0 to
CRV_T1and shield 1 toCRV_T2. None differ. The five newSlopeNoGeomhistograms are empty, and no entry is inunknown. For example,TrkCrvMatchCHmuXingShieldisCRV_T1840,CRV_T26.
Validation check
- Build/tests run: yes. I merged
1886f04bintomain7fb240fc(clean) and ranmuse buildon the CI buildmain/7fb240fc_ec057b9e_5999ffa2. It exited with status 0 and no warnings. I did not build with CMake. The threeval.fcljobs above exited with status 0. FNALbuild build 3593 passed at the previous head91c90167(merged at152833a9), excepttrigger, which is ":question: Return Code 1" on every build against currentmainsince Offline#2129, so not caused by this PR. - Config contract check: not applicable, no fcl changes.
- Cross-repo consistency: pass. No other repo needs a change.
Residual risk
- A cluster's sector is learned from the crossings seen so far in the job. So a cluster in a sector no crossing has reached goes to
unknown(1 of 64 here). A job that reads files made with different geometries could also carry the shield-to-sector mapping of one file into the next. I did not test that case; the nightly and PassN validation jobs read one configuration each. - Not checked: the PassN
Validation/valstep.
Author follow-ups
- Update the PR description (finding 1).
|
@FNALbuild run build test |
|
⌛ The following tests have been triggered for 1886f04: build (Build queue - API unavailable) |
|
☀️ The build tests passed at 1886f04.
N.B. These results were obtained from a build of this Pull Request at 1886f04 after being merged into the base branch at 7fb240f. For more information, please check the job page here. |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #2107
Reviewed at head 1886f04beedaa5e4bf8b719687f8ed00ff9aa204. Follow-up to my comment review at the same head.
Decision
- 🟢 approve. FNALbuild build 3600 has now built this head, merged on
main7fb240fc, and every test passed: the prof build, all eleven validation jobs,rootOverlaps,g4surfaceCheck,trigger,check_cmakeand the whitespace check. The FIXME/TODO count and clang-tidy rows are informational; both clang-tidy errors are in cetlib'sLibraryManager.h, not in this PR. This is the approval my comment review promised once the build was green. Finding 1, the out-of-date PR description, still stands and does not block.
Summary
This PR adds validation histograms for the
TrkCrvMatchandTrkCrvMatchMCproducts of the track-CRV associationPR (#2106), so that the nightly validation and the PassN
Validation/valstep monitor the association. Itdepends on that PR and changes nothing else.
The
Validationmodule fills one histogram set per product instance it finds in the event, like every otherVal*class, so no configuration is needed: any reco job that runsTrkCrvMatcher(andTrkCrvMatchMCMakeron MC)gets the histograms.
What is added
Validation/inc/ValTrkCrvMatch.hh,Validation/src/ValTrkCrvMatch.ccValidation/inc/ValTrkCrvMatchMC.hh,Validation/src/ValTrkCrvMatchMC.ccValidation/src/Validation_module.ccValidation/CMakeLists.txt,Validation/src/SConscriptValTrkCrvMatch(reco):nSharedistribution and thecandidate flag bits;
dtReffor all candidates and for the best one, anddtRefToF;dtXing,duanddv; and its χ²/ndof;dvagainst the across-bar track slope. A slope-dependentdvis a normal-directionoffset between tracker and CRV; the mean
dvis the in-plane offset. These double as an alignment monitor;pulse, its time difference, PEs, and the two-end position against the prediction.
ValTrkCrvMatchMC(truth):candidate, the best time candidate, and all geometric and time-only candidates;
its sector;
Tests
On the build of both branches:
histograms reproduce the association study (best geometric candidate is the track's own particle in 97.45 % of
tracks).
ValTrkCrvMatchfills, layer probe included.Notes
Production companion PR (Schedule the track-CRV association (TrkCrvMatcher, TrkCrvMatchMCMaker) in the reco and recoMC paths Production#595).
dtXingand the residuals are per shield, not per sector type, because sector-to-sector CRV time offsets of order10 ns would otherwise blur them.