Repository navigation
switched on calo materials - #2103
Conversation
|
Hi @sophiemiddleton,
which require these tests: build. @Mu2e/write, @Mu2e/fnalbuild-users have access to CI actions on main. ⌛ The following tests have been triggered for 5d97add: build (Build queue - API unavailable) |
|
☀️ The build tests passed at 5d97add.
N.B. These results were obtained from a build of this Pull Request at 5d97add after being merged into the base branch at b24d416. For more information, please check the job page here. |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #2103
Reviewed at head 2d9366033e4f64cb1be1f04e0ffda5fe0d883eaf. First pass.
Decision
- 🟡 comment only. This line turns on the calorimeter material in every production LoopHelix fit (
KKDe,KKUe,KKDmu,KKUmu), but the PR has no description and no validation. The comment it deletes said the option stays "off until validated against MC truth". My own run on the nightlyreco_03sample shows a small effect: the core momentum resolution ofKKDeagainst truth narrows by about 1%, and the same few converged fits are lost as in my #2081 review (finding 2). I would approve once the description says what changes and links the resolution and efficiency plots from today's meeting. FNALbuild passed at the first commit5d97addb, which differs from this head only in the comment;mu2e/buildtesthas not run at this head.
Scope understood
Mu2eKinKal/fcl/prolog.fcl:505setsAddCaloMaterial : trueinMu2eKinKal.LHDriftFit.KKFitSettingsand drops the comment. The second commit only removes the comment. Merging withmainb24d4163(git merge-tree) givesmainplus this one line.- With the option on,
KKFit::extendTrackcallsaddCaloMaterialonce the extended fit is usable (KKFit.hh:407). That adds the calorimeter front-panel layers and the crystal-case inner rings crossed between the last straw hit and the end of the fit, then refits once at the final temperature with no hit updaters (KKTrack.hh:335-347).ToCaloMaterial(prolog.fcl:395), which adds the same material beyond the end of the fit during extrapolation (KKExtrap.hh:151), staysfalse. - Which jobs change. I ran
fhicl-dumpon 23 Production, PassN and Validation configurations, withmain's prolog and with the PR's. The only difference anywhere isAddCaloMaterial, onKKDe,KKUe,KKDmu,KKUmuand Production'sRegrowLHtable.- The four fits run in
recoMC/OnSpill,OffSpill(LHPath), bothRetriggerconfigurations,reco/OnSpill, and the validation jobsreco.fcl,offSpill.fcl,ceSimReco.fcland nightlyreco_03. - Unchanged: the trigger,
KKCHmu,KKLine, the regrow jobs and PassNCombined_Pass1. mu2e-trig-config builds its fits fromMu2eKinKal.KKFIT, and the C++ default isfalse(KKFitSettings.hh:76). The dumps ofValidation/trigger.fclanddigitize/OnSpill.fclare identical. The regrow jobs use Offline'sRegrowLH, built fromKKFIT.Combined_Pass1runs onlyKKLineon its paths. - Production's own
RegrowLH(JobConfig/reco/prolog.fcl:123-138) inheritstruethrough@table::Mu2eKinKal.producers.KKDe.KKFitSettings, but no job puts it on a path.
- The four fits run in
Findings
-
🟠 [S1] A production reconstruction default changes with no description and no validation.
-
Evidence: the description is empty. The title says "calo materials", but only one of the two options changes. The deleted comment made the change conditional on validation against MC truth. On #2081 the author offered to show resolution and efficiency plots at the Wednesday meeting (issuecomment-6581229006), and nothing has been posted on either PR. brownd1978 noted there that calorimeter material "requires a different workflow", because the calorimeter hit comes after the material (issuecomment-6582318164). This PR does not say whether that was settled.
-
Impact: 91% of
KKDetracks and 65% ofKKDmutracks now carry calorimeter material in the fit, and most of their fitted values move slightly (table below). Anyone who sees these shifts in validation will need the reason, and this PR does not give one. -
Suggested fix. Best practice reminder: please add a meaningful PR description including intent, scope, and validation evidence. In particular:
- why only
AddCaloMaterialis turned on; - the expected effect on resolution and efficiency, with a link to the meeting plots (DocDB or Indico).
That clears this finding.
- why only
-
-
🟡 [S2] Carried over from #2081: some converged fits are dropped as
ParamsDivergedafter the material refit.- Evidence: on
reco_03,KKDeloses 9 tracks, 3 of them conversion-like (1430:1612:4983, 1430:1652:1372, 1430:967:4391).KKDmuloses 68, 9 of them in the conversion-like window. These are exactly the track counts I measured on #2081 with both options on, soToCaloMaterialplays no part. - I reran 1430:1652:1372 at this head with
KKDeprinting. The fit converges with probability 0.957. Three disk-0 front-panel layers and two disk-1 inner-ring crossings are added, and thenFit Status ParamsDiverged Meta-iteration 13 iteration 0 Chisq value 45.2518 nDOF 63 prob 0.955355makes the fit fail (goodFit = 0). Withmain's prolog the same event ends withgoodFit = 1, fitcon = 0.9567. - Impact: the rate is small, 3 of 10,769 conversion-like
KKDetracks. But each lost fit had converged with a high probability. - Suggested fix: find why a single refit iteration trips the divergence test on a converged fit. Until then, the description should say that this loss is known and accepted.
- Evidence: on
-
⚪ [S3] The deleted comment also said what the parameter does. Please keep "add the calorimeter front panel material crossed within the fit range" and drop only the caveat.
prolog.fcl:395still saysToCaloMaterialstays "off until validated against MC truth"; the description could say whether that is still the plan.
Checked, no action needed
-
🟢 Effect on all 16,508 events of
Production/Validation/nightly/reco_03.fcl, read in place. Both runs used the same p110 build, once withmain's prolog and once with the PR's. EveryKalSeedcollection was printed. All 12 jobs exited with status 0. Every triggerKalSeedcollection is identical.Fit tracks, main/ PRlost / gained conversion-like, main/ PRtracks with calorimeter material KKDe16,486 / 16,478 9 / 1 10,769 / 10,742 14,928 KKUe16,303 / 16,294 12 / 3 743 / 731 7,125 KKDmu16,316 / 16,249 68 / 1 8,542 / 8,505 10,529 KKUmu16,043 / 16,033 12 / 2 720 / 709 6,012 - Conversion-like here means 95-110 MeV/c at
TT_Front, fit consistency above 1e-3 and at least 20 active hits, as in #2081. - The 27 fewer conversion-like
KKDetracks are the 3 lost fits, plus 53 tracks that drop below the fit-consistency cut, minus 29 that rise above it. - On
KKDe, 14,927 of the 16,477 matched tracks change. - Momentum at t0: changes on 13,421 tracks, with a median change of 0.006 MeV/c. 64 tracks change by more than 1 MeV/c, all with few hits or a fit consistency near 0.
- Conversion-like
KKDe: the momentum atTT_Frontshifts by −0.011 MeV/c on average, and by at most 1.76 MeV/c. - t0: changes on 3,176 tracks, 15 of them by more than 1 ns (at most 3.2 ns).
- The last column comes from a third run with
SaveMaterialXings : trueon the four fits. Its fit results are identical to the PR run.
- Conversion-like here means 95-110 MeV/c at
-
🟢 Against MC truth. This is a cross-check, not a substitute for the author's validation.
- Method: I ran EventNtuple
main(6da9b4f) on both outputs and compared theKKDemomentum atTT_Frontwith the true momentum of the matched particle at the tracker front. - Sample: the 10,000 events with exactly one conversion-like
KKDetrack in both runs. - The half-width of the central 68% goes from 0.1791 to 0.1767 MeV/c.
- The median goes from −0.0163 to −0.0228 MeV/c.
- Tails: 234 tracks are low by more than 1 MeV/c in
mainand 236 with the PR; 28 are high by more than 1 MeV/c inmainand 22 with the PR. - Events with a conversion-like
KKDetrack matched to a 95-110 MeV/c true particle go from 10,857 to 10,847.
- Method: I ran EventNtuple
-
🟢 FNALbuild build 3553 at
5d97addbpassed every row, includingceSimReco, which runs these fits with the option on, plustriggerandcheck_cmake.5d97addband the head differ only in the comment. GitHub reports the PR as mergeable. No other open Offline PR touchesMu2eKinKal/fcl.
Validation check
- Build/tests run: yes.
- No new build was needed, since the PR changes only fcl. I used my p110 build from the #2100 review: Offline
7bccd295, which ismain5629b9f2plus p110, with Production94a4d81fand mu2e-trig-config5999ffa2. Since then,mainhas added only #2090 (CosmicTrack). - The PR's prolog went first in
FHICL_FILE_PATH, and it is byte-identical to thegit merge-treeresult. - Jobs: 18
reco_03chunks, 12 EventNtuple jobs, two single-event reruns, andfhicl-dumpof 23 configurations. All exited with status 0.
- No new build was needed, since the PR changes only fcl. I used my p110 build from the #2100 review: Offline
- Config contract check: pass.
AddCaloMaterialis an existingfhicl::Atom<bool>. - Cross-repo consistency: pass. No Production, mu2e-trig-config, PassN, EventNtuple, DQM, RefAna or EventDisplay fcl sets
AddCaloMaterial.
Residual risk
- Only one sample was checked: CeMLeadingLogMix1BB, on spill.
- Not checked:
- off-spill cosmics through
OffSpill'sLHPath; - real data;
- the CPU cost. With both options on, I measured 85.9 against 81.6 ms per event for the four fits on #2081.
- off-spill cosmics through
Author follow-ups
- Add a description, with the plots, as in finding 1.
- Say whether the
ParamsDivergedlosses are accepted for now or will be followed up (finding 2). - Optional: keep the parameter description in the comment (finding 3).
|
@FNALbuild run build test |
|
⌛ The following tests have been triggered for 2d93660: build (Build queue - API unavailable) |
|
☀️ The build tests passed at 2d93660.
N.B. These results were obtained from a build of this Pull Request at 2d93660 after being merged into the base branch at b24d416. For more information, please check the job page here. |
No description provided.