Skip to content

CRVResponse: binomial photon arrivals per sub-step and a binary search for lookup bins - #2145

Closed
RobMina wants to merge 2 commits into
Mu2e:mainfrom
RobMina:crv-binomial-arrivals
Closed

RobMina wants to merge 2 commits into
Mu2e:mainfrom
RobMina:crv-binomial-arrivals

Conversation

@RobMina

@RobMina RobMina commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Background

These optimizations were first implemented as part of the GPU-accelerated fastsim project (https://github.com/RobMina/mu2e-fastsim). The same ideas work on CPU, as shown below.

This PR does not preserve bit-identical output when binomialArrivals: true is set (which is the default). Reviewers, please double check that the CRV response simulation outputs are equivalent with and without binomialArrivals. The validation procedure I used is described below.

What

MakeCrvPhotons::MakePhotons creates N photons per 0.1 mm sub-step and photon type. It then draws one uniform per
created photon and keeps the photon with the bin's arrival probability p. A bar crossing creates of order 10^5
photons, and only tens arrive, so almost every draw rejects a photon. CrvPhotons is most of the CRV response CPU.

This PR makes two changes.

  • Binomial arrivals (new CrvPhotonGenerator parameter binomialArrivals, default true). Per sub-step,
    photon type and SiPM, the number of arriving photons is drawn as Binomial(N, p). That is the same distribution as
    N Bernoulli trials. Only the photons that arrive then get their arrival times, through the same code as before,
    now factored into AddArrivingPhoton. p ≥ 1 gives N arrivals and p ≤ 0 gives none, as before.
    • The draw is MakeCrvPhotons::GetNumberOfArrivals, an inversion sampler using one uniform per draw. It is exact
      at any n p and takes about n p + 1 steps.
    • CLHEP::RandBinomial is not used in the CRV regime. In CLHEP 2.4.7.1 it is biased low at small n p. Over 1M
      draws it returns exactly 0 at (n, p) = (10, 1e-4) and (50, 1e-4), and is 1-2 % low at (200, 1e-4). A first
      version of this PR that used it gave 0.5 % fewer PE on cosmics.
    • RandBinomial remains only as a fallback where the probability of no arrival would underflow (n p above about
      700); it is accurate there.
  • Binary search in LookupBinDefinitions::findBin (always on). It uses std::lower_bound over the ascending bin
    edges. A value on an inner edge goes to the lower bin, and a value outside [front, back] (or NaN) is "not found",
    as with the linear scan it replaces. This change is exact.

With binomialArrivals: false, the per-photon loop and its random sequence are those of main.

Files: CRVResponse/inc/MakeCrvPhotons.hh, CRVResponse/src/MakeCrvPhotons.cc,
CRVResponse/src/CrvPhotonGenerator_module.cc.

Validation

The reference is Offline main ca7b6580 (the CI build), with the same Production and mu2e-trig-config commits in both
arms.

binomialArrivals: false: bitwise identical to main.

  • Mix1BB digitisation of 2,000 CeMLeadingLog.MDC2025ap events with the four pileup streams: identical in every
    event, and TrigReport is identical.
  • Extracted-geometry cosmic digitisation plus CRV reco of 10,000 CosmicCRYExtracted.MDC2025as events: CRV digis,
    digi MC, reco pulses and coincidence clusters are identical.

binomialArrivals: true (default): statistically equal to main. The test sample is the same 10,000 cosmic events,
paired per event. The control is main against main with only the seed changed. The PR's values:

PR vs main control (main vs main)
pulse PE per event +0.03 ± 0.44 (z +0.07) z +1.2
coincidence-cluster PE per event z +0.46 z +0.6
mean PE per pulse, per channel (1,920 channels) 1.0001 ± 0.0011 1.0003 ± 0.0010
pulse PE spectrum chi2/ndf 186/198 232/198
digis, pulses, clusters per event z +1.2, +0.5, +0.5 z +2.4, +1.3, +0.6
fraction of events with a coincidence z +1.1 z +1.1

Coincidence counts agree for all four sector types (|z| ≤ 0.05). In the mixed sample all non-CRV products are
identical. CRV digis per event change by -0.02 ± 0.07 (z -0.25), and TrigReport is identical. With
binomialArrivals: false, after the final build, the 10,000 cosmic events are again bitwise identical to main.

Performance

Both arms ran on the same host, in two repeats alternating base and PR. Times are art TimeTracker means.

PR / main
CrvPhotons, cosmic (10,000 events) 0.093 / 0.092 (17.4 -> 1.6 ms/event) main repeat spread 0.01 %
cosmic CRV digitisation + reco, full event 0.249 / 0.245
CrvPhotons, Mix1BB (1,000 events) 0.235 / 0.243 (15.3 -> 3.7 ms/event) main repeat spread 0.1 %
Mix1BB full event 0.947 / 0.971 main repeat spread 0.6 %

The binary search alone (binomialArrivals: false) gives CrvPhotons 0.97 on cosmics.

🤖 Generated with Claude Code

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #2145

Reviewed at head c8575b8d15f20bc4cd14e38947e957aa8e68d331. First pass.

Decision

  • 🟡 comment now, approve when green. I would approve this code.

    • The binomial draw is exact: it matches the binomial distribution in every case I tested.
    • The binary search returns exactly what the linear scan returned.
    • With binomialArrivals: false the output is bit-identical to main.
    • With the default, the CRV response on 10,000 off-spill cosmics agrees with main as closely as main agrees with itself under a new seed, and CrvPhotons runs 8 times faster.

    Two small items do not block (finding 1). FNALbuild has not run on this PR: there is no status and no FNALbuild comment at this head (checked 17:47 UTC). I am triggering it and will approve once the build at c8575b8d is green.

Scope understood

  • MakeCrvPhotons::MakePhotons draws, for each sub-step, photon type and SiPM, the number of arriving photons from a binomial distribution (MakeCrvPhotons.cc:396-411). It then gives only the arriving photons their times, in the new AddArrivingPhoton, which uses the same draws as before.
    • GetNumberOfArrivals (:522-539) samples by inversion. It falls back to CLHEP::RandBinomial only where (1-p)^n would underflow.
    • The new CrvPhotonGenerator parameter binomialArrivals (default true, CrvPhotonGenerator_module.cc:78) switches this on.
  • LookupBinDefinitions::findBin uses std::lower_bound (:121-130). This part is always on.

Findings

  1. ⚪ [S3] Small items.
    • The default lives only in C++ (fhicl::Atom<bool> binomialArrivals{..., true}, :78), and no prolog sets it.
      • Every versioned configuration therefore switches algorithm with no fcl change: prolog_v03 to prolog_v12 and the epilog_extracted_v0* files.
      • A job that reproduces an older production with its matching prolog now gets different, statistically equivalent, CRV output.
      • Setting binomialArrivals : true in prolog_v12.fcl, with no C++ default, would keep the choice in the configuration, as the module's other parameters are.
    • The description still has a placeholder: "In the mixed sample all non-CRV products are identical. [mixed CRV digi result]".

Checked, no action needed

  • 🟢 The binomial draw is the same distribution as one draw per photon.

    • Within one sub-step and type, every created photon has the same bin and therefore the same probability. The count of arrivals is then Binomial(N, p).
    • Each arriving photon gets its time from the same code as before.
    • The order is unchanged: scintillation photons first, then Cherenkov.
    • p ≥ 1 and p ≤ 0 give N and 0 arrivals, as before.
    • The flag reaches shared MakeCrvPhotons instances too: it is set on each new instance before sectors share it.
  • 🟢 The sampler is exact. I copied GetNumberOfArrivals and drew 4M times each on CLHEP MixMaxRng, against the exact Binomial(n, p) probabilities:

    (n, p) χ²/ndf mean pull
    (10, 1e-4) 3.0/2 −1.2
    (50, 1e-4) 0.8/3 +0.7
    (200, 1e-4) 1.2/4 +0.4
    (500, 1e-3) 9.8/7 −2.0
    (2000, 3e-4) 3.6/8 −0.5
    (300, 0.01) 11.5/15 +0.9
    (40, 0.2) 21.7/22 −1.4
    (20, 0.7) 11.1/17 −0.8

    Your note on CLHEP::RandBinomial is right. In the same test it never returned a non-zero value at (10, 1e-4) or (50, 1e-4), and it is 2.2 % low at (200, 1e-4) (pull −5.8). Keeping it only for n p above about 700 is the right choice. Nothing else in Offline calls it.

  • 🟢 findBin is exact.

    • All 14 CRVConditions/v6_0 lookup tables load into a check program. These are the only tables prolog_v12, epilog_run1a_v01 and epilog_extracted_v05 load, and LoadLookupTable rejects anything but version 6.
    • All 98 bin-edge vectors are strictly ascending.
    • On 19.6M probes, the old linear scan and the new search give the same bin and the same not-found flag every time. The probes are every edge, each edge ±1 ulp, every bin centre, 200k random values per vector extending 10 % beyond the range, and NaN.
  • 🟢 With binomialArrivals: false, bit-identical to main. main is the CI build 152833a9. The PR is this head merged on 152833a9 together with #2142. Products were dumped with PrintModule.

    • Off-spill cosmics: Production digitize/OffSpill.fcl plus CRV reco, 10,000 CosmicCRYAll MDC2025ap events. CrvDigis, CrvDigiMCs, CrvRecoPulses, CrvCoincidenceClusters and CaloDigis are identical in every event.
    • On-spill CE, 1,000 events, and FNALbuild's ceMix and ceDigi: all products identical.
  • 🟢 With the default, statistically equal to main. Same 10,000 cosmic events, paired per event. The control is main against main with only the seed changed (baseSeed 9 instead of 8):

    this PR vs main control
    reco pulses per event z −0.37 z +0.72
    pulse PE per event +0.12 ± 0.33 (z +0.38) z −0.32
    digis per event z +0.85 z −0.74
    ADC sum per event z +0.83 z −0.61
    coincidence clusters per event z −0.59 z +0.88
    cluster PE per event z +0.20 z +0.07
    events with a coincidence z −0.75 z +0.75
    pulse PE spectrum, χ²/ndf (KS p) 184/195 (0.69) 200/195 (0.96)
    pulse height spectrum 161/194 (0.85) 173/194 (0.63)
    pulse times after the first pulse 154/130 (0.15) 136/129 (0.58)
    cluster PE spectrum 120/121 (0.82) 98/123 (0.97)
    mean PE per pulse, per channel (≥ 20 pulses) 1.0047 ± 0.0013 1.0023 ± 0.0013
    • The per-channel ratio of means is biased slightly above 1 for any two independent samples, as the control shows. The difference between the two columns is 1.3σ.
    • On FNALbuild's ceMix, the calorimeter and all straw-digi collections stay identical. The CRV digis differ in the one event that has any.
  • 🟢 Timing. main and this PR ran at the same time on the same node, 3,000 cosmic events; TimeTracker means:

    main false default
    CrvPhotons 14.1 ms 13.9 ms 1.76 ms
    full event (digitisation + CRV reco) 24.2 ms 23.5 ms 11.5 ms

Validation check

  • Build/tests run: yes. A muse build of this head merged with #2142 on 152833a9 (p110) exited with status 0 and no warnings. All the jobs above exited with status 0.
  • Config contract check: pass. The new parameter is validated; see finding 1 for where its default lives.
  • Cross-repo consistency: pass. No Production or other repository sets the parameter. This PR merges cleanly with #2142.

Residual risk

  • My cosmic sample has the off-spill configuration, which turns CRV thermal noise off, and its clusters are all sector type 1. I did not run the extracted geometry; your validation covers it. I did not run on-spill with real CRV activity: the CE sample has no CRV digis, and ceMix has 3 in 53 events.
  • When (1-p)^n is just above the underflow limit, the inversion stops as soon as a term underflows. The mass this can misplace is at most about 1e-15 per draw.

Author follow-ups

  1. Optional: finding 1.

@oksuzian

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

@FNALbuild

Copy link
Copy Markdown
Collaborator

Hi @RobMina,
You have proposed changes to files in these packages:

  • CRVResponse

which require these tests: build.

@Mu2e/fnalbuild-users, @Mu2e/write have access to CI actions on main.

⌛ The following tests have been triggered for c8575b8: build (Build queue - API unavailable)

About FNALbuild. Code review on Mu2e/Offline.

@FNALbuild

Copy link
Copy Markdown
Collaborator

☀️ The build tests passed at c8575b8.

Test Result Details
test with ✅ Command did not list any other PRs to include
merge ✅ Merged c8575b8 at fb72055
build (prof) ✅ Log file. Build time: 04 min 37 sec
ceSimReco ✅ Log file.
g4test_03MT ✅ Log file.
transportOnly ✅ Log file.
POT ✅ Log file.
g4study ✅ Log file.
cosmicSimReco ✅ Log file.
cosmicOffSpill ✅ Log file.
ceSteps ✅ Log file.
ceDigi ✅ Log file.
muDauSteps ✅ Log file.
ceMix ✅ Log file.
rootOverlaps ✅ Log file.
g4surfaceCheck ✅ Log file.
trigger ❓ Log file. Return Code 1.
check_cmake ✅ Log file.
FIXME, TODO ➡️ TODO (1) FIXME (0) in 3 files
clang-tidy ➡️ 1 errors 107 warnings
whitespace check ✅ no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at c8575b8 after being merged into the base branch at fb72055.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

@oksuzian

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

1 similar comment
@oksuzian

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

…e in prolog_v12

Older CRVResponse prologs keep the per-photon draw, bit-identical to main; current configurations, which include prolog_v12, get the binomial arrivals.

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #2145

Reviewed at head 67228d5358fe9306b4924b5dd2d5445585940c4c. Re-review of c8575b8.

Decision

  • 🟡 comment now, approve when green. The new commit settles my nit about where the default lives, and current jobs behave exactly as at c8575b8d. FNALbuild has not built this head (no mu2e/buildtest result at 03:20 UTC), so I am requesting a build and will approve once it is green.

What changed since c8575b8

  • One commit, 67228d5 "CRVResponse: binomialArrivals defaults to false in C++ and is set true in prolog_v12" (+2 −1). Not a rebase.
    • CrvPhotonGenerator_module.cc:78: the binomialArrivals default goes from true to false.
    • prolog_v12.fcl:94 sets binomialArrivals : true inside the CrvPhotons table.

Prior findings (from my review at c8575b8d)

  • 🟢 [was ⚪ S3, first item] The default lived only in C++, so every versioned prolog switched algorithm with no fcl change. ADDRESSED.
    • prolog_v03 to prolog_v11 now keep the per-photon draws, so a job that reproduces an older production with its matching prolog gets the old CRV output again.
    • prolog.fcl includes prolog_v12.fcl, so every current job still runs the binomial draw. That covers Production's digitization, mixing and cosmic jobs, which only override other CrvPhotons parameters. Current output is therefore what I validated at c8575b8d.
    • I had suggested no C++ default at all. With false as the default, the old prologs need no edits, which is a fair choice.
  • 🟢 [was ⚪ S3, second item] The placeholder in the description is filled: the mixed-sample CRV digis change by −0.02 ± 0.07 per event (z −0.25), and TrigReport is identical.

Validation check

  • Build/tests run: no new jobs. The commit moves a default and sets it in the current prolog, which I checked by reading the diff and prolog_v12.fcl at this head. My runs at c8575b8d cover both settings: the default distribution study with true, and bit-identity to main with false.
  • Config contract check: pass.
  • Cross-repo consistency: pass. No Production config includes an older CRVResponse prolog, and none sets binomialArrivals.

@oksuzian

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

1 similar comment
@oksuzian

Copy link
Copy Markdown
Collaborator

@FNALbuild run build test

@FNALbuild

Copy link
Copy Markdown
Collaborator

⌛ The following tests have been triggered for 67228d5: build (already triggered: build) (Build queue - API unavailable)

@FNALbuild

Copy link
Copy Markdown
Collaborator

☀️ The build tests passed at 67228d5.

Test Result Details
test with ✅ Command did not list any other PRs to include
merge ✅ Merged 67228d5 at 30088f3
build (prof) ✅ Log file. Build time: 09 min 24 sec
ceSimReco ✅ Log file.
g4test_03MT ✅ Log file.
transportOnly ✅ Log file.
POT ✅ Log file.
g4study ✅ Log file.
cosmicSimReco ✅ Log file.
cosmicOffSpill ✅ Log file.
ceSteps ✅ Log file.
ceDigi ✅ Log file.
muDauSteps ✅ Log file.
ceMix ✅ Log file.
rootOverlaps ✅ Log file.
g4surfaceCheck ✅ Log file.
trigger ✅ Log file.
check_cmake ✅ Log file.
FIXME, TODO ➡️ TODO (1) FIXME (0) in 3 files
clang-tidy ➡️ 1 errors 107 warnings
whitespace check ✅ no whitespace errors found

N.B. These results were obtained from a build of this Pull Request at 67228d5 after being merged into the base branch at 30088f3.

For more information, please check the job page here.
Build artifacts are deleted after 5 days. If this is not desired, select Keep this build forever on the job page.

@oksuzian oksuzian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary — #2145

Reviewed at head 67228d5358fe9306b4924b5dd2d5445585940c4c. Follow-up to my comment review at the same head.

Decision

  • 🟢 approve. FNALbuild build 3602 has now built this head, merged on main 30088f39, and every test passed: the prof build, all eleven validation jobs, rootOverlaps, g4surfaceCheck, trigger, check_cmake and the whitespace check. The FIXME/TODO count and clang-tidy rows are informational; the one clang-tidy error is in cetlib's LibraryManager.h, not in this PR. This is the approval my comment review promised once the build was green.

@RobMina

RobMina commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Closing this as redundant with #2150

@RobMina RobMina closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants