[zichunhao]

Supersedes #366: the same changes on top of the current main, regrouped into one commit per topic: correctionlib JECs, archive, JES/JER of the AK4-jet inputs (skimmer and postprocessing, with the fix of the shifted AK4 jet pT loop for chunks without JECs), Type-1 PUPPI MET recomputed (with the BDT reading it), its JES/JER/UES variations (with their use in the shifted BDT inputs of postprocessing).

Summary

• corrections.JECs builds the AK4 and AK8 jet factories of every Run 3 year (2022-2025) from the official JME correctionlib files on CVMFS (/cvmfs/cms-griddata.cern.ch/cat/metadata/JME/, dated folders pinned in cat_jerc), with the coffea 0.7 adapters of processors/jec_clib.py that 2024 and 2025 already use. No text files for Run 3: the compiled text-file JECs are read for Run 2 only.
• Versions (the newest on CVMFS):

year	before (text files)	now (correctionlib)
2022, 2022EE	Summer22(EE)_22Sep2023_V2, JER JRV1	Summer22(EE)_22Sep2023_V4, JER JRV2
2023, 2023BPix	Summer23(BPix)Prompt23_V1, JER JRV1	Summer23(BPix)Prompt23_V4, JER RunCv1234_JRV3 / RunD_JRV3
2024	Summer24Prompt24_V5, JER JRV2 (correctionlib)	unchanged
2025	Summer24Prompt25_V3, JER JRV2 (correctionlib)	unchanged

• Data: the residuals of these versions are run dependent, so jec_clib passes the run number per jet instead of picking a per-era stack by dataset name.
• JES/JER of the other BDT inputs (third commit): the AK4 jets away from the bb jets and the VBF jet mass get JES/JER-shifted columns, which the shifted BDT inputs of the skimmer and of postprocessing use; the loop over the shifted AK4 jet pT no longer fails in chunks where no JECs were applied (section below).
• MET (last two commits): the Type-1 PUPPI MET is recomputed from RawPuppiMET with the re-applied AK4 PUPPI JECs, for data and MC, with its JES, JER and unclustered-energy variations, which the shifted BDT inputs of the skimmer and of postprocessing use (section below).

Relation to DAZSLE/hbb-run3#74

Same approach as DAZSLE/hbb-run3#74: the official JSON files with the same versions, and the compound L1L2L3Res correction fed into coffea's CorrectedJetsFactory. There the stack is coffea's CorrectionLibJECStack (newer coffea); our coffea 0.7 has no such class, so jec_clib.py provides the equivalent adapters. The data corrections take the run number as an input, which jec_clib passes per jet; the MC stack keeps the total JES uncertainty and the JER scale-factor variations.

Effect on 2022 and 2023

Jets of the first 20,000 events of a 2022 HH4b MC file, a 2022C JetHT, a 2023 HH4b (κλ = 5) MC and a 2023Cv1 JetMET0 file, corrected with the old code and the new one, with the call of bbbbSkimmer:

check	2022 MC	2022 data	2023 MC	2023 data
method: text-file JEC against correctionlib of the same version, max relative difference AK8 / AK4	3.1e-07 / 1.7e-06	1.4e-06 / 3.4e-06	3.0e-06 / 9.6e-07	3.3e-06 / 1.0e-06
JEC factor new / old, AK8	1.0000	1.0000	1.0000	1.0000
JEC factor new / old, AK4	1.0000; only 4.25 < |η| < 4.5: −0.03%	1.0000; only 4.25 < |η| < 4.5: −0.18%	1.0000	1.0000; raw pT 15-30 GeV at 1.3 < |η| < 2.5: median −1.1% (−0.6% to −1.7% per η bin)
total JES uncertainty (median), old / new	AK8 0.78% / 0.78%, AK4 2.57% / 2.57%	not applied to data	AK8 0.78% / 0.78%, AK4 2.81% / 2.81%	not applied to data
JER factor of gen-matched jets (median), old / new	AK8 1.0018 / 1.0018, AK4 1.0009 / 1.0009	not applied to data	AK8 1.0012 / 1.0012, AK4 1.0001 / 1.0001	not applied to data

• The switch of method is exact (a few 1e-6).
• The update to V4 leaves the AK8 jets unchanged in MC and data. AK4 jets move only in the most forward 2022 bin (below 0.2%) and at low pT in 2023 data (about −1%, which reaches the 25-30 GeV VBF-jet candidates). The JES uncertainty and the JER smearing are unchanged.
• So 2022, 2022EE and 2023BPix need no rerun for the JEC switch alone. The 2023 Run C v4 data do: the text-file path picked 2023_runCv4 only if Run2023Cv4 is in the dataset name, which JetMET_Run2023C-22Sep2023_v4-v1 never matches, so all Run C v4 files (56% of the 2023 pre-BPix JetMET files) got the Run C v1-3 residuals. The V4 data correction is 4.5% (raw pT 50 GeV) to 5.8% (600 GeV) higher for central AK4 jets at run 368000 (v4) than at run 367300 (v1-3); in the skims the AK8 pT is 0.956 and the AK4 pT 0.963 of the NanoAOD values (2023BPix: 1.0001). The GloParT-v3 masses come from the NanoAOD raw mass and are unaffected. With the run passed per jet this is fixed.

Type-1 PUPPI MET with the re-applied JECs

• Before: the data MET went through coffea's CorrectedMETFactory, which in coffea <= 0.7.26 (we pin 0.7.21) adds the change of the jet pT relative to the raw pT with the wrong sign. Applied on top of the Type-1 PuppiMET, it undid the Type-1 correction (data MET close to RawPuppiMET), while MC kept NanoAOD PuppiMET. NanoAOD PuppiMET itself carries the Type-1 correction of the production JECs, not of the JECs the skimmer re-applies. In the 2024C sidebands the Type-1 data MET changes the data above BDT score 0.5 by −3.9% (data-driven QCD BDT) and −2.7% (MC QCD BDT).
• Now (jec_clib.type1_puppi_met, corrections.JECs.type1_puppi_met): for the correctionlib years, i.e. all of Run 3 with this PR, the MET is recomputed from RawPuppiMET with the re-applied AK4 PUPPI JECs, as CMSSW's PFJetMETcorrInputProducer does for PUPPI: every Jet and CorrT1METJet with muon-subtracted corrected pT > 15 GeV and EM fraction < 0.9 shifts it by −(f_L1L2L3Res − f_L1) × the muon-subtracted raw pT, unsmeared, for data and MC alike. MET_pt and the new MET_phi hold it; PuppiMET_pt/phi and RawPuppiMET_pt/phi are saved as well. ttSkimmer gets the same MET. The skimmer's BDT scores and the v15 BDT config take this MET_pt as their MET input (the config through key_map).
• Variations (MC): MET_pt/MET_phi for JES_up/down, JER_up/down and UES_up/down, and the JES/JER-shifted BDT scores use the shifted MET. They are the variations of the unsmeared Type-1 MET in CMSJMECalculators (gitlab.cern.ch/cms-analysis/general/CMSJMECalculators, Type1METVariationsCalculator with isT1SmearedMET false), the same as the MET_T1 variations of nanoAOD-tools' jetmetUncertainties, for the jets of the Type-1 sum at their corrected pT (muon-subtracted part corrected, muon added back): JES up/down puts each jet (Jet and CorrT1METJet) at that pT × (1 ± the total JES uncertainty there); JER up and down are both the Type-1 MET with the Jets smeared by their nominal smearing factors (the CorrT1METJets unsmeared, as coffea's Type-1 MET does without their factors); UES up/down adds/subtracts NanoAOD PuppiMET's unclustered shift. coffea's Type-1 CorrectedMETFactory scales the jets by the JER-varied over the nominal smeared pT instead, which blows up for jets that the nominal smearing pushes to ~0 GeV (a JER_down MET shift of +724 GeV on average in 2024 ttbar). Against a per-jet loop written after CMSJMECalculators' addVariations, all variations agree to 1e-13 GeV (20,000 2022 and 2024 TTto4Q events); the mean (RMS) shifts there are JES up +0.8 GeV (1.9), JES down −0.7 GeV (1.9), JER +2.2 GeV (5.7 in 2024, 6.4 in 2022), UES ±0.2-0.3 GeV (1.3).
• Validation: with the JECs of the production, the recomputed MET reproduces NanoAOD PuppiMET (CMSSW's own Type-1 output) to 0.31 GeV RMS (2024 TTto4Q, 20,000 events) and 0.28 GeV (2022), and it is bit-identical to correct_met of Framework Updates DAZSLE/hbb-run3#74 (coffea's Type-1 CorrectedMETFactory, feat: implement CMS Type-1 MET correction in CorrectedMETFactory scikit-hep/coffea#1537) for the same JECs. With our JECs, recomputed − NanoAOD PuppiMET (mean / RMS): −1.6 / 6.7 GeV in 2024C data, −3.1 / 8.5 GeV in 2024 TTto4Q, −1.8 / 9.8 GeV in 2022C data, −0.05 / 1.7 GeV in 2023 TTto4Q. One-chunk skims of this branch run for 2024 TTto4Q, 2023 JetMET Run C v4 and 2022 TTto4Q.
• A TODO in the code: use coffea's factory once we are on coffea >= 2026.9.0 or a 0.7.x release after 0.7.31; it gives the same nominal and, with the total JES uncertainty passed as jes_uncertainties, the same JES variations, but not this JER variation.
• The MET changes the skims of every year (MET columns, BDT inputs), so it goes with the next re-skim.

JES/JER of the other BDT inputs

• The skims had no JES/JER-shifted columns for the two AK4 jets away from the bb jets and none for the VBF jet mass, so in every JES/JER variation the BDT inputs H1AK4JetAway1mass and H2AK4JetAway2mass stayed nominal and the VBF dijet mass used the nominal jet masses. The MC signal-region skims now save AK4JetAway{Pt,Mass}_{JES,JER}_{up,down} and VBFJetMass_*, from the AK4 PUPPI JERC of these jets, and the skimmer's shifted BDT scores use them.
• In postprocessing, hh_vars.jec_vars gets VBFJetMass, AK4JetAwayPt and AK4JetAwayMass, and their shifted columns are loaded with the systematics where the skims have them (utils.load_samples(optional_columns=...)); for skims made before, postprocessing.fill_ak4_jec_shifts derives the shifted VBF jet mass from the shifted VBF pT (the jet factory scales pT and mass by the same factor) and keeps the AK4-away jets nominal. The unit test of load_run3_samples that checks the requested columns stubs this fill, since its fake frames have no jet columns. The MET-recompute commit makes the v15 BDT config read the MET through key_map, and the MET-variations commit adds MET_pt to the same list (so the shifted BDT inputs use the shifted MET) and keeps the MET nominal for older skims.
• The loop over the shifted AK4 jet pT is skipped when get_jec_jets applied no JECs (it returns None, e.g. in a chunk without AK4 jets, and the loop failed with a TypeError), as the other shifted jet variables already were.
• Checks: on the current 2024 VBF HH skim (114,012 events, no new columns) only VBFjjMass of the shifted inputs changes (0.15% RMS, in the 57% of events with two VBF jets) and the nominal inputs are identical; one-chunk 2024 TTto4Q skims of the branch (at this commit and at the tip) write the new columns, and load_run3_samples reads them (each shifted column differs from its nominal for every event with the jet); unit tests: 286 passed at every commit from this one on. For scale, shifting the MET with the jets changes the JES/JER-shifted ggF BDT score of the 2022-2025 BDT by about 0.001 (JES) and 0.002 (JER) per event on average (one-chunk 2022 and 2024 TTto4Q skims).

Archive

The Run 3 text-file JECs are no longer read, so the second commit moves jec_compiled.pkl.gz, jec_compiled_upto2023BPix.pkl.gz and their builder build_jec.py to src/HH4b/corrections/archive/, with a README on what they contain and which skims used them (e.g. 20260820_glopartv3_v15_signal). corrections/README.md now describes the correctionlib JECs (a version update is a new dated folder and tag in cat_jerc, nothing to build), and the comments that named build_jec.py point to the new path. Run 2 keeps jec_compiled_run2.pkl.gz and build_jec_run2.py; JECs(year) loads for 2022-2025 and 2018.

[cmantill]

LGTM, before merging could you post a comparison of the ntuple w Aashwin and Thierry to make sure that the JECs make the jet pt etc agree?

[zichunhao]

So far we have one ntuple comparison, with Aashwin's (approach 1): AK8 jets of a 2024 SM ggHH NanoAOD file (10,030 events), Summer24Prompt24_V5 JECs on both sides, no JER, pT > 180 GeV.

• Same 10,030 events. 4,811 jets are matched one to one (median ΔR 7e-5), out of 4,812 (ours) and 4,816 (Aashwin's). The other 6 are the same jets, with the two pT values (within 0.12%) on either side of the 180 GeV cut.
• pT, η, φ and the GloParT scores agree to NanoAOD storage precision (pT within 0.15%, from the rounding of pt and rawFactor in NanoAOD). The pT and TXbb orderings are identical.
• Softdrop mass: NanoAOD msoftdrop (subjets with the production Winter24Prompt24 JEC) is 0.57% above Aashwin's (subjets re-corrected with V5). With V5 subjets we agree to 2.5e-5 (median). No event of the file changes in our selection.
• Event by event, our AK8 preselection keeps the same events in both ntuples (514 → 441 → 395), with the same H1/H2 jets in all 395 and m_HH within 0.1%.

This validates the 2024 AK8 JECs, which this PR does not change. For 2022-2023, the JECs of this PR are compared with our results before it in the table of the description: the same versions through correctionlib agree with the text-file JECs to a few 1e-6, and the update to V4 leaves the AK8 jets unchanged.

[cmantill]

Thanks, Thierry's file can be found in this link

Can you produce plots like this (i.e. comparing Pocket Coffea PC, Approach One from Ashwin, and Approach Two yours?)

Additionally the directory from Aashwin now should contain MET and AK4 jet information as well: https://cernbox.cern.ch/s/ryJNVocadBXpsxm

[cmantill]

Why are the 2022 and 2023 corrections not defined directly in these cat_corrections dictionary? What are the choices for jetveto, pileup_json or pileup for those years?

[zichunhao]

I had left the old logic for 2022-2023 intact (jet veto maps and pileup weights from jsonpog-integration), but for readability they are now in cat_corrections too, with the same keys as the other years (jme, jec, jer, jetveto, pileup_json, pileup). For 2022-2023:

• jetveto: Summer22_23Sep2023_RunCD_V1, Summer22EE_23Sep2023_RunEFG_V1, Summer23Prompt23_RunC_V1, Summer23BPixPrompt23_RunD_V1
• pileup_json: puWeights.json.gz of the LUM folders Run3-22CDSep23-Summer22-NanoAODv12/2024-01-31 (and the 22EFG, 23C, 23D ones)
• pileup: Collisions2022_355100_357900_eraBCD_GoldenJson, Collisions2022_359022_362760_eraEFG_GoldenJson, Collisions2023_366403_369802_eraBC_GoldenJson, Collisions2023_369803_370790_eraD_GoldenJson

The files are the same as before, so the skims do not change.

Note: The 2022-2023 weights are still evaluated at Pileup_nPU; Pileup_nTrueInt comes in the next PR.

[zichunhao]

Thanks, Thierry's file can be found in this link

Can you produce plots like this (i.e. comparing Pocket Coffea PC, Approach One from Ashwin, and Approach Two yours?)

Additionally the directory from Aashwin now should contain MET and AK4 jet information as well: https://cernbox.cern.ch/s/ryJNVocadBXpsxm

Here is the comparison of the three ntuples on the same events. Names in the plots and below:

• PC: PocketCoffea, Thierry's NanoAOD-based ntuple
• A1: approach 1, the MiniAOD-based ntuple from Aashwin
• A2: approach 2, ours (the skimmer of this PR, NanoAOD-based)

Plots: https://cernbox.cern.ch/s/zipzR0stIAkZccK (comparison_plots/, PNG and PDF, with the printed report.txt; our AK8 table is there too, approach2_ak8_jet.{root,parquet,pkl} with readme_AK8_ours.txt). They are Thierry's comparison_plots, made with his compare_outputs.py extended to the three ntuples, on the 2024 SM ggHH file (10,030 events) and the A1 tree he used (AK8 $p_\mathrm{T} &gt; 170$ GeV, $\lvert\eta\rvert &lt; 2.5$). Every plot shows all three pairs: PC vs A1 (blue, thicker line), A2 vs A1 (orange) and A2 vs PC (green); "X / Y" compares X with the reference Y.

$\Delta R$ plot (deltaR.png): for every AK8 jet of one ntuple, the $\Delta R$ to the closest AK8 jet of the other one; solid lines are "X jet $\to$ closest Y", dashed lines the reverse, and the last bin collects $\Delta R \geq 4$ and events where the other ntuple has no AK8 jet. The grey line at 0.4 is the matching cut. The spike at 0 holds the matched jets: 5430 for PC and A2 vs A1, and every jet for A2 vs PC. The entries above 0.4 are jets without a partner, from the different jet thresholds of NanoAOD and of the A1 tree: 139 PC (and A2) jets have no A1 partner (104 at $\lvert\eta\rvert \geq 2.5$ and 34 below 170 GeV, which the A1 tree cuts away, plus 1 at the cut), and 201 A1 jets have no PC (and A2) partner (corrected $p_\mathrm{T}$ 170-177 GeV, below NanoAOD's 170 GeV threshold with the production JEC).

Our AK8 jets are identical to PC's: every quantity agrees to float precision, except the softdrop mass, where our skim keeps NanoAOD's msoftdrop.

Numbers, including AK4 jets and data: ratios of matched jets, median (68% half-width), for jets with $p_\mathrm{T} &gt; 180$ GeV and $\lvert\eta\rvert &lt; 2.5$ (AK8) or $p_\mathrm{T} &gt; 20$ GeV and $\lvert\eta\rvert &lt; 4.7$ (AK4), $p_\mathrm{T}$ after JEC; data = the 52,241 2024C JetMET0 events of the A1 data tree (run 380195; PC has no data output). Every jet in this acceptance has a partner, except 82 of 206,444 A1 AK4 data jets that NanoAOD does not store (raw $p_\mathrm{T} \approx 18$ GeV at $\lvert\eta\rvert \approx 3$).

	A2 / A1, ggHH	PC / A1, ggHH	A2 / PC, ggHH	A2 / A1, data
AK8 $p_\mathrm{T}$	$1.00003\ (3.8\times10^{-4})$	$1.00003\ (3.8\times10^{-4})$	$1.000000\ (4\times10^{-8})$	$1.00013\ (7.0\times10^{-4})$
AK8 softdrop mass, subjets with the AK4 JEC	$1.00002\ (5.7\times10^{-4})$	$1.00002\ (5.7\times10^{-4})$	$1.00000\ (3\times10^{-5})$	$1.00000\ (6.7\times10^{-4})$
AK4 JEC factor	$1.000000\ (5\times10^{-6})$	PC's $p_\mathrm{T}$ includes JER	PC's $p_\mathrm{T}$ includes JER	$1.000000\ (3\times10^{-5})$
AK4 $p_\mathrm{T}$	$1.00002\ (5.8\times10^{-4})$	PC's $p_\mathrm{T}$ includes JER	$1.000000\ (7\times10^{-8})$, JER on both	$1.00001\ (6.0\times10^{-4})$

The spreads are NanoAOD's storage precision. $\eta$, $\phi$, $T_\mathrm{Xbb}$, regressed mass, jet ID and MET-filter bits agree too, and $H_\mathrm{T}$ to $3\times10^{-4}$. The few AK4 jets whose JEC factor differs by more than 1% (0.04%) have a NanoAOD-rounded $\eta$ on the other side of a JEC $\eta$-bin edge.

What the comparison found:

• $\phi$ at $\pm\pi$. NanoAOD stores $\phi$ near $\pm\pi$ as $\pm 3.1416016$, outside the $\pm 3.1416$ $\phi$ binning of the 2023BPix-2025 L2Relative, which returns 1 there. These jets (about 0.01%) got no L2Relative and too little $p_\mathrm{T}$, in PC and in A2. This PR now clips $\phi$ to $\pm 3.1415$ for the JEC inputs, as we already do for the jet veto maps; afterwards these jets agree with A1.
• $H_\mathrm{T}$ in the A1 tree. The HT branch is a running sum over the events of the $p_\mathrm{T}$ of all stored AK4 jets: it is not reset per event and has no readme cuts. Recomputed from the A1 jets with the readme definition (corrected $p_\mathrm{T} \geq 30$ GeV, $\lvert\eta\rvert \leq 2.5$, jet ID $&gt; 0$), $H_\mathrm{T}$ agrees with ours (median ratio 1.00001 in MC and 1.00000 in data).
• Jets below the NanoAOD threshold. NanoAOD stores AK4 jets only above 15 GeV with the production JEC. In 0.7% of the data events (0.04% in MC) an A1 jet above 15 GeV after the re-applied JEC lies in a vetoed region but is not in NanoAOD, so the jet veto map removes fewer events in PC and A2.
• MET (A1 stores none). PC's PuppiMET_pt is within 1.8 GeV (68% half-width) of NanoAOD's PuppiMET, while our Type-1 MET with the re-applied JECs differs from both by about 8 GeV. With the production JECs, ours reproduces NanoAOD PuppiMET to 0.3 GeV RMS, so PC's MET does not seem to include the re-applied JECs.
• Softdrop mass. Our skim saves NanoAOD msoftdrop (subjets with the production AK4 JEC): $+0.57%$ (MC) and $-1.6%$ (data) compared with the re-corrected one.

[cmantill]

Lgtm, we could understand better the differences in MET and softdrop. To follow up with Ashwin, but merging this now