Skip to content

feat(v4): item 7a of the 4.0 cut - flip the df_convention default to "cluster" (M-004..M-006, M-128..M-131) - #879

Merged
igerber merged 3 commits into
mainfrom
feat/v4-pr7a-df-convention-flip
Oct 29, 2026
Merged

igerber merged 3 commits into
mainfrom
feat/v4-pr7a-df-convention-flip

Conversation

@igerber

@igerber igerber commented Oct 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • Item 7a of the 4.0 cut (docs/v4-design.md section 9 item 7, first half): the df_convention default flips from "residual" to "cluster" on DifferenceInDifferences (M-004), TwoWayFixedEffects (M-005, inherits DiD's __init__), LinearRegression (M-006), SunAbraham (M-128), WooldridgeDiD OLS arms (M-129, incl. the second hard-coded default in _validate_constructor_args), StackedDiD (M-130) and ImputationDiD pretrends leads (M-131). LPDiD adopted "cluster" at 3.9 and has no row. Clustered analytical p-values / CIs now use the Stata/fixest t(G - 1) reference; point estimates, SEs and t-statistics never move; df_convention="residual" restores 3.x inference bit-for-bit.
  • The two inert-config warnings (WooldridgeDiD GLM arms, non-pretrends ImputationDiD) compare against the new default, so an explicit "residual" warns there and the default never does. SyntheticDiD inherits the flipped attribute without reading it (no behaviour change, pinned).
  • Ledger rows M-004/M-005/M-006/M-128/M-129/M-130/M-131 -> done with tests/test_v4_inference_defaults.py as test_ref; spec section 7 / item 7 SHIPPED notes; item 10 records the deferred bulk re-execution of the nine tutorials whose clustered-default outputs moved (01, 02, 04, 05, 07, 09, 11, 13, 16_wooldridge_etwfe). Item 7b (M-080: SpilloverDiD auto-cluster, uniform cluster=False, metadata gaps) follows in its own PR.
  • Docs: REGISTRY TWFE clustered-CR1 inference-df note relabelled a plain **Note:** (the default now matches fixest) and reworded; SunAbraham / ImputationDiD / StackedDiD / WooldridgeDiD / LPDiD tail-df notes read the 4.0 default; docs/methodology/variance-conventions.md matrix regenerated (residual-df rows kept as explicit "residual" arms, eight G-1 _default_df twins added) and its tail-df prose updated; migration guide gains the worked before/after block and the move / no-move scope; DEFERRED.md Version-gated (v4) row retired; llms-full.txt signature lines updated; changelog fragment added.
  • Tutorial 24 re-executed on the Rust backend (2x2 coverage 0.796 -> 0.800, paired power 0.90 -> 0.89) with its drift-test pins updated; the MDE prose (~0.37 / ~0.60 / ~0.55) still holds.

Methodology references (required if estimator / math changes)

  • Method name(s): clustered-CR1 analytical tail-df convention (df_convention) on DiD / TWFE / LinearRegression / SunAbraham / WooldridgeDiD-OLS / StackedDiD / ImputationDiD pretrends leads
  • Paper / source link(s): fixest::feols(cluster=...) / ssc() and Stata vce(cluster) use t(G - 1) (REGISTRY TwoWayFixedEffects "clustered CR1 inference df" Note; docs/methodology/variance-conventions.md "Tail-df landscape"); the locked test(twfe): tighten live-R fixest parity bands to measured precision (SE-audit C7/C8) #663 direction recorded in docs/v4-design.md section 7
  • Any intentional deviations from the source (and why): none introduced - the flip REMOVES the 3.x deviation from fixest/Stata; the residual-df convention survives as the explicit "residual" option and is documented as such in the REGISTRY note. The "cluster" value stays inert on unclustered and Conley fits (no documented G - 1 reference for the Conley+cluster product kernel), and survey df / hc2_bm Bell-McCaffrey DOF keep precedence under every value (unchanged, REGISTRY).

Validation

  • Tests added/updated: NEW tests/test_v4_inference_defaults.py (signature defaults, get/set_params round trips, every surface's default G-1 tail against its own cluster count with explicit "residual" twins, bootstrapped Wooldridge/Imputation analytical lanes, inert surfaces, the unchanged one-effective-cluster raise, the ledger); tests/test_estimators_vcov_type.py (TestDfConvention inverted; every default-vs-"cluster" comparison became default-vs-"residual" so none passes vacuously), tests/test_sun_abraham.py, tests/test_stacked_did.py, tests/test_wooldridge.py, tests/test_imputation.py (knob suites inverted the same way), tests/test_methodology_twfe.py (test_moderate_t_pins_cluster_df_convention; test_ci_matches_r_twfe tightened to rtol 1e-9 - R/fixest parity now at machine precision, verified locally against live fixest), tests/test_variance_conventions.py (explicit "residual" on the kept rows, eight G-1 twins, three in the fast set, test_disconnected_end_to_end_twfe keeps its 115.0 residual pin plus a G-1 default arm), tests/test_aggregate_contract.py + tests/helpers/twfe_es_oracle_lanes.py + tests/fixtures/v4_twfe_event_study_oracles.json echo (the oracle lanes name the "residual" convention they were captured under - metadata-only, no oracle number touched), tests/test_t24_staggered_vs_collapsed_power_drift.py (two quoted literals), docstring-only edits in tests/test_fixest_did_twfe_parity.py, tests/test_event_study_surface.py, tests/test_lpdid.py.
  • Backtest / simulation / notebook evidence (if applicable): docs/tutorials/24_staggered_vs_collapsed_power.ipynb re-executed (Rust backend) and validated with nbmake; full default suite 15305 passed / 202 skipped; tests/test_variance_conventions.py -m '' full sweep green; tests/test_aggregate_contract.py green on both backends; CI-pinned mypy clean (111 files).

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Changelog

  • changelog.d/ fragment added (or N/A - no user-visible change): Yes - changelog.d/20261028-v4-pr7a-df-convention-flip.md

…"cluster" (M-004..M-006, M-128..M-131)

The analytical tail-df convention for clustered fits defaults to "cluster"
(the Stata/fixest t(G-1) reference) instead of "residual" (the fitted
residual df t(n - K_full)) on DifferenceInDifferences, TwoWayFixedEffects
(inherits DiD's __init__), LinearRegression, SunAbraham, WooldridgeDiD (OLS
arms, including the second hard-coded default in _validate_constructor_args),
StackedDiD and ImputationDiD (pretrends leads). LPDiD adopted "cluster" at
3.9 and has no row. The two inert-config warnings (Wooldridge GLM arms,
non-pretrends ImputationDiD) now compare against the new default, so an
explicit "residual" warns there and the default never does. SyntheticDiD
inherits the flipped attribute without reading it. Docstrings, comments and
the llms-full guide read the 4.0 default; the REGISTRY TWFE clustered-CR1
inference-df note is relabelled a plain Note (the default now matches
fixest) and the SunAbraham / ImputationDiD / StackedDiD / WooldridgeDiD /
LPDiD tail-df notes read the 4.0 default.

What moves: p-values / CIs on clustered analytical fits (the auto-clustered
TWFE / SunAbraham / Wooldridge-OLS / StackedDiD defaults, the ImputationDiD
lead rows, DiD / LinearRegression with an explicit cluster, the analytical
lanes inside bootstrapped Wooldridge and ImputationDiD fits, and the placebo
diagnostics / power simulators that relay those p-values). What does not:
point estimates, SEs, t-statistics; survey df and hc2_bm BM dof (precedence);
bootstrap percentiles; unclustered and Conley fits; BJS overall inference;
Wooldridge GLM arms. df_convention="residual" restores 3.x bit-for-bit.

Tests: new tests/test_v4_inference_defaults.py (the rows' test_ref) pins the
signature defaults, get/set_params round trips, every surface's default
G-1 tail against its own cluster count with an explicit "residual" twin,
the inert surfaces, the unchanged one-effective-cluster raise and the
ledger. TestDfConvention, test_moderate_t_pins_cluster_df_convention (the
inverted methodology pin) and the per-estimator knob suites pin the G-1
default against explicit "residual" baselines - every former
default-vs-"cluster" comparison became default-vs-"residual" so none passes
vacuously; test_ci_matches_r_twfe tightens to rtol 1e-9 (the old 0.003 band
was the df difference). The variance-conventions matrix keeps its
residual-df rows as explicit "residual" arms (the K_full accounting pins)
and gains eight G-1 _default_df twins (three in the fast set); the doc table
is regenerated. The committed TWFE event-study oracle lanes
pooled_cluster_unit / within_cluster_unit and the ImputationDiD pretrends_all
contract lanes name the "residual" convention they were captured under
(metadata-only; no oracle number touched). Tutorial 24 re-executed on the
Rust backend (2x2 coverage 0.796 -> 0.800, power 0.90 -> 0.89) with its
drift pins updated.

Ledger: M-004/M-005/M-006/M-128/M-129/M-130/M-131 -> done. Spec section 7 /
item 7 SHIPPED notes; item 10 records the deferred bulk re-execution of the
nine tutorials whose clustered-default outputs moved. Migration guide gains
the worked before/after block and the move / no-move scope; the DEFERRED.md
Version-gated (v4) row is retired.
@github-actions

Copy link
Copy Markdown

Overall assessment: ✅ Looks good

No unmitigated P0 or P1 findings identified. One nonblocking documentation/debt-tracking item remains.

Executive summary

  • The seven changed defaults consistently select clustered analytical inference at t(G−1).
  • Independent calculations reproduce p-values and confidence levels from all four committed fixest clustered reference cases.
  • Default propagation, survey/Bell–McCaffrey precedence, Conley exclusions, and analytical bootstrap paths appear consistent.
  • Static checks and changelog validation passed. Runtime tests could not run because pytest and scientific Python dependencies are unavailable.

Methodology

Affected surfaces: DiD, TWFE, LinearRegression, Sun–Abraham, Wooldridge OLS, StackedDiD, and BJS pretrends leads. The changes affect reference distributions; estimator equations, identification assumptions, weighting, and variance formulas remain unchanged.

Cross-checks used the registry, in-code references to Sun–Abraham, Wooldridge, Wing et al., and Borusyak et al., plus the committed fixest generator and golden results. External source retrieval was unavailable because network resolution failed.

  • P3 — Informational: documented inference conventions.
    Location: docs/methodology/REGISTRY.md:L595-L624
    Impact: Explicit "residual" retains legacy inference; documented Conley and higher-precedence inference exceptions remain intentional. These are not defects.
    Concrete fix: None required.

Code Quality

No findings. All four LinearRegression construction sites propagate self.df_convention; the diff introduces no inline inference computation or partial NaN guard.

Performance

No findings. The executable changes select existing inference branches without adding material computation.

Maintainability

No findings. Constructor defaults, inert-configuration warnings, ledger entries, and parameter round-trip expectations are updated consistently.

Tech Debt

  • P2 — Record deferred tutorial refreshes in the canonical backlog.
    Location: docs/v4-design.md:L1215-L1223
    Impact: Nine tutorials retain outputs generated under the previous default. Their refresh is scheduled in the design document, but lacks a corresponding TODO.md or DEFERRED.md row under the stated tracking convention.
    Concrete fix: Add a row referencing this PR, listing the nine notebooks and their release-time re-execution requirement; alternatively, refresh their outputs now.

Security

No findings in the reviewed changes.

Documentation/Tests

The required changelog fragment is present and passes validation. Added and updated tests distinguish the new default from explicit "residual" and cover inference exceptions and analytical lanes within bootstrap fits.

AST parsing passed for 24 changed Python files. Notebook code parses and contains no stored execution errors. Runtime test results reported in the PR could not be independently verified in this environment.

…on flip

CI review on PR #879 (P2): the nine tutorials whose committed outputs
still show the 3.x residual-df p-values/CIs were scheduled only in the
spec's item 10. Add the TODO.md Actionable Backlog row (Testing / docs)
listing the notebooks, the Rust-backend re-execution recipe and the
release-PR timing, so the obligation lives in the canonical backlog.
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 9c0645bec9c761695c9064a707ced896e26aa217


Overall assessment: ✅ Looks good

No unmitigated P0 or P1 findings. The prior P2 backlog-tracking finding is resolved.

Executive summary

Methodology

Affected methods: DiD, TWFE, LinearRegression, Sun–Abraham, Wooldridge OLS, StackedDiD, and ImputationDiD pretrends leads. The changes select an existing reference distribution; estimator equations, identification assumptions, weights, and variance formulas remain unchanged.

Cross-checks covered the Methodology Registry, paper-linked estimator docstrings, and committed fixest generator/results. External source retrieval failed because network resolution was unavailable.

  • P3 — Informational: documented inference conventions.
    Location: docs/methodology/REGISTRY.md:L595-L624
    Impact: The default matches fixest/Stata’s clustered reference. Explicit "residual", Conley exclusions, and survey/Bell–McCaffrey precedence remain documented choices. No undocumented mismatch identified.
    Concrete fix: None required.

Code Quality

No findings. All four LinearRegression construction sites propagate self.df_convention. The diff introduces no inline inference computation or partial NaN guards.

Performance

No findings. The default changes select existing inference branches without material additional computation.

Maintainability

No findings. Constructor defaults, validation defaults, inert-configuration warnings, ledger entries, and parameter expectations are consistent.

Tech Debt

Security

No findings in the reviewed changes.

Documentation/Tests

No additional findings. The required changelog fragment is present, and changelog validation passed. All 24 changed Python files parse; Tutorial 24’s nine code cells parse and contain no stored execution errors.

Tests cover default-versus-residual behavior, parameter round trips, inference precedence, and analytical paths within bootstrap fits. Reported runtime results could not be independently verified here.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Oct 28, 2026
CI on PR #879 collected tests/test_v4_inference_defaults.py and failed at
import on every leg: the module imported pyyaml, which is not a test
dependency (the library depends on numpy, pandas and scipy only, and the
matrix gate parses docs/v4-deprecations.yaml with its own restricted-YAML
parser for that reason). The ledger pin now reads the rows the gate already
parsed (tests.test_v4_matrix.ROWS, the same sibling import
tests/test_naming_guard.py uses); the local run had passed only because the
dev venv happens to carry pyyaml.
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: aedf4d3118276e255b23455033041690e24c4ddd


Overall assessment: ✅ Looks good

No unmitigated P0 or P1 findings. The prior tutorial-refresh tracking finding remains resolved.

Executive summary

Methodology

Affected methods: DiD, TWFE, LinearRegression, Sun–Abraham, Wooldridge OLS, StackedDiD, and ImputationDiD pretrends leads.

Cross-checks covered the Methodology Registry, estimator docstrings/references, and committed fixest generator/results. External paper/documentation retrieval failed because network resolution was unavailable.

  • P3 — Informational: documented inference conventions.
    Location: docs/methodology/REGISTRY.md:L595-L627
    Impact: The new default matches fixest’s clustered reference. Explicit "residual", Conley exclusions, and survey/Bell–McCaffrey precedence remain documented choices. No undocumented methodology mismatch identified.
    Concrete fix: None required.

The diff changes reference-distribution defaults without changing estimator equations, weights, identification assumptions, or variance formulas.

Code Quality

No findings. Constructor defaults propagate through callers, parameter handling, aggregation, and result metadata. No new inline inference computation or partial NaN guards were introduced.

Performance

No findings. The changes select existing inference branches without material additional computation.

Maintainability

No findings. Defaults, validation, inert-configuration warnings, and ledger entries are consistent. The new ledger-test import reuses the existing parser without requiring PyYAML.

Tech Debt

Security

No findings in the reviewed changes.

Documentation/Tests

No additional findings. The required changelog fragment is present, and changelog validation passed. All 24 changed Python files parse; Tutorial 24’s nine code cells parse and contain no stored execution errors.

Tests cover default-versus-residual behavior, parameter round trips, inference precedence, and analytical paths within bootstrap fits. Runtime results could not be independently verified because pytest, NumPy, SciPy, and pandas are unavailable.

@igerber
igerber merged commit ed8d0b5 into main Oct 29, 2026
35 checks passed
@igerber
igerber deleted the feat/v4-pr7a-df-convention-flip branch October 29, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant