Repository navigation
feat(v4): item 7a of the 4.0 cut - flip the df_convention default to "cluster" (M-004..M-006, M-128..M-131) - #879
Conversation
…"cluster" (M-004..M-006, M-128..M-131) The analytical tail-df convention for clustered fits defaults to "cluster" (the Stata/fixest t(G-1) reference) instead of "residual" (the fitted residual df t(n - K_full)) on DifferenceInDifferences, TwoWayFixedEffects (inherits DiD's __init__), LinearRegression, SunAbraham, WooldridgeDiD (OLS arms, including the second hard-coded default in _validate_constructor_args), StackedDiD and ImputationDiD (pretrends leads). LPDiD adopted "cluster" at 3.9 and has no row. The two inert-config warnings (Wooldridge GLM arms, non-pretrends ImputationDiD) now compare against the new default, so an explicit "residual" warns there and the default never does. SyntheticDiD inherits the flipped attribute without reading it. Docstrings, comments and the llms-full guide read the 4.0 default; the REGISTRY TWFE clustered-CR1 inference-df note is relabelled a plain Note (the default now matches fixest) and the SunAbraham / ImputationDiD / StackedDiD / WooldridgeDiD / LPDiD tail-df notes read the 4.0 default. What moves: p-values / CIs on clustered analytical fits (the auto-clustered TWFE / SunAbraham / Wooldridge-OLS / StackedDiD defaults, the ImputationDiD lead rows, DiD / LinearRegression with an explicit cluster, the analytical lanes inside bootstrapped Wooldridge and ImputationDiD fits, and the placebo diagnostics / power simulators that relay those p-values). What does not: point estimates, SEs, t-statistics; survey df and hc2_bm BM dof (precedence); bootstrap percentiles; unclustered and Conley fits; BJS overall inference; Wooldridge GLM arms. df_convention="residual" restores 3.x bit-for-bit. Tests: new tests/test_v4_inference_defaults.py (the rows' test_ref) pins the signature defaults, get/set_params round trips, every surface's default G-1 tail against its own cluster count with an explicit "residual" twin, the inert surfaces, the unchanged one-effective-cluster raise and the ledger. TestDfConvention, test_moderate_t_pins_cluster_df_convention (the inverted methodology pin) and the per-estimator knob suites pin the G-1 default against explicit "residual" baselines - every former default-vs-"cluster" comparison became default-vs-"residual" so none passes vacuously; test_ci_matches_r_twfe tightens to rtol 1e-9 (the old 0.003 band was the df difference). The variance-conventions matrix keeps its residual-df rows as explicit "residual" arms (the K_full accounting pins) and gains eight G-1 _default_df twins (three in the fast set); the doc table is regenerated. The committed TWFE event-study oracle lanes pooled_cluster_unit / within_cluster_unit and the ImputationDiD pretrends_all contract lanes name the "residual" convention they were captured under (metadata-only; no oracle number touched). Tutorial 24 re-executed on the Rust backend (2x2 coverage 0.796 -> 0.800, power 0.90 -> 0.89) with its drift pins updated. Ledger: M-004/M-005/M-006/M-128/M-129/M-130/M-131 -> done. Spec section 7 / item 7 SHIPPED notes; item 10 records the deferred bulk re-execution of the nine tutorials whose clustered-default outputs moved. Migration guide gains the worked before/after block and the move / no-move scope; the DEFERRED.md Version-gated (v4) row is retired.
Overall assessment: ✅ Looks goodNo unmitigated P0 or P1 findings identified. One nonblocking documentation/debt-tracking item remains. Executive summary
MethodologyAffected surfaces: DiD, TWFE, LinearRegression, Sun–Abraham, Wooldridge OLS, StackedDiD, and BJS pretrends leads. The changes affect reference distributions; estimator equations, identification assumptions, weighting, and variance formulas remain unchanged. Cross-checks used the registry, in-code references to Sun–Abraham, Wooldridge, Wing et al., and Borusyak et al., plus the committed fixest generator and golden results. External source retrieval was unavailable because network resolution failed.
Code QualityNo findings. All four PerformanceNo findings. The executable changes select existing inference branches without adding material computation. MaintainabilityNo findings. Constructor defaults, inert-configuration warnings, ledger entries, and parameter round-trip expectations are updated consistently. Tech Debt
SecurityNo findings in the reviewed changes. Documentation/TestsThe required changelog fragment is present and passes validation. Added and updated tests distinguish the new default from explicit AST parsing passed for 24 changed Python files. Notebook code parses and contains no stored execution errors. Runtime test results reported in the PR could not be independently verified in this environment. |
…on flip CI review on PR #879 (P2): the nine tutorials whose committed outputs still show the 3.x residual-df p-values/CIs were scheduled only in the spec's item 10. Add the TODO.md Actionable Backlog row (Testing / docs) listing the notebooks, the Rust-backend re-execution recipe and the release-PR timing, so the obligation lives in the canonical backlog.
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall assessment: ✅ Looks goodNo unmitigated P0 or P1 findings. The prior P2 backlog-tracking finding is resolved. Executive summary
MethodologyAffected methods: DiD, TWFE, LinearRegression, Sun–Abraham, Wooldridge OLS, StackedDiD, and ImputationDiD pretrends leads. The changes select an existing reference distribution; estimator equations, identification assumptions, weights, and variance formulas remain unchanged. Cross-checks covered the Methodology Registry, paper-linked estimator docstrings, and committed fixest generator/results. External source retrieval failed because network resolution was unavailable.
Code QualityNo findings. All four PerformanceNo findings. The default changes select existing inference branches without material additional computation. MaintainabilityNo findings. Constructor defaults, validation defaults, inert-configuration warnings, ledger entries, and parameter expectations are consistent. Tech Debt
SecurityNo findings in the reviewed changes. Documentation/TestsNo additional findings. The required changelog fragment is present, and changelog validation passed. All 24 changed Python files parse; Tutorial 24’s nine code cells parse and contain no stored execution errors. Tests cover default-versus-residual behavior, parameter round trips, inference precedence, and analytical paths within bootstrap fits. Reported runtime results could not be independently verified here. |
CI on PR #879 collected tests/test_v4_inference_defaults.py and failed at import on every leg: the module imported pyyaml, which is not a test dependency (the library depends on numpy, pandas and scipy only, and the matrix gate parses docs/v4-deprecations.yaml with its own restricted-YAML parser for that reason). The ledger pin now reads the rows the gate already parsed (tests.test_v4_matrix.ROWS, the same sibling import tests/test_naming_guard.py uses); the local run had passed only because the dev venv happens to carry pyyaml.
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall assessment: ✅ Looks goodNo unmitigated P0 or P1 findings. The prior tutorial-refresh tracking finding remains resolved. Executive summary
MethodologyAffected methods: DiD, TWFE, LinearRegression, Sun–Abraham, Wooldridge OLS, StackedDiD, and ImputationDiD pretrends leads. Cross-checks covered the Methodology Registry, estimator docstrings/references, and committed fixest generator/results. External paper/documentation retrieval failed because network resolution was unavailable.
The diff changes reference-distribution defaults without changing estimator equations, weights, identification assumptions, or variance formulas. Code QualityNo findings. Constructor defaults propagate through callers, parameter handling, aggregation, and result metadata. No new inline inference computation or partial NaN guards were introduced. PerformanceNo findings. The changes select existing inference branches without material additional computation. MaintainabilityNo findings. Defaults, validation, inert-configuration warnings, and ledger entries are consistent. The new ledger-test import reuses the existing parser without requiring PyYAML. Tech Debt
SecurityNo findings in the reviewed changes. Documentation/TestsNo additional findings. The required changelog fragment is present, and changelog validation passed. All 24 changed Python files parse; Tutorial 24’s nine code cells parse and contain no stored execution errors. Tests cover default-versus-residual behavior, parameter round trips, inference precedence, and analytical paths within bootstrap fits. Runtime results could not be independently verified because pytest, NumPy, SciPy, and pandas are unavailable. |
Summary
docs/v4-design.mdsection 9 item 7, first half): thedf_conventiondefault flips from"residual"to"cluster"onDifferenceInDifferences(M-004),TwoWayFixedEffects(M-005, inherits DiD's__init__),LinearRegression(M-006),SunAbraham(M-128),WooldridgeDiDOLS arms (M-129, incl. the second hard-coded default in_validate_constructor_args),StackedDiD(M-130) andImputationDiDpretrends leads (M-131).LPDiDadopted"cluster"at 3.9 and has no row. Clustered analytical p-values / CIs now use the Stata/fixestt(G - 1)reference; point estimates, SEs and t-statistics never move;df_convention="residual"restores 3.x inference bit-for-bit.WooldridgeDiDGLM arms, non-pretrendsImputationDiD) compare against the new default, so an explicit"residual"warns there and the default never does.SyntheticDiDinherits the flipped attribute without reading it (no behaviour change, pinned).donewithtests/test_v4_inference_defaults.pyastest_ref; spec section 7 / item 7 SHIPPED notes; item 10 records the deferred bulk re-execution of the nine tutorials whose clustered-default outputs moved (01, 02, 04, 05, 07, 09, 11, 13, 16_wooldridge_etwfe). Item 7b (M-080: SpilloverDiD auto-cluster, uniformcluster=False, metadata gaps) follows in its own PR.**Note:**(the default now matches fixest) and reworded; SunAbraham / ImputationDiD / StackedDiD / WooldridgeDiD / LPDiD tail-df notes read the 4.0 default;docs/methodology/variance-conventions.mdmatrix regenerated (residual-df rows kept as explicit"residual"arms, eight G-1_default_dftwins added) and its tail-df prose updated; migration guide gains the worked before/after block and the move / no-move scope;DEFERRED.mdVersion-gated (v4) row retired;llms-full.txtsignature lines updated; changelog fragment added.Methodology references (required if estimator / math changes)
df_convention) on DiD / TWFE / LinearRegression / SunAbraham / WooldridgeDiD-OLS / StackedDiD / ImputationDiD pretrends leadsfixest::feols(cluster=...)/ssc()and Statavce(cluster)uset(G - 1)(REGISTRY TwoWayFixedEffects "clustered CR1 inference df" Note;docs/methodology/variance-conventions.md"Tail-df landscape"); the locked test(twfe): tighten live-R fixest parity bands to measured precision (SE-audit C7/C8) #663 direction recorded indocs/v4-design.mdsection 7"residual"option and is documented as such in the REGISTRY note. The"cluster"value stays inert on unclustered and Conley fits (no documentedG - 1reference for the Conley+cluster product kernel), and survey df / hc2_bm Bell-McCaffrey DOF keep precedence under every value (unchanged, REGISTRY).Validation
tests/test_v4_inference_defaults.py(signature defaults, get/set_params round trips, every surface's default G-1 tail against its own cluster count with explicit"residual"twins, bootstrapped Wooldridge/Imputation analytical lanes, inert surfaces, the unchanged one-effective-cluster raise, the ledger);tests/test_estimators_vcov_type.py(TestDfConventioninverted; every default-vs-"cluster"comparison became default-vs-"residual"so none passes vacuously),tests/test_sun_abraham.py,tests/test_stacked_did.py,tests/test_wooldridge.py,tests/test_imputation.py(knob suites inverted the same way),tests/test_methodology_twfe.py(test_moderate_t_pins_cluster_df_convention;test_ci_matches_r_twfetightened to rtol 1e-9 - R/fixest parity now at machine precision, verified locally against live fixest),tests/test_variance_conventions.py(explicit"residual"on the kept rows, eight G-1 twins, three in the fast set,test_disconnected_end_to_end_twfekeeps its 115.0 residual pin plus a G-1 default arm),tests/test_aggregate_contract.py+tests/helpers/twfe_es_oracle_lanes.py+tests/fixtures/v4_twfe_event_study_oracles.jsonecho (the oracle lanes name the"residual"convention they were captured under - metadata-only, no oracle number touched),tests/test_t24_staggered_vs_collapsed_power_drift.py(two quoted literals), docstring-only edits intests/test_fixest_did_twfe_parity.py,tests/test_event_study_surface.py,tests/test_lpdid.py.docs/tutorials/24_staggered_vs_collapsed_power.ipynbre-executed (Rust backend) and validated with nbmake; full default suite 15305 passed / 202 skipped;tests/test_variance_conventions.py -m ''full sweep green;tests/test_aggregate_contract.pygreen on both backends; CI-pinned mypy clean (111 files).Security / privacy
Changelog
changelog.d/20261028-v4-pr7a-df-convention-flip.md