Skip to content

Commit ed8d0b5

Browse files
authored
feat(v4): item 7a of the 4.0 cut - flip the df_convention default to "cluster" (M-004..M-006, M-128..M-131) (#879)
1 parent 7cea471 commit ed8d0b5

36 files changed

Lines changed: 1154 additions & 410 deletions

‎DEFERRED.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,9 @@ provenance and AI-review deviation-documentation: a row here (or in
88
- Shippable work lives in [TODO.md](TODO.md); the public feature roadmap in
99
[ROADMAP.md](ROADMAP.md); monitoring and current-state notes in
1010
[docs/dev-status.md](docs/dev-status.md).
11-
- Version-gated lifecycle items (deprecated-kwarg removals, v4 default flips such as
12-
`SyntheticDiD` `lambda_reg`/`zeta` and the clustered-df convention) are canonically
13-
tracked in the CI-enforced ledger `docs/v4-deprecations.yaml`; rows here cross-link
14-
their `M-xxx` ids where applicable.
11+
- Version-gated lifecycle items (deprecated-kwarg removals and default flips) are
12+
canonically tracked in the CI-enforced ledger `docs/v4-deprecations.yaml`; rows here
13+
cross-link their `M-xxx` ids where applicable.
1514
- The `PR` column points to the originating PR number or review tag.
1615

1716
---
@@ -130,7 +129,8 @@ here carry the remaining implementation work and cross-link their `M-xxx` ids.
130129

131130
| Issue | Location | PR | Priority |
132131
|-------|----------|----|----------|
133-
| Clustered inference df **default flip at v4** (lifecycle + roster: docs/v4-deprecations.yaml M-004..M-006 + M-128..M-131; knob semantics: v4-design §7). Implementation work the ledger rows do not carry: the migration note, and flipping the test expectations that pin the current default (`TestDfConvention`, `test_moderate_t_pins_residual_df_convention`, the per-estimator knob suites). | `diff_diff/linalg.py::LinearRegression`, `diff_diff/estimators.py`, `diff_diff/twfe.py`, `diff_diff/sun_abraham.py`, `diff_diff/wooldridge.py`, `diff_diff/stacked_did.py`, `diff_diff/imputation.py` | — | Medium |
132+
133+
No open rows: the clustered-df default flip shipped with the 4.0 cut.
134134

135135
## Decision record — won't-fix / waived
136136

‎TODO.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
8585

8686
| Issue | Location | Origin | Effort | Priority |
8787
|-------|----------|--------|--------|----------|
88+
| Re-execute the nine tutorials whose committed outputs still show the 3.x residual-df p-values/CIs after the `df_convention` default moved to `"cluster"` (t(G-1)): `01_basic_did`, `02_staggered_did`, `04_parallel_trends`, `05_honest_did`, `07_pretrends_power`, `09_real_world_examples`, `11_imputation_did`, `13_stacked_did`, `16_wooldridge_etwfe` (not `16_survey_did`: the survey df wins). Run under `DIFF_DIFF_BACKEND=rust` with the per-worktree kernel, reconcile any prose that restates a moved number, and `--nbmake` each; scheduled for the 4.0 release PR (`docs/v4-design.md` section 9 item 10) together with any item-7b/8/9 movers (24 was refreshed in #879; 23 belongs to 7b). | `docs/tutorials/*.ipynb` | #879 | Mid | Medium |
8889
| TWFE event-study one-way HC2-BM absorbed-FE R parity against the `mpd_absorbed_fe_did` golden (`clubSandwich::vcovCR(cluster=1:n, type="CR2")`): the within spec with hc2_bm auto-clusters at unit, so the one-way lane needs an explicit singleton-cluster column (`cluster=<row id>` + hc2_bm = CR2 with G=n, PT2018 §3.3) once the G=n CR2 path is verified; the hc2 lane is pinned by `TestTWFEEventStudyAbsorbedFERParity` and the clustered hc2_bm per-period lane by `test_twfe_event_study_cluster_hc2_bm_per_period_df_matches_clubsandwich`; the MultiPeriodDiD lanes died at the 4.0 cut (PR 4c-ii) | `tests/test_estimators_vcov_type.py` | #v4-4c-i | Mid | Medium |
8990
| TWFE event-study power profile: `simulate_power`/`simulate_mde`/`simulate_sample_size` have no `_EstimatorProfile` for `TwoWayFixedEffects.fit(event_study=True)` since the MultiPeriodDiD registry entry went with the class (PR 4c-ii); `EventStudyResults` carries no scalar headline, so the profile needs a design (post-period average via the `tests/helpers/event_study_average.py` arithmetic, or a named period) plus its `_SURVEY_FIT_BUILDERS` twin | `diff_diff/power.py` | #v4-4c-ii | Mid | Low |
9091
| Clustered hc2_bm Bell-McCaffrey DOF guard is batch-relative (`linalg._cr2_bm_dof_inner`: a contrast is NaN'd when its scaled max\|B\| falls below 1e-10 of the batch maximum), so the event-study core keeps the otherwise-unused post-period-average contrast column as the anchor to hold the 3.x per-period NaN masks bit-identical (PR 4c-ii, user decision 2026-10-25). Reproduction: `x_nc = treated * 1[period >= 4] + 3e-6 * default_rng(seed).normal(size=n)` as a covariate on the oracle panel flips the mask between `[eye(k), avg]` and `eye(k)` on 20/20 seeds (cond(X) ~ 9e5; 1e-7 never flips, <= 3e-8 rank-drops) - too BLAS-sensitive for a committed oracle, hence the structural pin `test_hc2_bm_contrast_batch_keeps_post_average_anchor`. Decide whether the guard should use a batch-independent anchor (the treatment contrast, or the absolute floor only) and then drop the column | `diff_diff/estimators.py`, `diff_diff/linalg.py` | #v4-4c-ii | Mid | Medium |
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
### Breaking Changes
2+
3+
- **4.0 cut, `df_convention` default flip (`M-004`, `M-005`, `M-006`, `M-128`, `M-129`, `M-130`, `M-131`).** The analytical tail-df convention for CLUSTERED fits defaults to `"cluster"` (the Stata/fixest `t(G - 1)` reference) instead of `"residual"` (the fitted residual df `t(n - K_full)`) on `DifferenceInDifferences`, `TwoWayFixedEffects`, `LinearRegression`, `SunAbraham`, `WooldridgeDiD` (OLS arms), `StackedDiD` and `ImputationDiD` (pretrends leads); `LPDiD` already defaulted to `"cluster"` at 3.9. What moves: p-values and confidence intervals on clustered analytical fits - `TwoWayFixedEffects` (static, and event-study with `unit=`), `SunAbraham`, `WooldridgeDiD` OLS and `StackedDiD` cluster at `unit` by default, so their default fits move; `ImputationDiD` moves on the `pretrends=True` lead rows; `DifferenceInDifferences` / `LinearRegression` move only with an explicit `cluster=` / `cluster_ids=`; the analytical lanes inside bootstrapped fits (`WooldridgeDiD` per-cell inference and post-fit `aggregate()`, `ImputationDiD` bootstrapped leads) move too, as do the fake-timing / fake-group placebo diagnostics and the power simulators that relay those p-values. What does not: point estimates, standard errors and t-statistics; survey-design fits (survey df keeps precedence); `vcov_type="hc2_bm"` (Bell-McCaffrey DOF keep precedence); bootstrap percentile p-values / CIs; unclustered and Conley fits; the BJS overall/post inference of `ImputationDiD`; `WooldridgeDiD` logit/poisson arms; permutation and leave-one-out diagnostics. `df_convention="residual"` restores the 3.x inference bit-for-bit; on the surfaces where the knob is a documented no-op (non-`pretrends` `ImputationDiD`, `WooldridgeDiD` logit/poisson) the inert-config `UserWarning` now fires for an explicit `"residual"` rather than for an explicit `"cluster"`. `SyntheticDiD` inherits the flipped attribute from `DifferenceInDifferences` without reading it (no behaviour change).
4+
5+
### Internal
6+
7+
- Ledger rows `M-004`..`M-006` and `M-128`..`M-131` flip to `done` with `tests/test_v4_inference_defaults.py` as their `test_ref`; the spec's section 7 / item 7 SHIPPED notes and item 10 now record the deferred bulk re-execution of the nine tutorials whose clustered-default outputs moved (24 re-executed here if its outputs moved); the REGISTRY TwoWayFixedEffects clustered-CR1 inference-df note is relabelled a plain Note (the default now matches fixest) and the SunAbraham / ImputationDiD / StackedDiD / WooldridgeDiD / LPDiD tail-df notes read the 4.0 default; `TestDfConvention`, `test_moderate_t_pins_cluster_df_convention` and the per-estimator knob suites pin the G-1 default against explicit `"residual"` baselines (every default-vs-`"cluster"` comparison became default-vs-`"residual"` so none passes vacuously); `test_ci_matches_r_twfe` tightens to machine precision; the variance-conventions audit matrix keeps its residual-df rows as explicit `"residual"` arms (the K_full accounting pins) and gains eight G-1 `_default_df` twins (three in the fast set), with `docs/methodology/variance-conventions.md` regenerated; the committed TWFE event-study oracle lanes `pooled_cluster_unit` / `within_cluster_unit` and the ImputationDiD `pretrends_all` contract lanes name the `"residual"` convention they were captured under (metadata-only, no oracle number touched); the `DEFERRED.md` Version-gated (v4) row is retired.

‎diff_diff/estimators.py‎

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -158,22 +158,24 @@ class DifferenceInDifferences(BaseEstimator):
158158
path — absent ``cluster=``, pure Conley spatial HAC applies.
159159
``survey_design=`` + Conley and ``inference='wild_bootstrap'`` +
160160
Conley both raise ``NotImplementedError``.
161-
df_convention : {"residual", "cluster", "normal"}, default "residual"
161+
df_convention : {"residual", "cluster", "normal"}, default "cluster"
162162
Degrees-of-freedom convention for analytical t-statistics, p-values,
163-
and CIs. ``"residual"`` (default) uses the fitted residual df
164-
(``n − K_full``); ``"cluster"`` uses the Stata/fixest cluster df
165-
``G − 1`` on clustered fits — it has no effect on unclustered fits
163+
and CIs. ``"cluster"`` (default since 4.0) uses the Stata/fixest
164+
cluster df ``G − 1`` on clustered fits — it has no effect on unclustered fits
166165
or on ``vcov_type="conley"`` (the combined Conley+cluster product
167166
kernel has no documented ``G − 1`` df reference and keeps the
168-
residual df); ``"normal"`` deliberately uses normal-theory z
169-
inference at the fallback level on every fit, clustered or not.
167+
residual df); ``"residual"`` (the 3.x default) uses the fitted
168+
residual df (``n − K_full``); ``"normal"`` deliberately uses
169+
normal-theory z inference at the fallback level on every fit,
170+
clustered or not.
170171
Applies only at the fallback level of the df resolution under every
171172
value: survey df and per-coefficient Bell-McCaffrey DOF
172173
(``vcov_type="hc2_bm"``) are more refined small-sample corrections
173174
and always take precedence. Point estimates, SEs, and t-statistics
174175
are unaffected — only the reference distribution changes. The
175-
default flips to ``"cluster"`` at v4 (see the REGISTRY clustered-CR1
176-
inference-df deviation note).
176+
default flipped from ``"residual"`` to ``"cluster"`` at 4.0 (rows
177+
M-004/M-005; pass ``df_convention="residual"`` to reproduce 3.x
178+
inference; see the REGISTRY clustered-CR1 inference-df note).
177179
178180
Attributes
179181
----------
@@ -238,7 +240,7 @@ def __init__(
238240
conley_metric: str = "haversine",
239241
conley_kernel: str = "bartlett",
240242
conley_lag_cutoff: Optional[int] = None,
241-
df_convention: str = "residual",
243+
df_convention: str = "cluster",
242244
):
243245
# Resolve/validate vcov_type via the shared helper so __init__ and
244246
# set_params use identical validation logic.
@@ -282,12 +284,11 @@ def __init__(
282284
# arrays are auto-derived from data[time].values + data[unit].values
283285
# at fit-time (panel estimators already take time/unit as column names).
284286
self.conley_lag_cutoff = conley_lag_cutoff
285-
# Inference df convention for clustered analytical fits: "residual"
286-
# (default; t/p/CI at the fitted residual df) or "cluster" (the
287-
# Stata/fixest G-1 convention). Survey df and per-coefficient
288-
# Bell-McCaffrey DOF always take precedence over either. The default
289-
# flips to "cluster" at v4 (REGISTRY clustered-CR1 inference-df
290-
# deviation note).
287+
# Inference df convention for clustered analytical fits: "cluster"
288+
# (default since 4.0; the Stata/fixest G-1 convention) or
289+
# "residual" (the 3.x default; t/p/CI at the fitted residual df).
290+
# Survey df and per-coefficient Bell-McCaffrey DOF always take
291+
# precedence over either (REGISTRY clustered-CR1 inference-df note).
291292
self.df_convention = df_convention
292293

293294
self.is_fitted_ = False
@@ -2009,7 +2010,8 @@ def _refit_mp_absorb(w_r):
20092010
n_eff_df = int(round(np.sum(survey_weights)))
20102011
df = n_eff_df - k_effective - n_absorbed_effects
20112012
_df_cluster_knob_invalid = False
2012-
# Opt-in Stata/fixest cluster-df convention (df_convention="cluster"):
2013+
# Stata/fixest cluster-df convention (df_convention="cluster", the 4.0
2014+
# default):
20132015
# the shared analytical df becomes G - 1 on a clustered fit. Placed
20142016
# BEFORE the survey/replicate overrides below (which overwrite df, so
20152017
# survey df keeps precedence) and upstream of the per-period BM-DOF

‎diff_diff/guides/llms-full.txt‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ DifferenceInDifferences(
6060
bootstrap_weights: str = "rademacher", # "rademacher", "webb", or "mammen"
6161
seed: int | None = None, # Random seed
6262
rank_deficient_action: str = "warn", # "warn", "error", or "silent"
63-
df_convention: str = "residual", # Analytical t/p/CI df: "residual" (n-K, default), "cluster" (Stata/fixest G-1, clustered fits), or "normal" (deliberate z); survey df + hc2_bm BM-DOF keep precedence; default flips at v4
63+
df_convention: str = "cluster", # Analytical t/p/CI df: "cluster" (Stata/fixest G-1 on clustered fits; default since 4.0), "residual" (n-K; the 3.x default), or "normal" (deliberate z); survey df + hc2_bm BM-DOF keep precedence
6464
)
6565
```
6666

@@ -108,7 +108,7 @@ Two-Way Fixed Effects estimator for panel data. Inherits from DifferenceInDiffer
108108
TwoWayFixedEffects(
109109
cluster: str | None = None, # Auto-clusters at unit level if None
110110
alpha: float = 0.05,
111-
df_convention: str = "residual", # Analytical t/p/CI df: "residual" (default), "cluster" (G-1), or "normal" (z); flips at v4
111+
df_convention: str = "cluster", # Analytical t/p/CI df: "cluster" (G-1; default since 4.0), "residual" (n-K; 3.x default), or "normal" (z)
112112
)
113113
```
114114

@@ -376,7 +376,7 @@ SunAbraham(
376376
conley_metric: str = "haversine", # "haversine" | "euclidean"
377377
conley_kernel: str = "bartlett", # "bartlett" | "uniform"
378378
conley_lag_cutoff: int | None = None, # within-unit Bartlett max lag (0 = spatial-only)
379-
df_convention: str = "residual", # Analytical t/p/CI df for cells AND aggregates (3.9: aggregates share the cells' df — previously z): "residual" (default), "cluster" (G-1), "normal" (z); survey/BM DOF keep precedence; flips at v4
379+
df_convention: str = "cluster", # Analytical t/p/CI df for cells AND aggregates (3.9: aggregates share the cells' df — previously z): "cluster" (G-1; default since 4.0), "residual" (3.x default), "normal" (z); survey/BM DOF keep precedence
380380
)
381381
```
382382

@@ -424,7 +424,7 @@ ImputationDiD(
424424
aux_partition: str = "cohort_horizon", # "cohort_horizon", "cohort", or "horizon"
425425
pretrends: bool = False, # Include pre-treatment horizons in event study
426426
leave_one_out: bool = False, # BJS 2024 App. A.9 leave-one-out finite-sample variance (larger, less-downward-biased SE)
427-
df_convention: str = "residual", # Pretrends lead-regression per-lead t/p/CI only (3.9: leads moved z -> t(residual)): "residual" (default), "cluster" (G-1), "normal" (z); BJS overall/post inference + Wald F knob-independent; flips at v4
427+
df_convention: str = "cluster", # Pretrends lead-regression per-lead t/p/CI only (3.9: leads moved z -> t(residual); 4.0: default G-1): "cluster" (default since 4.0), "residual" (3.x default), "normal" (z); BJS overall/post inference + Wald F knob-independent
428428
)
429429
```
430430

@@ -1104,7 +1104,7 @@ StackedDiD(
11041104
rank_deficient_action: str = "warn",
11051105
vcov_type: str = "hc1", # {"hc1","hc2_bm"}; classical/hc2 rejected (intrinsically clustered), conley deferred. survey_design=... requires hc1
11061106
balance: str = "none", # {"none","entropy"}; "entropy" = CBWSDID covariate balancing (Ustyuzhanin 2026), requires fit(covariates=[...]) + weighting="aggregate", no survey_design
1107-
df_convention: str = "residual", # Analytical t/p/CI df (3.9: hc1 lane moved z -> t(pooled residual df)): "residual" (default), "cluster" (G-1, positive-weight clusters), "normal" (z); BM/survey df keep precedence; flips at v4
1107+
df_convention: str = "cluster", # Analytical t/p/CI df (3.9: hc1 lane moved z -> t(pooled residual df); 4.0: default G-1): "cluster" (G-1, positive-weight clusters; default since 4.0), "residual" (3.x default), "normal" (z); BM/survey df keep precedence
11081108
)
11091109
```
11101110

@@ -1613,7 +1613,7 @@ WooldridgeDiD(
16131613
# column is dropped per Sec 5.4, so cohort_trend_coefs
16141614
# carries G-1 entries. Rejected with survey_design= and
16151615
# with control_group="never_treated".
1616-
df_convention: str = "residual", # OLS analytical t/p/CI df (3.9: default-hc1 arms moved z -> t(residual)): "residual" (default), "cluster" (G-1, hc1-clustered only), "normal" (z); survey/BM DOF keep precedence; GLM arms knob-independent (explicit non-default warns); flips at v4
1616+
df_convention: str = "cluster", # OLS analytical t/p/CI df (3.9: hc1 arms moved z -> t(residual); 4.0: default G-1): "cluster" (G-1, hc1-clustered only; default since 4.0), "residual" (3.x default), "normal" (z); survey/BM DOF keep precedence; GLM arms knob-independent (explicit non-default warns)
16171617
unsupported_period_action: str = "drop", # "drop" warns and filters; "error" refuses unsupported periods
16181618
)
16191619
```
@@ -1834,7 +1834,7 @@ Returned by `SunAbraham.fit()`.
18341834
| `cohort_effects` | `dict` | Cohort-level effects |
18351835
| `df_convention` | `str | None` | The configured df convention echoed onto the results ("residual" | "cluster" | "normal"; 3.9) |
18361836
| `inference_df` | `float | None` | The df the overall-ATT p/CI actually used (BM contrast df / survey df / knob-resolved fallback; None under bootstrap or "normal") |
1837-
| `event_study_df` | `dict[int, float] | None` | Per-event-time df provenance (finite residual df on plain analytic fits since 3.9; None under bootstrap) |
1837+
| `event_study_df` | `dict[int, float] | None` | Per-event-time df provenance (G-1 on plain clustered analytic fits under the 4.0 default; the finite residual df under `df_convention="residual"`; None under bootstrap) |
18381838

18391839
**Methods:** `summary()`, `print_summary()`, `to_dataframe(level="event_study"|"cohort")`
18401840

0 commit comments

Comments
 (0)