diff --git a/TODO.md b/TODO.md
index cec98268d..e0a95e517 100644
--- a/TODO.md
+++ b/TODO.md
@@ -86,7 +86,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
| Issue | Location | Origin | Effort | Priority |
|-------|----------|--------|--------|----------|
-| Re-execute the nine tutorials whose committed outputs still show the 3.x residual-df p-values/CIs after the `df_convention` default moved to `"cluster"` (t(G-1)): `01_basic_did`, `02_staggered_did`, `04_parallel_trends`, `05_honest_did`, `07_pretrends_power`, `09_real_world_examples`, `11_imputation_did`, `13_stacked_did`, `16_wooldridge_etwfe` (not `16_survey_did`: the survey df wins). Run under `DIFF_DIFF_BACKEND=rust` with the per-worktree kernel, reconcile any prose that restates a moved number, and `--nbmake` each; scheduled for the 4.0 release PR (`docs/v4-design.md` section 9 item 10) together with any item-7c/8/9 movers (24 was refreshed in #879; 23 in 7b and again in 7c). 7b's label changes also drift any committed analytical DiD/TWFE summary (", G=" on the variance line) and every SunAbraham summary (the new "Std. errors:" line) - covered by the same re-execution. | `docs/tutorials/*.ipynb` | #879 | Mid | Medium |
+| Re-execute the nine tutorials whose committed outputs still show the 3.x residual-df p-values/CIs after the `df_convention` default moved to `"cluster"` (t(G-1)): `01_basic_did`, `02_staggered_did`, `04_parallel_trends`, `05_honest_did`, `07_pretrends_power`, `09_real_world_examples`, `11_imputation_did`, `13_stacked_did`, `16_wooldridge_etwfe` (not `16_survey_did`: the survey df wins). Run under `DIFF_DIFF_BACKEND=rust` with the per-worktree kernel, reconcile any prose that restates a moved number, and `--nbmake` each; scheduled for the 4.0 release PR (`docs/v4-design.md` section 9 item 10) together with any item-8/9 movers (24 was refreshed in #879; 23 in 7b, re-validated in 7c - it prints no p-values or CIs). 7b's label changes also drift any committed analytical DiD/TWFE summary (", G=" on the variance line) and every SunAbraham summary (the new "Std. errors:" line) - covered by the same re-execution. | `docs/tutorials/*.ipynb` | #879 | Mid | Medium |
| TWFE event-study one-way HC2-BM absorbed-FE R parity against the `mpd_absorbed_fe_did` golden (`clubSandwich::vcovCR(cluster=1:n, type="CR2")`): since 4.0 (item 7b) `cluster=False` + hc2_bm reaches the one-way lane directly (no singleton-cluster column needed; `tests/test_v4_cluster_policy.py` pins finiteness and the static-surface LinearRegression parity only), so the open item is the R pin itself; the hc2 lane is pinned by `TestTWFEEventStudyAbsorbedFERParity` and the clustered hc2_bm per-period lane by `test_twfe_event_study_cluster_hc2_bm_per_period_df_matches_clubsandwich`; the MultiPeriodDiD lanes died at the 4.0 cut (PR 4c-ii) | `tests/test_estimators_vcov_type.py` | #v4-4c-i | Mid | Medium |
| TWFE event-study power profile: `simulate_power`/`simulate_mde`/`simulate_sample_size` have no `_EstimatorProfile` for `TwoWayFixedEffects.fit(event_study=True)` since the MultiPeriodDiD registry entry went with the class (PR 4c-ii); `EventStudyResults` carries no scalar headline, so the profile needs a design (post-period average via the `tests/helpers/event_study_average.py` arithmetic, or a named period) plus its `_SURVEY_FIT_BUILDERS` twin | `diff_diff/power.py` | #v4-4c-ii | Mid | Low |
| Clustered hc2_bm Bell-McCaffrey DOF guard is batch-relative (`linalg._cr2_bm_dof_inner`: a contrast is NaN'd when its scaled max\|B\| falls below 1e-10 of the batch maximum), so the event-study core keeps the otherwise-unused post-period-average contrast column as the anchor to hold the 3.x per-period NaN masks bit-identical (PR 4c-ii, user decision 2026-10-25). Reproduction: `x_nc = treated * 1[period >= 4] + 3e-6 * default_rng(seed).normal(size=n)` as a covariate on the oracle panel flips the mask between `[eye(k), avg]` and `eye(k)` on 20/20 seeds (cond(X) ~ 9e5; 1e-7 never flips, <= 3e-8 rank-drops) - too BLAS-sensitive for a committed oracle, hence the structural pin `test_hc2_bm_contrast_batch_keeps_post_average_anchor`. Decide whether the guard should use a batch-independent anchor (the treatment contrast, or the absolute floor only) and then drop the column | `diff_diff/estimators.py`, `diff_diff/linalg.py` | #v4-4c-ii | Mid | Medium |
@@ -99,7 +99,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
| Evaluate adding the `BaseEstimator` param surface (get_params/set_params) to the exported classes that never had it - `PowerAnalysis`, `LinearRegression`, `BusinessReport`, `DiagnosticReport`, `TWFEWeightsResult` (a NEW public surface, deliberately out of the 2(c)-i pure-refactor scope; `LinearRegression` is the one `fit`-bearing class excluded from the contract suite's roster-completeness test). | `diff_diff/linalg.py`, `diff_diff/power.py` | mixin PR | Mid | Low |
| Tighten the mypy suppressions that back the enforced-zero posture: burn down `prep_dgp`'s per-module `[index]` override (needs a None-vs-array restructure that preserves the seeded RNG stream), and evaluate re-enabling the globally disabled codes (`arg-type`, `return-value`, `var-annotated`, `assignment`) one at a time — `assignment` alone hid several real annotation drifts found during the 2026-07 triage. | `pyproject.toml` `[tool.mypy]`, `diff_diff/prep_dgp.py` | lint-CI | Mid | Low |
| Route LWDiD into the practitioner code surfaces: `practitioner_next_steps` (`diff_diff/practitioner.py`), `agent_workflow.py`, `DiagnosticReport`, and `BusinessReport` never dispatch to or mention LWDiD — the AI guides now cover it, but the code surfaces need contract decisions (when to recommend it, what diagnostics to run) plus tests | `diff_diff/practitioner.py`, `diff_diff/agent_workflow.py`, `diff_diff/diagnostic_report.py`, `diff_diff/business_report.py` | LWDiD release audit | Mid | Low |
-| df-provenance completion follow-up (PR C descoping): a Wooldridge public per-row df channel (a per-key dict that can also carry the hc2_bm BM contrast dofs the scalar `_df_analytic_fallback` cannot represent — the unified surface's Wooldridge df column stays NaN until then); ImputationDiD NON-survey per-lead df (the survey-fit scalar `event_study_df` + its bootstrap clearing shipped with the M-092 completion; what remains is surfacing the `resolve_tail_df`-derived per-lead dfs on non-survey fits, which needs the `_compute_lead_coefficients` return-path change); and the `inference_df` (DiD/SA/StackedDiD) vs `df_inference` (CS/dCDH) adapter-naming unification. NOTE (M-024): StackedDiD's `df_survey=None` on analytical fits is DELIBERATE - the container resolver's `df_inference` fallback not matching `inference_df` is load-bearing (normal-theory FLCI; see the REGISTRY StackedDiD M-024 Note and the pin in `tests/test_event_study_consumers.py`) - the unification must preserve that outcome or renegotiate it explicitly. | `diff_diff/wooldridge_results.py`, `diff_diff/imputation.py`, `diff_diff/results_base.py` | #variance-PR-C | Mid | Low |
+| df-provenance completion follow-up (PR C descoping): a Wooldridge public per-row df channel (a per-key dict that can also carry the hc2_bm BM contrast dofs the scalar `_df_analytic_fallback` cannot represent — the unified surface's Wooldridge df column stays NaN until then); ImputationDiD NON-survey per-lead df (the survey-fit scalar `event_study_df` + its bootstrap clearing shipped with the M-092 completion; what remains is surfacing the `resolve_tail_df`-derived per-lead dfs on non-survey fits, which needs the `_compute_lead_coefficients` return-path change); and the `inference_df` (DiD/SA/StackedDiD/SpilloverDiD) vs `df_inference` (CS/dCDH) adapter-naming unification. NOTE (M-024): StackedDiD's `df_survey=None` on analytical fits is DELIBERATE - the container resolver's `df_inference` fallback not matching `inference_df` is load-bearing (normal-theory FLCI; see the REGISTRY StackedDiD M-024 Note and the pin in `tests/test_event_study_consumers.py`) - the unification must preserve that outcome or renegotiate it explicitly. | `diff_diff/wooldridge_results.py`, `diff_diff/imputation.py`, `diff_diff/results_base.py` | #variance-PR-C | Mid | Low |
| `worktree-rm` safety via a tested argv helper: the prose rewrite (ask-before-remove confirmation gate, detached-HEAD reachability/rescue, tip-identity force-delete guard) was reverted to the main version because editing the prose repeatedly reintroduced shell-injection (last: sourcing a state file built from a git-derived branch name). Restore those guards in a `worktree_rm.py` that takes the name via file ingress, invokes git through argv arrays, and has a metacharacter-branch/path injection regression test — the pattern that worked for `pr_prepare.py`/`premerge_scan.py`. | `.claude/commands/worktree-rm.md`, `.claude/scripts/` | skill-audit | Heavy | Medium |
| `premerge_scan.py` should scan the staged blob (`git show :path`) for staged methodology files, not the working-tree copy — a stage-then-revert-working-copy edit currently reads the safe working version and misses the staged violation. Union staged-index findings with unstaged/untracked filesystem findings. | `.claude/scripts/premerge_scan.py` | skill-audit | Mid | Low |
| Carousel PDF drift guard: the committed deck PDFs are build artifacts of their `generate_*.py` scripts (README rule: regenerate in the same commit), but nothing verifies a committed PDF matches its generator — a source edit without regeneration, or a stale raster chart, passes every claims test. Applies to ALL decks, not just the MMM one. fpdf embeds creation-date metadata, so a byte-compare needs fixed metadata or a per-page rendered-text manifest. Origin: mmm-carousel local review R1. | `carousel/`, `tests/` | mmm-carousel | Mid | Low |
diff --git a/changelog.d/20261029-v4-pr7c-m154-spillover-df-convention.md b/changelog.d/20261029-v4-pr7c-m154-spillover-df-convention.md
new file mode 100644
index 000000000..661b634ef
--- /dev/null
+++ b/changelog.d/20261029-v4-pr7c-m154-spillover-df-convention.md
@@ -0,0 +1,7 @@
+### Breaking Changes
+
+- `SpilloverDiD` gains the three-value `df_convention` knob with the 4.0 default `"cluster"` ([M-154], item 7c of the 4.0 cut): p-values and confidence intervals on its clustered lanes - the unit auto-cluster and an explicit `cluster=` - now use the Stata/fixest `t(G - 1)` reference like every other auto-clustered default, with `G` the effective cluster count reported as `n_clusters`. Point estimates, standard errors and t-statistics never move. `df_convention="residual"` restores the 3.x residual-df inference bit-for-bit; `"normal"` is deliberate z. `"cluster"` falls back to the residual df on `cluster=False` and on every `vcov_type="conley"` fit, an explicit product-kernel `cluster=` included (`"normal"` still selects z there); a `survey_design` df keeps precedence under every value; a saturated stage-2 sample stays all-NaN under every value. `SpilloverDiDResults` reports `df_convention` and `inference_df` (also in `to_dict()`).
+
+### Internal
+
+- Ledger: M-154 -> `done`, pinned by `tests/test_v4_spillover_df_convention.py`; the df ladder is the pure helper `spillover._resolve_inference_df` (survey df -> saturation -> `utils.resolve_tail_df`). The variance-conventions matrix gains the `spillover_cluster_unit` / `spillover_cluster_unit_default_df` rows; the REGISTRY `df_convention` knob roster, `docs/methodology/variance-conventions.md`, the migration guide, `llms-full.txt` and `docs/api/spillover.rst` describe the knob; tutorial 23 re-validated (it prints no p-values or CIs).
diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt
index b3b41c946..734b97525 100644
--- a/diff_diff/guides/llms-full.txt
+++ b/diff_diff/guides/llms-full.txt
@@ -520,6 +520,7 @@ SpilloverDiD(
conley_cutoff_km: float | None = None,
conley_lag_cutoff: int | None = None,
cluster: str | None | False = None, # None = auto-cluster at unit (4.0; dropped on Conley, never a survey PSU); False = observation-level HC1 (the 3.x default); str = that column
+ df_convention: str = "cluster", # Analytical t/p/CI df (4.0, M-154): "cluster" (G-1 on the unit auto-cluster / explicit cluster=; default), "residual" (n-k; the 3.x inference), "normal" (z); "cluster" falls back to n-k on Conley and cluster=False ("normal" still z there); survey df keeps precedence under every value; results carry df_convention / inference_df
alpha: float = 0.05,
anticipation: int = 0, # non-negative integer; bool rejected
event_study: bool = False, # Wave C: per-event-time × ring decomposition (Butts Table 2)
diff --git a/diff_diff/spillover.py b/diff_diff/spillover.py
index 087d6b36f..b40913e90 100644
--- a/diff_diff/spillover.py
+++ b/diff_diff/spillover.py
@@ -45,9 +45,11 @@
from diff_diff.utils import (
ClusterArg,
_iterative_fe_solve,
+ resolve_tail_df,
safe_inference,
validate_anticipation,
validate_cluster,
+ validate_df_convention,
)
if TYPE_CHECKING:
@@ -827,7 +829,7 @@ def _extract_event_study_results(
rectangular_grid: List[Tuple[str, Optional[str], int]],
n_obs_per_col: np.ndarray,
ref_period: int,
- df_resid: int,
+ df_resid: Optional[float],
alpha: float,
ring_labels: List[str],
weight_sum_per_col: Optional[np.ndarray] = None,
@@ -1080,6 +1082,80 @@ def _extract_event_study_results(
)
+def _resolve_inference_df(
+ *,
+ df_convention: str,
+ n_obs_eff: int,
+ k_effective: int,
+ n_clusters: Optional[int],
+ vcov_mode: str,
+ survey_df: Optional[int],
+ has_survey: bool,
+) -> Optional[float]:
+ """Resolve the ONE tail df every SpilloverDiD ``safe_inference`` call uses.
+
+ Precedence ladder (4.0, [M-154]):
+
+ 1. ``survey_design`` present: ``ResolvedSurveyDesign.df_survey`` (the
+ standard survey DOF; ``None`` -> the ``0`` fail-closed sentinel). The
+ survey df beats every ``df_convention`` value, ``"normal"`` included.
+ 2. Saturated stage-2 sample (``n_obs_eff - k_effective <= 0``): the
+ residual df itself, which the guard below turns into the ``0.0``
+ all-NaN sentinel under every knob value (``resolve_tail_df`` would
+ fall through to normal theory here - misleading on a degenerate
+ sample, so it is never consulted).
+ 3. Otherwise ``diff_diff.utils.resolve_tail_df``: ``"cluster"`` ->
+ ``G - 1`` with ``G`` the effective cluster count of the Wave D
+ ``"cluster"`` variance mode (the 4.0 unit auto-cluster or an explicit
+ ``cluster=``); ``G`` is forced to ``None`` (residual df) on EVERY
+ Conley fit - including one with an explicit product-kernel
+ ``cluster=`` - and on the observation-level HC1 lane
+ (``cluster=False``), the SunAbraham precedent; ``"residual"`` ->
+ ``n_obs_eff - k_effective``; ``"normal"`` -> ``None`` (z).
+
+ Returns the value handed to ``safe_inference``: a positive float,
+ ``None`` (normal theory) or ``0.0`` (all-NaN inference). A non-positive
+ df warns with the saturation message (``stacklevel=3``: the helper is
+ called from ``fit``, so the warning points at the user's call site).
+ """
+ df_resid = n_obs_eff - k_effective
+ df: Optional[float]
+ if has_survey:
+ df = float(survey_df) if survey_df is not None else 0.0
+ elif df_resid <= 0:
+ df = float(df_resid)
+ else:
+ df = resolve_tail_df(
+ df_convention,
+ residual_df=float(df_resid),
+ n_clusters=(n_clusters if vcov_mode == "cluster" else None),
+ )
+ if df is not None and df <= 0:
+ # Saturated. Either OLS-saturated (n - k <= 0) or
+ # survey-saturated (df_survey = 0; lonely_psu='remove' may
+ # have removed all strata). Force NaN inference by setting
+ # df = 0 (safe_inference treats df = 0 as no usable degrees
+ # of freedom). Distinct from df = None which would fall
+ # through to a normal-distribution approximation -
+ # misleading on a degenerate sample.
+ survey_note = (
+ " (survey-saturated: df_survey = "
+ f"{int(survey_df) if survey_df is not None else 0}; "
+ "lonely_psu='remove' may have removed all strata)"
+ if has_survey
+ else ""
+ )
+ warnings.warn(
+ f"SpilloverDiD inference df = {int(df)} (n_obs="
+ f"{n_obs_eff}, effective_rank={k_effective}{survey_note}). "
+ "Inference (t-stat, p-value, CI) will be NaN.",
+ UserWarning,
+ stacklevel=3,
+ )
+ return 0.0
+ return df
+
+
def _build_ring_indicators(
d_values: np.ndarray,
rings: List[float],
@@ -1701,9 +1777,25 @@ class SpilloverDiD(BaseEstimator):
on every lane: CR1, the combined Conley cluster product kernel when
paired with ``vcov_type="conley"``, or the effective PSU under a
``survey_design`` without its own PSU. Column labels must be ``str``.
- Inference uses the residual df on every non-survey lane, the
- default CR1 lane included; under a ``survey_design`` the design df
- takes precedence (``df_convention`` arrives with [M-154]).
+ Inference df: see ``df_convention``.
+ df_convention : {"residual", "cluster", "normal"}, default "cluster"
+ Degrees-of-freedom convention for the analytical t-statistics,
+ p-values and CIs (4.0, row M-154). ``"cluster"`` (default) uses the
+ Stata/fixest cluster df ``G - 1`` on the clustered lanes - the 4.0
+ ``unit`` auto-cluster or an explicit ``cluster=`` column, with ``G``
+ the effective cluster count reported as ``n_clusters``; it is inert
+ (residual df) on ``cluster=False`` and on EVERY ``vcov_type="conley"``
+ fit, including one with an explicit ``cluster=`` (the combined
+ Conley+cluster product kernel has no documented ``G - 1`` df
+ reference). ``"residual"`` uses the stage-2 residual df
+ ``n_obs_eff - k_effective`` (the 3.x inference, and the 7b inference
+ on the default lane); ``"normal"`` deliberately uses normal-theory z
+ inference at the fallback level on every fit. Applies only at the
+ fallback level: a ``survey_design`` df always takes precedence, and
+ a saturated stage-2 sample is all-NaN under every value. Point
+ estimates, SEs and t-statistics never move - only the reference
+ distribution does. Results report the configuration as
+ ``df_convention`` and the df actually used as ``inference_df``.
alpha : float, default=0.05
Significance level for confidence intervals.
anticipation : int, default=0
@@ -1764,6 +1856,7 @@ def __init__(
conley_cutoff_km: Optional[float] = None,
conley_lag_cutoff: Optional[int] = None,
cluster: ClusterArg = None,
+ df_convention: str = "cluster",
alpha: float = 0.05,
anticipation: int = 0,
event_study: bool = False,
@@ -1782,8 +1875,9 @@ def __init__(
if vcov_type in ("hc2", "hc2_bm"):
raise NotImplementedError(
f"SpilloverDiD does not yet support vcov_type='{vcov_type}'. "
- "The current stage-2 inference uses a generic residual df "
- "(n - effective_rank) for t-distribution lookups, but "
+ "The stage-2 inference uses one fallback df per fit (the "
+ "df_convention-resolved df: t(G-1) on clustered lanes by "
+ "default, the residual df otherwise), but "
"hc2 / hc2_bm require per-coefficient Bell-McCaffrey / CR2 "
"degrees of freedom for correct p-values and CIs. Routing "
"stage 2 through LinearRegression (which supplies the "
@@ -1798,6 +1892,7 @@ def __init__(
"vcov_type='hc1' or 'conley', or leave default; combine "
"with cluster= for CR1."
)
+ validate_df_convention(df_convention)
self.rings = rings
self.d_bar = d_bar
self.vcov_type = vcov_type
@@ -1806,6 +1901,7 @@ def __init__(
self.conley_cutoff_km = conley_cutoff_km
self.conley_lag_cutoff = conley_lag_cutoff
self.cluster = validate_cluster(cluster)
+ self.df_convention = df_convention
self.alpha = alpha
self.anticipation = validate_anticipation(anticipation)
self.event_study = event_study
@@ -2237,6 +2333,8 @@ def fit(
# Same direct-mutation defense for the shared cluster contract
# (str | None | False, [M-155]); the lane resolution lives below.
validate_cluster(self.cluster)
+ # ... and for the df_convention knob ([M-154]); resolved in Step 16a.
+ validate_df_convention(self.df_convention)
# Wave C: event-study path is now supported. Validate horizon_max
# up front (fail-fast before any stage-1 work).
if self.horizon_max is not None:
@@ -2312,8 +2410,9 @@ def fit(
if self.vcov_type in ("hc2", "hc2_bm"):
raise NotImplementedError(
f"SpilloverDiD does not yet support vcov_type='{self.vcov_type}'. "
- "The current stage-2 inference uses a generic residual df "
- "(n - effective_rank) for t-distribution lookups, but "
+ "The stage-2 inference uses one fallback df per fit (the "
+ "df_convention-resolved df: t(G-1) on clustered lanes by "
+ "default, the residual df otherwise), but "
"hc2 / hc2_bm require per-coefficient Bell-McCaffrey / CR2 "
"degrees of freedom for correct p-values and CIs. Routing "
"stage 2 through LinearRegression (which supplies the "
@@ -3455,16 +3554,28 @@ def fit(
else:
vcov = vcov_kept
- # Step 16a: shared df_for_inference computation.
+ # Step 16a: shared df_for_inference computation - ONE df for every
+ # safe_inference call below (aggregate tau_total, per-ring delta_j,
+ # event-study per-cell tau_k / delta_jk and the scalar lincom att).
+ #
+ # Precedence ladder in `_resolve_inference_df` (4.0, [M-154]):
+ # 1. Wave E.1 (survey): df = resolved_survey_fit.df_survey, the
+ # standard survey-statistics DOF (PSU + strata -> n_PSU - n_strata;
+ # PSU only -> n_PSU - 1; strata only -> n_obs - n_strata; neither
+ # -> n_obs - 1; see `ResolvedSurveyDesign.df_survey`). The Binder
+ # TSL meat is design-consistent; the OLS residual df is no longer
+ # the right t-distribution DOF - and the survey df beats every
+ # df_convention value.
+ # 2. Saturated stage-2 sample (n_obs_eff - k_effective <= 0): warn +
+ # the df = 0 all-NaN sentinel under every knob value.
+ # 3. Otherwise the shared `df_convention` knob via
+ # `utils.resolve_tail_df`: "cluster" (the 4.0 default) -> G - 1 on
+ # the Wave D "cluster" variance mode (the unit auto-cluster or an
+ # explicit cluster=), residual df on the hc1 lane (cluster=False)
+ # and on EVERY Conley fit (G forced to None, the SunAbraham
+ # precedent); "residual" -> n_obs_eff - k_effective (the 3.x / 7b
+ # inference); "normal" -> None (z).
#
- # Wave D (non-survey): df = n_obs - effective_rank (OLS residual df).
- # Wave E.1 (survey): df = resolved_survey_fit.df_survey, which
- # encodes the standard survey-statistics DOF
- # (PSU + strata → n_PSU - n_strata; PSU only → n_PSU - 1;
- # strata only → n_obs - n_strata; neither → n_obs - 1; see
- # `ResolvedSurveyDesign.df_survey` at survey.py:619-627). The
- # Binder TSL meat is design-consistent; the OLS residual df is
- # no longer the right t-distribution DOF.
# Wave E.3 (codex R8 P2 fix): under the survey path, the effective
# estimation sample EXCLUDES zero-weight subpop rows because they
# are filtered out of the gamma_hat / Psi construction sample by
@@ -3476,35 +3587,29 @@ def fit(
count_mask = survey_finite_mask if resolved_survey_fit is not None else finite_mask
n_obs_eff = int(count_mask.sum())
k_effective = int(np.isfinite(coef).sum())
- df_resid = n_obs_eff - k_effective
- if resolved_survey_fit is not None:
- df_survey_val = resolved_survey_fit.df_survey
- df_for_inference: int = int(df_survey_val) if df_survey_val is not None else 0
- else:
- df_for_inference = df_resid
- if df_for_inference <= 0:
- # Saturated. Either OLS-saturated (n - k <= 0) or
- # survey-saturated (df_survey = 0; lonely_psu='remove' may
- # have removed all strata). Force NaN inference by setting
- # df = 0 (safe_inference treats df = 0 as no usable degrees
- # of freedom). Distinct from df = None which would fall
- # through to a normal-distribution approximation —
- # misleading on a degenerate sample.
- survey_note = (
- " (survey-saturated: df_survey = "
- f"{int(df_survey_val) if df_survey_val is not None else 0}; "
- "lonely_psu='remove' may have removed all strata)"
- if resolved_survey_fit is not None
- else ""
- )
- warnings.warn(
- f"SpilloverDiD inference df = {df_for_inference} (n_obs="
- f"{n_obs_eff}, effective_rank={k_effective}{survey_note}). "
- "Inference (t-stat, p-value, CI) will be NaN.",
- UserWarning,
- stacklevel=2,
- )
- df_for_inference = 0
+ # Effective cluster count of the ids the Wave D meat received (G on
+ # the default lane, the PSU count under a PSU survey design, None on
+ # Conley-auto / cluster=False); feeds both the df ladder and the
+ # results' `n_clusters`.
+ _n_clusters_fit: Optional[int] = (
+ int(len(np.unique(cluster_ids_fit))) if cluster_ids_fit is not None else None
+ )
+ df_for_inference = _resolve_inference_df(
+ df_convention=self.df_convention,
+ n_obs_eff=n_obs_eff,
+ k_effective=k_effective,
+ n_clusters=_n_clusters_fit,
+ vcov_mode=_wave_d_vcov_mode,
+ survey_df=(resolved_survey_fit.df_survey if resolved_survey_fit is not None else None),
+ has_survey=resolved_survey_fit is not None,
+ )
+ # Results metadata: the df the reported t/p/CI actually used - None
+ # under normal theory ("normal") and under the all-NaN sentinel.
+ _inference_df_used: Optional[float] = (
+ float(df_for_inference)
+ if df_for_inference is not None and df_for_inference > 0
+ else None
+ )
# Step 16b: branch on event_study mode for result extraction.
att_dynamic_df: Optional[pd.DataFrame] = None
@@ -3628,10 +3733,10 @@ def fit(
r_squared=None,
inference_method="analytical",
n_bootstrap=None,
- n_clusters=(
- int(len(np.unique(cluster_ids_fit))) if cluster_ids_fit is not None else None
- ),
+ n_clusters=_n_clusters_fit,
vcov_type=self.vcov_type,
+ df_convention=self.df_convention,
+ inference_df=_inference_df_used,
# Wave E.1: when survey.psu wins the warn-and-use-PSU override
# (`_resolve_effective_cluster`), the EFFECTIVE clustering label
# is `survey_design.psu`, not `self.cluster`. Report that so
diff --git a/diff_diff/utils.py b/diff_diff/utils.py
index 366accd81..1a7b8d91e 100644
--- a/diff_diff/utils.py
+++ b/diff_diff/utils.py
@@ -679,7 +679,8 @@ def resolve_tail_df(
The single implementation of the ``df_convention`` knob's fallback slot
for the standalone estimators (SunAbraham aggregates, WooldridgeDiD OLS,
- StackedDiD, ImputationDiD pretrends leads, LPDiD). Survey/replicate df
+ StackedDiD, ImputationDiD pretrends leads, LPDiD, SpilloverDiD (4.0,
+ [M-154])). Survey/replicate df
and per-coefficient Bell-McCaffrey DOF keep precedence at the CALL SITE
under every value — callers consult this helper only when no
higher-precedence df applies. ``LinearRegression.get_inference`` and
diff --git a/docs/api/spillover.rst b/docs/api/spillover.rst
index 60f6eeb6c..6ea09ff67 100644
--- a/docs/api/spillover.rst
+++ b/docs/api/spillover.rst
@@ -57,6 +57,11 @@ SpilloverDiDResults
Results container with per-ring spillover-effect table.
+Inference df: ``df_convention`` echoes the estimator's knob and ``inference_df``
+the df the reported t / p / CI used (``G - 1`` on the clustered lanes under the
+4.0 default, the residual df under ``"residual"`` and on ``cluster=False`` /
+Conley fits, ``None`` under ``"normal"``; the survey design df takes precedence).
+
.. autoclass:: diff_diff.SpilloverDiDResults
:no-index:
:members:
@@ -493,9 +498,10 @@ and planned follow-up enhancements:
- **Data-driven d_bar selection** — Butts (2021b) / Butts (2023) JUE
Insight propose cross-validation under stronger parallel-trends
assumptions. Not in this PR.
-- **HC2 / HC2_BM (Bell-McCaffrey / CR2) variance** — current stage-2
- inference uses a generic residual-df (n - effective rank) for
- t-distribution lookups. ``vcov_type="hc2"`` / ``"hc2_bm"`` require
+- **HC2 / HC2_BM (Bell-McCaffrey / CR2) variance** — stage-2 inference
+ uses one fallback df per fit (the ``df_convention``-resolved df: t(G-1)
+ on clustered lanes by default, the residual df otherwise) rather than
+ per-coefficient DOF. ``vcov_type="hc2"`` / ``"hc2_bm"`` require
per-coefficient BM / CR2 DOF and raise ``NotImplementedError``.
Routing stage 2 through ``LinearRegression`` (which supplies the
per-coefficient DOF metadata) is queued.
diff --git a/docs/doc-deps.yaml b/docs/doc-deps.yaml
index 6c3d4570d..62f8b745d 100644
--- a/docs/doc-deps.yaml
+++ b/docs/doc-deps.yaml
@@ -414,6 +414,11 @@ sources:
- path: diff_diff/guides/llms.txt
section: "Estimators"
type: user_guide
+ - path: docs/methodology/variance-conventions.md
+ type: methodology
+ note: "Clustered-variance conventions inventory (CR1 k + tail-df map, D/L classification); table generated by tests/test_variance_conventions.py"
+ - path: docs/tutorials/23_spillover_tva.ipynb
+ type: tutorial
# ── EfficientDiD (efficient_did group) ──────────────────────────────
diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md
index 5d74aea0e..af81c42f8 100644
--- a/docs/methodology/REGISTRY.md
+++ b/docs/methodology/REGISTRY.md
@@ -644,8 +644,12 @@ This matches the behavior of R's `fixest::feols()` with absorbed FE.
(2026-07, widened 3.9 / M-127):** `df_convention ∈ {"residual", "cluster", "normal"}` on the
library-wide knob roster — `DifferenceInDifferences` / `TwoWayFixedEffects` /
`LinearRegression` (the TWFE event-study mode included) plus (since 3.9) `SunAbraham`, `WooldridgeDiD` (OLS
- arms), `StackedDiD`, `ImputationDiD` (pretrends lead regression), and `LPDiD` (default
- `"cluster"`, its Stata-anchored convention). `"cluster"` switches clustered analytical
+ arms), `StackedDiD`, `ImputationDiD` (pretrends lead regression), `LPDiD` (default
+ `"cluster"`, its Stata-anchored convention) and, since the 4.0 cut (item 7c, [M-154]),
+ `SpilloverDiD` - its Wave D GMM sandwich lanes make no shared-CR1 call but resolve the same
+ fallback df through `utils.resolve_tail_df`: `G − 1` on the unit auto-cluster / an explicit
+ `cluster=`, inert on Conley (an explicit product-kernel cluster included) and on
+ `cluster=False`. `"cluster"` switches clustered analytical
t/p/CI to the Stata/fixest `G − 1` convention and is inert on unclustered and conley fits;
`"normal"` is deliberate normal-theory z at the fallback level on EVERY fit (clustered,
unclustered, and conley alike — on lanes that were already t it is a new deliberate option,
@@ -656,8 +660,7 @@ This matches the behavior of R's `fixest::feols()` with absorbed FE.
`docs/v4-deprecations.yaml` M-004..M-006 + M-128..M-131, pinned by
`tests/test_v4_inference_defaults.py`; the flip moved every clustered p-value/CI, which is
why it waited for a major version), and `"residual"` restores the 3.x residual-df inference
- bit-for-bit. **Out of the knob's scope** (own inference stacks): `SpilloverDiD`
- (until [M-154], item 7c - its default CR1 lane runs residual-df t at 7b), the IF-based
+ bit-for-bit. **Out of the knob's scope** (own inference stacks): the IF-based
family — `CallawaySantAnna`, `TwoStageDiD`, `ChaisemartinDHaultfoeuille`, `EfficientDiD`,
`ContinuousDiD`, `ChangesInChanges`, `SyntheticDiD`, `HeterogeneousAdoptionDiD` — and the
OLS-based `TripleDifference` / `TROP` (own `safe_inference` df conventions, outside the
@@ -7074,12 +7077,27 @@ should be a deliberate user choice.
shared `str | None | False` contract, [M-155]). Results report `cluster_name="unit"`
/ `n_clusters=G` on the default lane. Roster in
`docs/methodology/variance-conventions.md` "Default clustering roster".
-- **Note:** inference df unchanged at 7b - `t(n_obs_eff - k_effective)` on every
- non-survey lane INCLUDING the new default CR1 lane (the survey-design df keeps its
- precedence under `survey_design=`, as before); SpilloverDiD has no `df_convention`
- knob yet, so it is the one auto-clustered default not on t(G-1); [M-154] / item 7c
- adds the knob with the 4.0 default `"cluster"` (t(G-1) on the clustered lanes, inert
- on Conley, survey df still first).
+- **Note (inference df, [M-154], item 7c, 2026-10-29):** ONE df feeds every
+ `safe_inference` call (aggregate `tau_total`, per-ring `delta_j`, event-study per-cell
+ `tau_k` / `delta_jk` and the scalar lincom `att`), resolved by the pure helper
+ `_resolve_inference_df` (`diff_diff/spillover.py`) with the precedence ladder: (1) a
+ `survey_design` df (`ResolvedSurveyDesign.df_survey`) wins under every knob value; (2) a
+ saturated stage-2 sample (`n_obs_eff - k_effective <= 0`) warns and forces the `df = 0`
+ all-NaN sentinel under every value (never normal theory); (3) otherwise the shared
+ three-value `df_convention` knob via `utils.resolve_tail_df` - `"cluster"` (the 4.0
+ default) = `t(G − 1)` with `G` the effective cluster count of the Wave D `"cluster"`
+ variance mode (the unit auto-cluster or an explicit `cluster=`; reported as
+ `n_clusters`), `G` forced to `None` (residual df) on EVERY Conley fit - an explicit
+ product-kernel `cluster=` included, the SunAbraham precedent - and on the `cluster=False`
+ HC1 lane; `"residual"` = `t(n_obs_eff - k_effective)` (the 3.x and 7b inference);
+ `"normal"` = z. Point estimates, SEs and t-statistics never move. Results carry
+ `df_convention` (the configuration) and `inference_df` (the df actually used; `None`
+ under z and the NaN sentinel). Pre-existing provenance gap, unchanged here: the
+ event-study container's `results_base._resolve_scalar_df_survey` reads a `df_inference`
+ attribute while `DiDResults` carriers expose `inference_df`, so a non-survey Spillover
+ event-study container reports `df_survey=None` (tracked in `TODO.md`, the df-provenance
+ completion row). Pinned by `tests/test_v4_spillover_df_convention.py` and the
+ variance-conventions rows `spillover_cluster_unit` / `spillover_cluster_unit_default_df`.
- **Note (rank-guarded Wave D bread):** The Wave D variance bread `A_22 = (X_2'WX_2)^{-1}` is inverted by the shared `_rank_guarded_inv` (`diff_diff/linalg.py`) on the already-column-dropped kept submatrix. `np.linalg.solve` raised only on an exactly-singular bread (prior fallback: dense `lstsq`); a **near**-singular kept Gram would otherwise flow a garbage inverse (~1e13) into the SE. The rank-guard truncates redundant directions → finite SE on the identified subspace (NaN at rank 0), re-inflated to (k, k) with NaN at the dropped positions, and warns. See the CallawaySantAnna "rank-guarded IF standard errors" Note. Sibling of axis-A finding #17.
**Identification spec (committed):**
@@ -7187,7 +7205,7 @@ The correction applies unconditionally (no opt-out kwarg). Point estimates (`tau
The `_compute_stratified_meat_from_psu_scores` helper at `diff_diff/survey.py` implements the Binder TSL formula and handles `lonely_psu ∈ {"remove", "certainty", "adjust"}` and FPC consistently across the package.
-Degrees of freedom for the t-distribution lookup use `ResolvedSurveyDesign.df_survey` (the standard survey 4-way branch: PSU+strata → `n_PSU - n_strata`; PSU only → `n_PSU - 1`; strata only → `n_obs - n_strata`; neither → `n_obs - 1`). Threaded through all four `safe_inference` call sites: aggregate `tau_total`, per-ring `delta_j`, event-study per-event-time `tau_k` / `delta_jk`, and the scalar `att` lincom in event-study mode.
+Degrees of freedom for the t-distribution lookup use `ResolvedSurveyDesign.df_survey` (precedence over the `df_convention` knob under every value, [M-154]; the standard survey 4-way branch: PSU+strata → `n_PSU - n_strata`; PSU only → `n_PSU - 1`; strata only → `n_obs - n_strata`; neither → `n_obs - 1`). Threaded through all four `safe_inference` call sites: aggregate `tau_total`, per-ring `delta_j`, event-study per-event-time `tau_k` / `delta_jk`, and the scalar `att` lincom in event-study mode.
- **Note (documented synthesis):** Wave E.1 composes Gerber (2026, arXiv:2605.04124) Proposition 1 — Binder Taylor Series Linearization for IF representations of smooth functionals; explicitly derived for TwoStageDiD in the paper's Appendix — with the Wave D Gardner GMM first-stage uncertainty correction (Butts 2021 §3.1 + Gardner 2022 §4) applied to SpilloverDiD's ring-indicator stage-2 design. The composition is mechanical: SpilloverDiD's Wave D Psi is aggregated to PSU level and passed to the audited Binder TSL meat helper. Survey weights enter via Hájek normalization at the gamma_hat solve, eps construction, and bread inversion. No reference software combines all ingredients; Wave E.2 extends with the Conley × survey product-kernel composition (cross-sectional `conley_lag_cutoff = 0`) — see "Variance (Wave E.2)" subsection below; Wave E.2 follow-up further extends with the within-PSU serial Bartlett HAC for `conley_lag_cutoff > 0` — see "Variance (Wave E.2 follow-up)" subsection below.
- **Note (warn-and-use-PSU for cluster + survey):** when both `cluster=` and `survey_design.psu` are supplied with **different groupings**, the cluster argument emits a `UserWarning` and is overridden by PSU (mirrors `TwoStageDiD._resolve_effective_cluster`). PSU is the design-relevant cluster on survey panels; `cluster=` on SpilloverDiD is more often a spatial / unit-level label, so the design constraint wins. When both knobs are supplied with the **same** groupings, no warning fires and PSU still takes precedence (the inference is unchanged either way).
@@ -7273,7 +7291,7 @@ Degrees of freedom for the t-distribution lookup use `ResolvedSurveyDesign.df_su
- **Restrictions / unchanged from Wave E.1/E.2/follow-up:**
- Replicate-weight variance (BRR / Fay / JK1 / JKn / SDR) raises `NotImplementedError` (Wave E.1 gate at `spillover.py:2400` inherited; per-replicate refit + subpopulation is a separate follow-up).
- TwoStageDiD's analogous always-treated design-subset pattern was shipped in a follow-up PR (TwoStageDiD Wave E.3 parity) — see "Note (documented synthesis — Wave E.3 parity, full-domain survey design under always-treated drop)" in the TwoStageDiD section above. The two estimators now share the full-design retention contract; the trigger differs (TwoStageDiD: always-treated unit detection; SpilloverDiD: `finite_mask` warn-and-drop / subpopulation zero-weight rows) but the invariant is identical (zero-pad scores at the meat-helper boundary; retain full-domain `resolved_survey.psu / strata / fpc`).
- - `df_for_inference` at `spillover.py:3229-3234` reads `resolved_survey_fit.df_survey` which is the full-domain value under Wave E.3 (since `resolved_survey_fit = resolved_survey`). All four `safe_inference` call sites (aggregate `tau_total`, per-ring `delta_j`, event-study per-event-time `tau_k` / `delta_jk`, scalar `att` lincom) inherit the full-domain df.
+ - the shared df ladder `_resolve_inference_df` (`diff_diff/spillover.py`) reads `resolved_survey_fit.df_survey` first, which is the full-domain value under Wave E.3 (since `resolved_survey_fit = resolved_survey`). All four `safe_inference` call sites (aggregate `tau_total`, per-ring `delta_j`, event-study per-event-time `tau_k` / `delta_jk`, scalar `att` lincom) inherit the full-domain df.
**Implementation:** `spillover.py:2845-2896` design-subset block deleted; `survey_weights_fit = survey_weights[finite_mask]` retained for the stage-2 OLS solve (which still operates on the fit sample); `cluster_ids_full[finite_mask]` subset dropped on the survey path (`cluster_ids_fit = cluster_ids_full` under survey). The `_compute_gmm_corrected_meat` call at `spillover.py:3163` is rewired to receive SURVEY-FINITE-MASK gamma_hat-construction inputs (`X_1_sparse_fit`, `X_10_sparse_fit`, `eps_10_fit` built on `survey_finite_mask`; `X_2_kept_gamma`, `eps_2_fit_gamma`, `survey_weights_fit_gamma` projected from the fit-sample frame down to survey_finite_mask) plus FULL-LENGTH kernel-dispatch arrays (`cluster_ids_for_meat`, `conley_coords_for_meat`, `conley_time_for_meat`, `conley_unit_for_meat`, `resolved_survey_fit`) plus the new `score_pad_mask=survey_finite_mask` kwarg; no-survey path passes `score_pad_mask=None` and uses fit-length variables throughout (bit-identical to pre-E.3). `_compute_gmm_corrected_meat` at `two_stage.py` adds one new optional kwarg `score_pad_mask: Optional[np.ndarray] = None` and one post-Psi-construction zero-pad block; the `_validate_conley_kwargs` call uses `n_for_conley = len(score_pad_mask)` when the kwarg is set. Within-unit-constancy validator at `spillover.py:2913` updated to operate on full-length unit array (matches full-length cluster array under Wave E.3). Second `compute_survey_metadata` recompute at `spillover.py:2954-2959` now uses full-length `raw_w` (no `[finite_mask]` subset). Tests: `TestSpilloverDiDWaveE3SubpopulationFullDesign` + `TestSpilloverDiDWaveE3SubpopulationFullDesignEventStudy` at `tests/test_spillover.py`.
@@ -7295,7 +7313,7 @@ Degrees of freedom for the t-distribution lookup use `ResolvedSurveyDesign.df_su
- `survey_design=` for `vcov_type ∈ {"hc1"}` (plus `cluster=` for CR1) SHIPPED in Wave E.1 — see "Variance (Wave E.1)" subsection below. Threads Hájek-normalized survey weights through stage-1 FE estimation, gamma_hat solve, eps construction, and bread inversion; aggregates the Wave D Psi to PSU totals and routes through the audited `_compute_stratified_meat_from_psu_scores` Binder TSL meat helper. `vcov_type="conley"` combined with `survey_design=` SHIPPED in Wave E.2 for cross-sectional Conley (`conley_lag_cutoff = 0`) — see "Variance (Wave E.2)" subsection below (stratified-Conley sandwich on PSU totals). Wave E.2 follow-up adds the panel-block composition (`conley_lag_cutoff > 0`) via spatial + serial Bartlett HAC — see "Variance (Wave E.2 follow-up)" subsection below. `SurveyDesign.subpopulation()` and warn-and-drop full-design retention via zero-pad scores SHIPPED in Wave E.3 — see "Variance (Wave E.3)" subsection below (matches R `survey::svyrecvar(subset())` + in-library precedent at `imputation.py:2175-2183` and `prep.py:1401-1432`). Replicate-weight variance (BRR / Fay / JK1 / JKn / SDR) raises `NotImplementedError` — Gerber (2026) Appendix A notes the IF-reweighting shortcut does NOT apply to TwoStageDiD-class estimators because `gamma_hat` is weight-sensitive; correct support requires per-replicate full re-fit and is queued as a follow-up.
- `covariates=` raises `NotImplementedError` — Gardner-style stage-1 residualization not yet wired through; planned follow-up.
- `ring_method="count"` not exposed — only the nearest-treated-ring specification.
-- `vcov_type` ∈ {`"hc2"`, `"hc2_bm"`, `"hc3"`, `"classical"`} raises `NotImplementedError` — `hc2`/`hc2_bm` (at construction) because current stage-2 inference uses generic residual df rather than per-coefficient Bell-McCaffrey / CR2 DOF; `hc3` (at construction) because it is not implemented for the two-stage spillover variance; `classical` (at fit) because the Wave D Gardner GMM first-stage correction has not been derived for the classical homoskedastic variance (different meat structure `sigma_hat^2 * (X_10' X_10)` vs the Wave D IF outer product `Psi' Psi`). Use `"hc1"` or `"conley"`, or pair with `cluster=` for CR1 — all three apply the Wave D GMM correction.
+- `vcov_type` ∈ {`"hc2"`, `"hc2_bm"`, `"hc3"`, `"classical"`} raises `NotImplementedError` — `hc2`/`hc2_bm` (at construction) because stage-2 inference uses one fallback df per fit (the `df_convention`-resolved df, [M-154]) rather than per-coefficient Bell-McCaffrey / CR2 DOF; `hc3` (at construction) because it is not implemented for the two-stage spillover variance; `classical` (at fit) because the Wave D Gardner GMM first-stage correction has not been derived for the classical homoskedastic variance (different meat structure `sigma_hat^2 * (X_10' X_10)` vs the Wave D IF outer product `Psi' Psi`). Use `"hc1"` or `"conley"`, or pair with `cluster=` for CR1 — all three apply the Wave D GMM correction.
- **`rings[0]` must equal 0** — the partition must cover treated locations (`d_it = 0` belongs to Ring 1). Rings starting at a nonzero inner edge would leave units in `0 <= d_it < rings[0]` as exposed-but-unmodeled, silently biasing the estimator. Validator rejects such inputs.
- **Balanced panel required (Wave B MVP)** — every unit must observe every period. An unbalanced (unit, time) Ω₀ bipartite graph can produce disconnected FE components and unidentified stage-1 residuals on treated rows. Exact graph-connectivity-based identification (which would relax this to a strictly weaker condition) is queued as a follow-up extension. Validator rejects unbalanced inputs.
- **One row per `(unit, time)` cell required** — duplicate cells silently re-weight stage-1 FE estimation AND stage-2 OLS. Validator rejects duplicate cells.
diff --git a/docs/methodology/variance-conventions.md b/docs/methodology/variance-conventions.md
index f7692f0d0..c3ec2ddd3 100644
--- a/docs/methodology/variance-conventions.md
+++ b/docs/methodology/variance-conventions.md
@@ -19,6 +19,11 @@ y = N(0,1) + 0.3 unit + 0.2 t + 2 treated; n = 360, G = 60 under
cluster="unit". Each row's exact fit kwargs are in the test file — the
k values and SEs in any claim below come from the SAME fit (mixing panels
produced wrong figures three separate times while this inventory was drafted).
+The two SpilloverDiD rows fit the same panel with a deterministic coordinate
+overlay (`_with_coords`): ever-treated units at lat = 0.001 x unit, lon = 0 (a
+~7 km cluster), never-treated units alternately at lat 0.5 (~55 km, inside the
+100 km ring) and lat 3.0 (~330 km, the far-away Omega_0 controls);
+rings=[0, 100].
| surface | CR1 `k` (multiset) | tail df (multiset) | status | reason |
|---|---|---|---|---|
@@ -41,6 +46,7 @@ produced wrong figures three separate times while this inventory was drafted).
| `dml_did_bad_control` | — | None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None | **legitimate** | L3: Caetano et al. (2026) Eq. 11 augmented-score plug-in variance (same psi_bar - D*theta/pi centering as Chang Thm 2; per-unit influence function; normal-theory safe_inference on no-design fits; cluster= fits use the CR1 per-cell variance with df=df_survey t-inference) and the per-cell ATT_X diagnostic tuple sharing the same branch and df (one extra safe_inference call per retained cell) |
| `dml_did_rcs` | — | None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None | **legitimate** | L3: Chang (2020) Thm 2 lambda-corrected augmented-score plug-in variance (Case 2; per-observation influence function; normal-theory safe_inference on no-design fits; survey_design=/cluster= fits use the design-based CR1/weighted-IF per-cell variance — replicate designs the IF-reweighting per-cell variance — with df=df_survey t-inference outside the shared CR1 sandwich) |
| `duration_did` | — | None, None | **legitimate** | L3: Deaner & Ku (2026) whole-individual pooled bootstrap (centered absolute-deviation pointwise and simultaneous bands); no CR1 sandwich, normal-theory safe_inference gate only |
+| `spillover_cluster_unit` | — | 358, 358 | **legitimate** | L3: Wave D Gardner-GMM two-stage sandwich with its own CR1 multiplier G/(G-1)*(n-1)/(n-p_2) (two_stage meat, never the shared CR1). Explicit df_convention='residual' since the 4.0 default is G-1 (see the _default_df twin). |
| `did_absorb_hc1_cluster_unit_default_df` | 7 | 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 |
| `twfe_hc1_cluster_unit_default_df` | 3 | 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 |
| `wooldridge_hc1_within_default_df` | 15 | 59, 59, 59, 59, 59, 59, 59, 59, 59, 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 |
@@ -49,6 +55,7 @@ produced wrong figures three separate times while this inventory was drafted).
| `twfe_es_within_hc1_cluster_unit_default_df` | 11 | 59, 59, 59, 59, 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 |
| `twfe_es_pooled_hc1_cluster_time_default_df` | 7 | 5, 5, 5, 5, 5 | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 |
| `imputation_pretrends_event_study_default_df` | unpinned | 59, 59, 59, None, None, None, None, None | **legitimate** | t(G-1): the 4.0 df_convention default (M-004..M-006, M-128..M-131) - same CR1 k as the explicit-'residual' twin, tail df G-1 |
+| `spillover_cluster_unit_default_df` | — | 59, 59 | **legitimate** | t(G-1): the 4.0 df_convention default (M-154 - SpilloverDiD gained the knob at the cut) - no shared-CR1 call, as the explicit-'residual' twin; tail df G-1 |
cr1_k is the sorted multiset of K_reference counts reaching the shared
clustered CR1 denominator — visible columns + the signed cluster_k_adjustment
@@ -139,11 +146,15 @@ output).
library-wide 4.0 flip — bit-identical, no numbers moved there), with
"residual"/"normal" as opt-ins; the degenerate lanes (unclustered refit,
RA G<=1, saturated early return) keep literal df=None under every value.
-- **L3 — CallawaySantAnna / DMLDiD / TwoStageDiD / ImputationDiD (default) /
- DurationDiD**: different variance theory (influence functions / Chang (2020)
- augmented-score plug-in / two-stage / BJS imputation / Deaner & Ku (2026)
- whole-individual pooled bootstrap with centered absolute-deviation bands),
- never the shared CR1 sandwich. DurationDiD is bootstrap-only: its single
+- **L3 — CallawaySantAnna / DMLDiD / TwoStageDiD / SpilloverDiD / ImputationDiD
+ (default) / DurationDiD**: different variance theory (influence functions / Chang
+ (2020) augmented-score plug-in / two-stage / Gardner two-stage + Wave D GMM
+ first-stage correction / BJS imputation / Deaner & Ku (2026) whole-individual
+ pooled bootstrap with centered absolute-deviation bands), never the shared CR1
+ sandwich. SpilloverDiD's CR1 lane applies its own `G/(G-1) * (n-1)/(n-p_2)`
+ multiplier inside `two_stage._compute_gmm_corrected_meat`, outside the shared
+ sandwich, and since 4.0 ([M-154]) resolves its tail df through the shared
+ `df_convention` knob (its two matrix rows). DurationDiD is bootstrap-only: its single
`safe_inference` + `safe_inference_batch` gate passes `df=None` and the
centered-bootstrap p-values/intervals override the normal-theory values. CS is
anchored to Stata csdid outright; DMLDiD's panel lane to DoubleML at
@@ -172,7 +183,12 @@ ONE three-value knob (`df_convention ∈ {"residual", "cluster", "normal"}`)
now governs the analytical fallback df on every shared-CR1 surface:
DiD/TWFE/LinearRegression + SunAbraham/WooldridgeDiD-OLS/StackedDiD/
ImputationDiD-pretrends and LPDiD, all at the default "cluster" (the
-Stata-anchored t(G−1); LPDiD since 3.9, the rest since the 4.0 cut) with
+Stata-anchored t(G−1); LPDiD since 3.9, the rest since the 4.0 cut) - plus,
+since the 4.0 cut (item 7c, [M-154]), SpilloverDiD, whose Wave D GMM sandwich
+lanes make no shared-CR1 call (`cr1_k=()`) but resolve the same fallback df
+through `resolve_tail_df`: G−1 on the unit auto-cluster / an explicit cluster=,
+inert on Conley (an explicit product-kernel cluster included) and on
+cluster=False - with
"residual" restoring the 3.x residual df. Survey/replicate df and hc2_bm Bell-McCaffrey
DOF keep precedence under every value; "cluster" fails closed warn+NaN at
G<=1 and is inert on unclustered/conley fits; "normal" is deliberate z at
@@ -211,7 +227,7 @@ column lookup.
| Class | `cluster` param? | Default when omitted (v3.13.0 -> 4.0) | `cluster=False` since 4.0 | Category |
|---|---|---|---|---|
-| SpilloverDiD | yes | observation-level HC1 -> **CR1 at unit since 4.0** (`"unit"`, G; TWFE's carve-outs mirrored; inference df still the residual df until [M-154]) | plain HC1 SE (= the 3.x default) | **a** - default flipped to unit |
+| SpilloverDiD | yes | observation-level HC1 -> **CR1 at unit since 4.0** (`"unit"`, G; TWFE's carve-outs mirrored; inference df t(G-1) since 4.0, [M-154]) | plain HC1 SE (= the 3.x default) | **a** - default flipped to unit |
| TwoWayFixedEffects (static) | yes | CR1 at unit (`cluster_name="unit"`; reports `n_clusters` since 4.0 on non-survey analytical fits; the survey-lane auto label is None since 4.0) | plain HC sandwich (absorbed-FE full-K rescale, residual-df t; one-way CR2-BM under `hc2_bm`); rejected under `inference="wild_bootstrap"` | c1 |
| TwoWayFixedEffects (event-study) | yes | CR1 at unit since 3.9 (M-010) | plain HC sandwich (the core's unclustered lanes) | c1 |
| SunAbraham | yes | CR1 at unit (reports `cluster_name="unit"` and the appended `n_clusters` since 4.0) | plain HC sandwich (residual-df t); no unit PSU under `survey_design=`; pairs bootstrap unaffected; rejected with `hc2_bm` | c1 |
@@ -249,5 +265,5 @@ than open an unverified variance: `cluster=False` + `vcov_type="hc2_bm"` on
SunAbraham / WooldridgeDiD (their Bell-McCaffrey DOF helper is cluster-only; the
one-way port is a `TODO.md` row - TWFE's one-way CR2-BM lane already exists), and
the bootstraps that draw AT the analytical cluster (DiD/TWFE wild, WooldridgeDiD
-multiplier). SpilloverDiD's new default lane keeps residual-df inference until
-[M-154] (item 7c) adds its `df_convention` knob.
+multiplier). SpilloverDiD's default lane moved from residual-df to t(G-1)
+inference when [M-154] (item 7c, 2026-10-29) added its `df_convention` knob.
diff --git a/docs/migration-4.0.md b/docs/migration-4.0.md
index c35f4f160..1d1145912 100644
--- a/docs/migration-4.0.md
+++ b/docs/migration-4.0.md
@@ -307,6 +307,8 @@ Two changes alter results without changing any call:
leads); `LPDiD` was already there. Clustered analytical p-values and confidence intervals
now use the Stata/fixest `t(G − 1)` reference instead of the fitted residual df
`t(n − K)`; point estimates, standard errors and t-statistics never move.
+ (`SpilloverDiD` gained the knob at 4.0 with the same `"cluster"` default, [M-154] - no
+ flip, a new surface; see the bullet below.)
```python
TwoWayFixedEffects().fit(df, outcome="y", treatment="d", post="post", unit="unit")
@@ -381,9 +383,14 @@ Two changes alter results without changing any call:
auto-cluster is never injected as a PSU, so the former `"unit"` label described ids
the regression never received); summaries render ", G=..." and `SunAbraham` summaries
gain a "Std. errors:" line on analytical non-survey fits.
-- **`SpilloverDiD` p-values and CIs on the new default lane stay on the residual df** in
- this release step; the next PR adds its `df_convention` knob with the 4.0 `"cluster"`
- default ([M-154]), which moves them to t(G-1) like every other auto-clustered default.
+- **`SpilloverDiD` p-values and CIs on its clustered lanes move to t(G − 1)** ([M-154]):
+ `SpilloverDiD` gained the three-value `df_convention` knob with the 4.0 `"cluster"`
+ default, so the unit auto-cluster and an explicit `cluster=` now use the same reference
+ distribution as every other auto-clustered default. `SpilloverDiD(df_convention="residual")`
+ restores the 3.x residual-df inference bit-for-bit; `"cluster"` falls back to the residual df
+ on `cluster=False` and Conley fits (`"normal"` still selects z there); the survey design df
+ keeps precedence under every value. Results
+ report `df_convention` and `inference_df`.
## Removed functions and aliases
@@ -401,8 +408,8 @@ Bacon roster re-homing, and the family-wide `anticipation` validation below. The
the ledger-derived removals/flips; [M-144], [M-145], and [M-146] are behavior tightenings,
[M-147] is an additive policy, and [M-153] is a results-frame reshape. These have no
removal/deprecation fields and appear here only. [M-155] (the shared `cluster`
-argument contract, see "Inference defaults that move numbers") is the same kind of
-behavior tightening.
+argument contract) and [M-154] (the `SpilloverDiD` `df_convention` knob), both in
+"Inference defaults that move numbers", are the same kind of behavior tightening.
- `WooldridgeDiD(unsupported_period_action="error")` ([M-147]) refuses periods lacking
comparison support before removing them. The default `"drop"` preserves filtering and
@@ -602,7 +609,7 @@ does not mean no action is required, so read the `Fix` cell.
| M-002 | obligation-sdid-params | `diff_diff:SyntheticDiD[zeta]` | — | Drop it. It has been IGNORED since 3.0.0, so 3.x already auto-computes regularization - do NOT copy its value into `zeta_lambda=`, which activates an override that was inert and changes weights, ATT and inference. Set `zeta_lambda=` only as a deliberate new choice. |
| M-003 | obligation-sdid-params | `diff_diff:SyntheticDiDResults.placebo_effects` | `diff_diff:SyntheticDiDResults.variance_effects` | Read `variance_effects` instead of `placebo_effects`. |
| M-007 | obligation-warning-retirements | `diff_diff.estimators` | — | The MultiPeriodDiD `e=-1` transition FutureWarning is no longer emitted (the class is gone; the event-study mode defaults to the last pre-period silently); drop any warning filter that suppressed it. |
-| M-080 | policy-auto-cluster | `diff_diff:SpilloverDiD[cluster]` | — | `SpilloverDiD` now clusters at `unit` when `cluster=` is omitted (TWFE's carve-outs mirrored: explicit `cluster=` wins, dropped on the Conley path, never injected as a survey PSU; its p-values/CIs keep the residual df until [M-154]); pass `cluster=False` for the 3.x observation-level HC standard errors (`None`/omission means auto-cluster, not off). Every other panel estimator already clustered at unit or is unit-level by construction, so its numbers do not move. `cluster=False` is also newly accepted on `TwoWayFixedEffects`/`SunAbraham`/`WooldridgeDiD`/`LPDiD` (plain HC sandwich - new behaviour; residual-df t), as a no-op on the by-construction estimators, on `ImputationDiD`/`TwoStageDiD` and on the 2x2 `DifferenceInDifferences` / 2x2x2 `TripleDifference` surfaces; `StackedDiD` still rejects it. Rejected with `inference="wild_bootstrap"` (DiD/TWFE), `WooldridgeDiD(n_bootstrap > 0)` and `vcov_type="hc2_bm"` on `SunAbraham`/`WooldridgeDiD`. `cluster` accepts `str | None | False` only ([M-155]). |
+| M-080 | policy-auto-cluster | `diff_diff:SpilloverDiD[cluster]` | — | `SpilloverDiD` now clusters at `unit` when `cluster=` is omitted (TWFE's carve-outs mirrored: explicit `cluster=` wins, dropped on the Conley path, never injected as a survey PSU; its p-values/CIs use t(G − 1) since [M-154], item 7c); pass `cluster=False` for the 3.x observation-level HC standard errors (`None`/omission means auto-cluster, not off). Every other panel estimator already clustered at unit or is unit-level by construction, so its numbers do not move. `cluster=False` is also newly accepted on `TwoWayFixedEffects`/`SunAbraham`/`WooldridgeDiD`/`LPDiD` (plain HC sandwich - new behaviour; residual-df t), as a no-op on the by-construction estimators, on `ImputationDiD`/`TwoStageDiD` and on the 2x2 `DifferenceInDifferences` / 2x2x2 `TripleDifference` surfaces; `StackedDiD` still rejects it. Rejected with `inference="wild_bootstrap"` (DiD/TWFE), `WooldridgeDiD(n_bootstrap > 0)` and `vcov_type="hc2_bm"` on `SunAbraham`/`WooldridgeDiD`. `cluster` accepts `str | None | False` only ([M-155]). |
| M-032 | renames-cohort | `diff_diff:WooldridgeDiD.fit[cohort]` | `diff_diff:WooldridgeDiD.fit[first_treat]` | Rename the keyword: `cohort=` becomes `first_treat=`. |
| M-035 | renames-col-suffix | `diff_diff:HeterogeneousAdoptionDiD.fit[outcome_col]` | `diff_diff:HeterogeneousAdoptionDiD.fit[outcome]` | Rename the keyword: `outcome_col=` becomes `outcome=`. |
| M-036 | renames-col-suffix | `diff_diff:HeterogeneousAdoptionDiD.fit[dose_col]` | `diff_diff:HeterogeneousAdoptionDiD.fit[dose]` | Rename the keyword: `dose_col=` becomes `dose=`. |
diff --git a/docs/tutorials/23_spillover_tva.ipynb b/docs/tutorials/23_spillover_tva.ipynb
index 59755a2a9..4d641a92c 100644
--- a/docs/tutorials/23_spillover_tva.ipynb
+++ b/docs/tutorials/23_spillover_tva.ipynb
@@ -383,7 +383,10 @@
"spillover-aware estimator removes the bias the naive TWFE suffered.\n",
"The standard errors in the table are the 4.0 default: cluster-robust\n",
"(CR1) at the unit level, like `TwoWayFixedEffects` - `SpilloverDiD`\n",
- "auto-clusters at `unit` when `cluster=` is omitted.\n",
+ "auto-clusters at `unit` when `cluster=` is omitted. Its p-values and\n",
+ "confidence intervals use the t(G-1) reference distribution\n",
+ "(`df_convention=\"cluster\"`, the 4.0 default, with G = 200 units);\n",
+ "`df_convention=\"residual\"` restores the residual-df inference.\n",
"\n",
"The interpretation is straightforward: every period in the post window,\n",
"treated units lose 7.4 units of $y$ on average from the policy, AND\n",
diff --git a/docs/v4-deprecations.yaml b/docs/v4-deprecations.yaml
index dc364e3b2..8da38d381 100644
--- a/docs/v4-deprecations.yaml
+++ b/docs/v4-deprecations.yaml
@@ -1058,7 +1058,7 @@ rows:
phase: 5
test_ref: tests/test_v4_cluster_policy.py
code_refs: [diff_diff/spillover.py, diff_diff/twfe.py, diff_diff/estimators.py, diff_diff/sun_abraham.py, diff_diff/lpdid.py, diff_diff/stacked_did.py, diff_diff/wooldridge.py, diff_diff/imputation.py, diff_diff/two_stage.py, diff_diff/staggered.py, diff_diff/efficient_did.py, diff_diff/dml_did.py, diff_diff/lwdid.py, diff_diff/had.py, diff_diff/triple_diff.py, diff_diff/_staggered_triple_diff_engine.py, diff_diff/chaisemartin_dhaultfoeuille.py, docs/methodology/variance-conventions.md]
- notes: "DONE at the 4.0 cut (item 7b, 2026-10-29): SpilloverDiD auto-clusters at unit when cluster= is omitted (TWFE carve-outs mirrored: explicit cluster= wins, dropped on the Conley path, never injected as a survey PSU; inference df unchanged - residual df - pending [M-154]); uniform cluster=False via utils.validate_cluster - the shared contract is str | None | False ([M-155]), so non-str column labels are rejected everywhere (DMLDiD existing rule generalized) - plain HC on a/c1 incl. the new LPDiD / WooldridgeDiD-GLM unclustered lanes, the Wooldridge within-transform absorbed-FE full-K rescale and TWFE one-way hc2_bm; accepted no-op on b/c2 and the 2x2 surfaces (incl. the CS RC auto-PSU path, EfficientDiD.hausman_pretest and the LWDiD sensitivity helpers); rejected on StackedDiD (own vocabulary); the DiD/TWFE wild and WooldridgeDiD multiplier bootstraps and the cluster-only hc2_bm lanes of SunAbraham / WooldridgeDiD reject it (SunAbraham pairs bootstrap: accepted no-op, the hc2 precedent); metadata: DiD/TWFE analytical n_clusters from the regression (non-survey fits), SunAbrahamResults.n_clusters (appended) + cluster_name on clustered analytical fits, TWFE survey auto-label corrected to None. Pinned by tests/test_v4_cluster_policy.py; notebook 23 re-executed. RE-SCOPED 2026-10-18 (4.0-cut framing, user decision; measured roster in docs/methodology/variance-conventions.md and spec section 7). The original premise - static TwoWayFixedEffects runs HC-robust by default - was already false: static TWFE auto-clusters at unit (twfe.py, since 2026-01). Measured categories: (a) SpilloverDiD is the ONE required-unit estimator whose default SE moves (observation-level HC -> unit clustering, mirroring TWFE's carve-outs: explicit cluster= wins, Conley drops it, never injected as survey PSU); (c1) TWFE/SunAbraham/WooldridgeDiD/LPDiD already cluster at unit by regression-level default - cluster=False yields the plain HC sandwich; (c2) ImputationDiD/TwoStageDiD default cluster_var=unit INSIDE unit-level influence-function machinery - no unclustered variance exists, cluster=False is an accepted explicit no-op; (c3) StackedDiD clusters by construction and its validator accepts only unit/unit_subexp - cluster=False is REJECTED (REGISTRY Note); (b) CS/EfficientDiD/DMLDiD/dCDH/staggered-DDD (per-unit IF) and LWDiD/HAD (regression on a unit-level cross-section) are unit-level by construction - cluster=False is an accepted explicit no-op; (d) ContinuousDiD/DurationDiD/SDiD/SC/TROP/CiC have no cluster param; (n/a) cross-sectional DiD/2x2x2 DDD/RDD stay HC-robust or unclustered. Categories b and d carry NO auto-cluster metadata by decision (cluster_name stays None where the field exists; no field is added where it does not). The 4.0 flip therefore = the SpilloverDiD default + the uniform cluster=False off-switch on a/b/c1/c2 (today accepted nowhere: LPDiD's `self.cluster or unit` treats False as auto-cluster, most others look up a column named False, DMLDiD/StackedDiD reject on type, dCDH raises on any non-None) + metadata gaps (TWFE/DiD n_clusters on the analytical path; SunAbraham cluster_name + n_clusters). ORIGINAL NOTE: ONE auto-cluster policy at 4.0: panel estimators (required unit column) default to clustering at unit with cluster_name/n_clusters metadata; cluster=False disables; 2x2 cross-sectional estimators stay HC-robust. Changes some default SEs - migration guide with [M-004..M-006]. SCOPE NOTE (Phase 3(a), user decision 2026-08-07): the TWFE EVENT-STUDY mode shipped already auto-clustered at 3.9 (new API adopts the end-state immediately, with the static carve-outs mirrored - see [M-010]), so this 4.0 flip does not re-touch it."
+ notes: "DONE at the 4.0 cut (item 7b, 2026-10-29): SpilloverDiD auto-clusters at unit when cluster= is omitted (TWFE carve-outs mirrored: explicit cluster= wins, dropped on the Conley path, never injected as a survey PSU; inference df moved to t(G-1) by [M-154] in 7c); uniform cluster=False via utils.validate_cluster - the shared contract is str | None | False ([M-155]), so non-str column labels are rejected everywhere (DMLDiD existing rule generalized) - plain HC on a/c1 incl. the new LPDiD / WooldridgeDiD-GLM unclustered lanes, the Wooldridge within-transform absorbed-FE full-K rescale and TWFE one-way hc2_bm; accepted no-op on b/c2 and the 2x2 surfaces (incl. the CS RC auto-PSU path, EfficientDiD.hausman_pretest and the LWDiD sensitivity helpers); rejected on StackedDiD (own vocabulary); the DiD/TWFE wild and WooldridgeDiD multiplier bootstraps and the cluster-only hc2_bm lanes of SunAbraham / WooldridgeDiD reject it (SunAbraham pairs bootstrap: accepted no-op, the hc2 precedent); metadata: DiD/TWFE analytical n_clusters from the regression (non-survey fits), SunAbrahamResults.n_clusters (appended) + cluster_name on clustered analytical fits, TWFE survey auto-label corrected to None. Pinned by tests/test_v4_cluster_policy.py; notebook 23 re-executed. RE-SCOPED 2026-10-18 (4.0-cut framing, user decision; measured roster in docs/methodology/variance-conventions.md and spec section 7). The original premise - static TwoWayFixedEffects runs HC-robust by default - was already false: static TWFE auto-clusters at unit (twfe.py, since 2026-01). Measured categories: (a) SpilloverDiD is the ONE required-unit estimator whose default SE moves (observation-level HC -> unit clustering, mirroring TWFE's carve-outs: explicit cluster= wins, Conley drops it, never injected as survey PSU); (c1) TWFE/SunAbraham/WooldridgeDiD/LPDiD already cluster at unit by regression-level default - cluster=False yields the plain HC sandwich; (c2) ImputationDiD/TwoStageDiD default cluster_var=unit INSIDE unit-level influence-function machinery - no unclustered variance exists, cluster=False is an accepted explicit no-op; (c3) StackedDiD clusters by construction and its validator accepts only unit/unit_subexp - cluster=False is REJECTED (REGISTRY Note); (b) CS/EfficientDiD/DMLDiD/dCDH/staggered-DDD (per-unit IF) and LWDiD/HAD (regression on a unit-level cross-section) are unit-level by construction - cluster=False is an accepted explicit no-op; (d) ContinuousDiD/DurationDiD/SDiD/SC/TROP/CiC have no cluster param; (n/a) cross-sectional DiD/2x2x2 DDD/RDD stay HC-robust or unclustered. Categories b and d carry NO auto-cluster metadata by decision (cluster_name stays None where the field exists; no field is added where it does not). The 4.0 flip therefore = the SpilloverDiD default + the uniform cluster=False off-switch on a/b/c1/c2 (today accepted nowhere: LPDiD's `self.cluster or unit` treats False as auto-cluster, most others look up a column named False, DMLDiD/StackedDiD reject on type, dCDH raises on any non-None) + metadata gaps (TWFE/DiD n_clusters on the analytical path; SunAbraham cluster_name + n_clusters). ORIGINAL NOTE: ONE auto-cluster policy at 4.0: panel estimators (required unit column) default to clustering at unit with cluster_name/n_clusters metadata; cluster=False disables; 2x2 cross-sectional estimators stay HC-robust. Changes some default SEs - migration guide with [M-004..M-006]. SCOPE NOTE (Phase 3(a), user decision 2026-08-07): the TWFE EVENT-STUDY mode shipped already auto-clustered at 3.9 (new API adopts the end-state immediately, with the static carve-outs mirrored - see [M-010]), so this 4.0 flip does not re-touch it."
- id: M-081
kind: behavior
group: policy-n-bootstrap
@@ -1919,10 +1919,11 @@ rows:
introduced_in: "4.0"
deprecated_in: null
removed_in: null
- status: planned
+ status: done
phase: 5
+ test_ref: tests/test_v4_spillover_df_convention.py
code_refs: [diff_diff/spillover.py, diff_diff/utils.py, docs/methodology/variance-conventions.md]
- notes: "PLANNED (item 7c, the PR after 7b; user decision 2026-10-29): SpilloverDiD gains the three-value df_convention knob with the 4.0 default 'cluster' - t(G-1) on its clustered analytical lanes (the 4.0 unit auto-cluster and explicit cluster=), residual df under 'residual' and on unclustered (cluster=False) fits, z under 'normal'; inert on Conley (G forced to None even with an explicit product-kernel cluster, the SunAbraham precedent); survey df keeps precedence. Closes the gap 7b documents: at 7b the default CR1 lane runs t(n_obs_eff - k_effective) while every other auto-clustered default runs t(G-1). Moves p/CI on clustered SpilloverDiD fits; notebook 23 re-executed again in 7c; the REGISTRY TWFE knob Note moves SpilloverDiD from its out-of-scope list to the knob roster."
+ notes: "DONE at the 4.0 cut (item 7c, 2026-10-29): SpilloverDiD(df_convention='cluster') - t(G-1) on the clustered analytical lanes (the unit auto-cluster and explicit cluster=), residual df under 'residual' and on cluster=False, z under 'normal'; inert (residual df) on every Conley fit incl. an explicit product-kernel cluster=; survey df keeps precedence; the saturated-sample all-NaN contract is knob-independent (ladder: survey df -> saturation -> utils.resolve_tail_df, in spillover._resolve_inference_df); SpilloverDiDResults carries df_convention / inference_df; variance-conventions matrix rows spillover_cluster_unit / spillover_cluster_unit_default_df; notebook 23 re-validated (it prints no p-values or CIs). ORIGINAL: PLANNED (item 7c, the PR after 7b; user decision 2026-10-29): SpilloverDiD gains the three-value df_convention knob with the 4.0 default 'cluster' - t(G-1) on its clustered analytical lanes (the 4.0 unit auto-cluster and explicit cluster=), residual df under 'residual' and on unclustered (cluster=False) fits, z under 'normal'; inert on Conley (G forced to None even with an explicit product-kernel cluster, the SunAbraham precedent); survey df keeps precedence. Closes the gap 7b documents: at 7b the default CR1 lane runs t(n_obs_eff - k_effective) while every other auto-clustered default runs t(G-1). Moves p/CI on clustered SpilloverDiD fits; notebook 23 re-executed again in 7c; the REGISTRY TWFE knob Note moves SpilloverDiD from its out-of-scope list to the knob roster."
- id: M-155
kind: behavior
group: policy-auto-cluster
diff --git a/docs/v4-design.md b/docs/v4-design.md
index 8030ac5f2..5f6edc295 100644
--- a/docs/v4-design.md
+++ b/docs/v4-design.md
@@ -864,18 +864,22 @@ domain vocabulary, not drift.
with `hc2_bm` on SunAbraham/WooldridgeDiD pending the one-way BM DOF port
(TODO.md); DiD/TWFE analytical `n_clusters`, `SunAbrahamResults.n_clusters`
+ `cluster_name`, the TWFE survey auto-label corrected to None; pinned by
- `tests/test_v4_cluster_policy.py`. SpilloverDiD's default CR1 lane keeps
- residual-df inference until [M-154].)
-- **SpilloverDiD df_convention** [M-154] (item 7c, the PR after 7b; user
- decision 2026-10-29): SpilloverDiD gains the three-value `df_convention` knob
+ `tests/test_v4_cluster_policy.py`. SpilloverDiD's default CR1 lane kept
+ residual-df inference until [M-154] shipped in 7c.)
+- **SpilloverDiD df_convention** [M-154] (SHIPPED item 7c, 2026-10-29; pinned by
+ `tests/test_v4_spillover_df_convention.py` and the variance-conventions rows
+ `spillover_cluster_unit` / `spillover_cluster_unit_default_df`; notebook 23
+ re-validated - it prints no p-values or CIs): SpilloverDiD gained the three-value
+ `df_convention` knob
with the 4.0 default `"cluster"` - t(G-1) on its clustered analytical lanes
(the unit auto-cluster and explicit `cluster=`), residual df under
`"residual"` and on unclustered (`cluster=False`) fits, z under `"normal"`;
inert on Conley (G forced to None even with an explicit product-kernel
cluster, the SunAbraham precedent); survey df keeps precedence. Closes the
one gap 7b leaves: at 7b the default CR1 lane runs t(n_obs_eff -
- k_effective) while every other auto-clustered default runs t(G-1). Moves
- p/CI on clustered SpilloverDiD fits; notebook 23 is re-executed again there.
+ k_effective) while every other auto-clustered default ran t(G-1). Moved
+ p/CI on clustered SpilloverDiD fits; the saturated-sample all-NaN contract is
+ knob-independent (ladder: survey df -> saturation -> `utils.resolve_tail_df`).
- **df_convention default flip** [M-004]..[M-006] + [M-128]..[M-131]:
`"residual"` -> `"cluster"` (G-1) at 4.0 - the locked #663 direction.
PR C ([M-127], 3.9) resolved the "evaluate extending the knob" item EARLY:
@@ -1232,8 +1236,9 @@ tests-first PR that flips no rows).
df_convention flips [M-004]..[M-006] + [M-128]..[M-131], pinned by
`tests/test_v4_inference_defaults.py`; 7b SHIPPED 2026-10-29: [M-080] +
the shared cluster contract [M-155], pinned by
- `tests/test_v4_cluster_policy.py`, notebook 23 re-executed; 7c = [M-154]
- SpilloverDiD df_convention, the next PR.)
+ `tests/test_v4_cluster_policy.py`, notebook 23 re-executed; 7c SHIPPED
+ 2026-10-29: [M-154] SpilloverDiD df_convention, pinned by
+ `tests/test_v4_spillover_df_convention.py`; item 7 complete.)
8. [M-093] sentinel retirement + event-study schema enforcement.
9. [M-090] docs reorganization: Bacon to the diagnostics family, catalogue
counts, JOSS prose, REGISTRY archive framing for the removed classes.
@@ -1244,8 +1249,8 @@ tests-first PR that flips no rows).
04_parallel_trends, 05_honest_did, 07_pretrends_power,
09_real_world_examples, 11_imputation_did, 13_stacked_did,
16_wooldridge_etwfe (NOT 16_survey_did: the survey df wins) - plus any
- 7c/8/9 movers; 24 is handled in 7a and 23 in 7b and again in 7c (each
- re-executed there). 7b's label changes also drift every committed tutorial
+ 8/9 movers; 24 is handled in 7a and 23 in 7b and re-validated in 7c (it
+ prints no p-values or CIs, so 7c moved no committed output). 7b's label changes also drift every committed tutorial
that renders an analytical DiD/TWFE summary (", G=" on the variance line)
or a SunAbraham summary (the new "Std. errors:" line, e.g. 02) - all
within the nine-notebook set above.
diff --git a/tests/test_spillover.py b/tests/test_spillover.py
index 35d069305..aa2b4cea5 100644
--- a/tests/test_spillover.py
+++ b/tests/test_spillover.py
@@ -555,6 +555,7 @@ def test_get_params_returns_constructor_args(self):
"conley_cutoff_km",
"conley_lag_cutoff",
"cluster",
+ "df_convention",
"alpha",
"anticipation",
"event_study",
diff --git a/tests/test_t23_spillover_tva_drift.py b/tests/test_t23_spillover_tva_drift.py
index 57d88d7a6..a8041bc4c 100644
--- a/tests/test_t23_spillover_tva_drift.py
+++ b/tests/test_t23_spillover_tva_drift.py
@@ -796,3 +796,29 @@ def test_spillover_conley_fit_warning_policy_post_filter_clean(panel):
_silence_spillover_matmul_warnings() # mirror notebook §6 filter
est.fit(panel, outcome="y", unit="unit", time="time", treatment="D")
_assert_post_filter_warning_surface_is_clean(w)
+
+
+# ----------------------------------------------------------------------------
+# 7c ([M-154]): the df_convention knob on the tutorial's fits
+# ----------------------------------------------------------------------------
+
+
+def test_default_inference_is_t_g_minus_1(spillover_fit, panel):
+ """The default (CR1 at unit) lane runs t(G-1) since 4.0: G = 200 units."""
+ from scipy import stats
+
+ G = panel["unit"].nunique()
+ assert spillover_fit.df_convention == "cluster"
+ assert spillover_fit.inference_df == G - 1 == spillover_fit.n_clusters - 1 == 199
+ np.testing.assert_allclose(
+ spillover_fit.p_value,
+ 2 * stats.t.sf(abs(spillover_fit.t_stat), G - 1),
+ rtol=1e-12,
+ )
+
+
+def test_hc1_and_conley_lanes_keep_residual_df(spillover_hc1_fit, spillover_conley_lag0_fit, panel):
+ """cluster=False and every Conley fit are knob-inert: residual df n - k (k = 2)."""
+ n_obs = len(panel)
+ assert spillover_hc1_fit.inference_df == n_obs - 2
+ assert spillover_conley_lag0_fit.inference_df == n_obs - 2
diff --git a/tests/test_v4_cluster_policy.py b/tests/test_v4_cluster_policy.py
index 5f91a6be1..b3df0f7b3 100644
--- a/tests/test_v4_cluster_policy.py
+++ b/tests/test_v4_cluster_policy.py
@@ -896,5 +896,5 @@ def test_ledger_rows_are_done_and_point_here():
assert row["test_ref"] == THIS_TEST_REF, rid
assert rows["M-080"]["old"] == "diff_diff:SpilloverDiD[cluster]"
assert rows["M-155"]["old"] == "diff_diff.utils:validate_cluster"
- assert rows["M-154"]["status"] == "planned" # 7c: the SpilloverDiD df_convention knob
+ assert rows["M-154"]["status"] == "done" # 7c shipped the SpilloverDiD df_convention knob
assert rows["M-154"]["old"] == "diff_diff:SpilloverDiD[df_convention]"
diff --git a/tests/test_v4_matrix.py b/tests/test_v4_matrix.py
index 155fc5fd4..ecb77d101 100644
--- a/tests/test_v4_matrix.py
+++ b/tests/test_v4_matrix.py
@@ -143,7 +143,7 @@
# to_dataframe level twin (M-150, 4.0-cut framing) = 132; + the LWDiD
# results-contract rows (M-151/M-152 alias deprecations, M-153 to_dataframe
# reshape; 4.0-cut PR 1) = 135; + the 7b cluster-policy pair (M-154 SpilloverDiD
-# df_convention, planned for 7c; M-155 the shared cluster contract) = 137.
+# df_convention, shipped 7c; M-155 the shared cluster contract) = 137.
# Ids are never reused and terminal rows are never deleted, so the ledger
# only grows - raise the floor when rows are added; a lower parse count
# means scanner/format drift or an illegal row deletion.
diff --git a/tests/test_v4_spillover_df_convention.py b/tests/test_v4_spillover_df_convention.py
new file mode 100644
index 000000000..7b8ce1563
--- /dev/null
+++ b/tests/test_v4_spillover_df_convention.py
@@ -0,0 +1,435 @@
+"""Item 7c of the 4.0 cut: the SpilloverDiD ``df_convention`` knob (ledger row M-154).
+
+7b made SpilloverDiD auto-cluster at ``unit`` but left its inference on the OLS
+residual df ``t(n_obs_eff - k_effective)``; 7c adds the shared three-value knob
+with the 4.0 default ``"cluster"`` so the clustered lanes (the unit auto-cluster
+and an explicit ``cluster=``) run the Stata/fixest ``t(G - 1)`` like every other
+auto-clustered default. What moves: p-values and CIs on clustered fits. What does
+not: point estimates, SEs and t-statistics; ``cluster=False`` and EVERY Conley fit
+(G is forced to None, the SunAbraham precedent - an explicit product-kernel
+``cluster=`` included); the survey ``df_survey`` (precedence under every value);
+the saturated-sample all-NaN contract. ``df_convention="residual"`` restores the
+3.x / 7b inference bit-for-bit.
+
+The df ladder is the pure helper ``diff_diff.spillover._resolve_inference_df``
+(survey df -> saturation -> ``utils.resolve_tail_df``); its saturation branch is
+pinned here directly because a saturated SpilloverDiD FIT is unreachable through
+the balanced-panel + far-away validation (no test constructs one).
+
+The ledger import lives inside its test: ``tests.test_v4_matrix`` skips at import
+when ``docs/v4-deprecations.yaml`` is absent (the wheel-install CI legs copy only
+``tests/``), and the behavioral pins must still run there.
+"""
+
+from __future__ import annotations
+
+import inspect
+import warnings
+
+import numpy as np
+import pytest
+from scipy import stats
+
+from diff_diff import SpilloverDiD, SurveyDesign
+from diff_diff.spillover import _resolve_inference_df
+from diff_diff.utils import safe_inference
+from tests._dgp_utils import generate_butts_staggered_dgp
+from tests.test_t23_spillover_tva_drift import (
+ D_BAR_KM,
+ _build_t23_panel,
+ _silence_spillover_matmul_warnings,
+)
+
+THIS_TEST_REF = "tests/test_v4_spillover_df_convention.py"
+SP_KW = dict(outcome="y", unit="unit", time="time", treatment="D")
+N_RINGS = 1 # rings=[0, D_BAR_KM] -> one ring coefficient; k_effective = 1 + N_RINGS
+
+
+def _t_tail(t, df):
+ return 2.0 * stats.t.sf(np.abs(t), df)
+
+
+def _z_tail(t):
+ return 2.0 * stats.norm.sf(np.abs(t))
+
+
+def _fit(panel, *, survey_design=None, **kwargs):
+ est = SpilloverDiD(rings=[0.0, D_BAR_KM], conley_coords=("lat", "lon"), **kwargs)
+ with warnings.catch_warnings():
+ _silence_spillover_matmul_warnings()
+ warnings.simplefilter("ignore", UserWarning)
+ if survey_design is None:
+ return est.fit(panel, **SP_KW)
+ return est.fit(panel, survey_design=survey_design, **SP_KW)
+
+
+CONLEY_KW = dict(vcov_type="conley", conley_cutoff_km=D_BAR_KM, conley_lag_cutoff=0)
+
+
+@pytest.fixture(scope="module")
+def panel():
+ df = _build_t23_panel()
+ idx = df["unit"].str[1:].astype(int)
+ df["blk"] = idx % 10 # a unit-constant coarser cluster: 10 labels
+ df["w"] = 1.0 + (idx % 5) * 0.2 # unit-constant survey weights
+ df["stratum"] = idx % 4
+ return df
+
+
+@pytest.fixture(scope="module")
+def default(panel):
+ return _fit(panel)
+
+
+@pytest.fixture(scope="module")
+def residual(panel):
+ return _fit(panel, df_convention="residual")
+
+
+# --------------------------------------------------------------------------- #
+# (1) Signature / validation
+# --------------------------------------------------------------------------- #
+
+
+class TestKnobSurface:
+ def test_signature_default_is_cluster(self):
+ assert inspect.signature(SpilloverDiD).parameters["df_convention"].default == "cluster"
+
+ def test_get_set_params_round_trip(self):
+ est = SpilloverDiD(rings=[0.0, 50.0])
+ assert est.get_params()["df_convention"] == "cluster"
+ est.set_params(df_convention="residual")
+ assert est.df_convention == "residual" == est.get_params()["df_convention"]
+
+ def test_constructor_rejects_unknown_value(self):
+ with pytest.raises(ValueError, match="df_convention must be one of"):
+ SpilloverDiD(rings=[0.0, 50.0], df_convention="bogus")
+
+ def test_set_params_is_transactional(self):
+ est = SpilloverDiD(rings=[0.0, 50.0])
+ with pytest.raises(ValueError, match="df_convention must be one of"):
+ est.set_params(df_convention="bogus")
+ assert est.df_convention == "cluster"
+
+ def test_direct_mutation_rejected_at_fit(self, panel):
+ est = SpilloverDiD(rings=[0.0, D_BAR_KM], conley_coords=("lat", "lon"))
+ est.df_convention = "bogus"
+ with pytest.raises(ValueError, match="df_convention must be one of"):
+ est.fit(panel, **SP_KW)
+
+
+# --------------------------------------------------------------------------- #
+# (2)-(4) The three values on the default (CR1 at unit) lane
+# --------------------------------------------------------------------------- #
+
+
+class TestDefaultLane:
+ def test_default_is_t_g_minus_1(self, default, panel):
+ G = panel["unit"].nunique()
+ assert default.cluster_name == "unit" and default.n_clusters == G
+ assert default.df_convention == "cluster"
+ assert default.inference_df == G - 1 == default.n_clusters - 1
+ np.testing.assert_allclose(default.p_value, _t_tail(default.t_stat, G - 1), rtol=1e-12)
+ lo, hi = default.conf_int
+ np.testing.assert_allclose(
+ (hi - lo) / 2, stats.t.ppf(1 - default.alpha / 2, G - 1) * default.se, rtol=1e-12
+ )
+
+ def test_ring_rows_share_the_df(self, default, panel):
+ G = panel["unit"].nunique()
+ eff = default.spillover_effects
+ assert eff is not None and len(eff) == N_RINGS
+ for _, row in eff.iterrows():
+ np.testing.assert_allclose(row["p_value"], _t_tail(row["t_stat"], G - 1), rtol=1e-12)
+ np.testing.assert_allclose(
+ (row["ci_high"] - row["ci_low"]) / 2,
+ stats.t.ppf(1 - default.alpha / 2, G - 1) * row["se"],
+ rtol=1e-12,
+ )
+
+ def test_explicit_unit_cluster_is_bit_equal(self, default, panel):
+ explicit = _fit(panel, cluster="unit")
+ assert (explicit.p_value, explicit.conf_int, explicit.inference_df) == (
+ default.p_value,
+ default.conf_int,
+ default.inference_df,
+ )
+
+ def test_to_dict_carries_both_fields(self, default, panel):
+ d = default.to_dict()
+ assert d["df_convention"] == "cluster"
+ assert d["inference_df"] == panel["unit"].nunique() - 1
+
+ def test_explicit_coarser_cluster_uses_its_own_g(self, panel):
+ """G comes from the effective cluster labels, not the unit count."""
+ blk = _fit(panel, cluster="blk")
+ assert blk.cluster_name == "blk" and blk.n_clusters == 10
+ assert blk.inference_df == 9.0
+ np.testing.assert_allclose(blk.p_value, _t_tail(blk.t_stat, 9), rtol=1e-12)
+ blk_res = _fit(panel, cluster="blk", df_convention="residual")
+ assert blk_res.se == blk.se
+ assert blk_res.inference_df == blk.n_obs - (1 + N_RINGS)
+
+ def test_residual_restores_the_7b_inference(self, default, residual):
+ k = 1 + N_RINGS
+ assert residual.df_convention == "residual"
+ assert residual.inference_df == residual.n_obs - k
+ np.testing.assert_allclose(
+ residual.p_value, _t_tail(residual.t_stat, residual.n_obs - k), rtol=1e-12
+ )
+ assert residual.att == default.att and residual.se == default.se
+ assert residual.t_stat == default.t_stat
+ assert residual.p_value != default.p_value
+ assert residual.conf_int != default.conf_int
+
+ def test_normal_is_deliberate_z(self, default, panel):
+ z = _fit(panel, df_convention="normal")
+ assert z.inference_df is None and z.df_convention == "normal"
+ assert z.se == default.se and z.t_stat == default.t_stat
+ np.testing.assert_allclose(z.p_value, _z_tail(z.t_stat), rtol=1e-12)
+ lo, hi = z.conf_int
+ np.testing.assert_allclose(
+ (hi - lo) / 2, stats.norm.ppf(1 - z.alpha / 2) * z.se, rtol=1e-12
+ )
+ assert "inference_df" not in z.to_dict()
+
+
+# --------------------------------------------------------------------------- #
+# (5) Inert lanes: cluster=False and every Conley fit
+# --------------------------------------------------------------------------- #
+
+
+class TestInertLanes:
+ def test_cluster_false_is_knob_inert(self, panel):
+ off = _fit(panel, cluster=False)
+ off_res = _fit(panel, cluster=False, df_convention="residual")
+ assert off.n_clusters is None and off.cluster_name is None
+ assert off.inference_df == off.n_obs - (1 + N_RINGS)
+ assert (off.p_value, off.conf_int, off.inference_df) == (
+ off_res.p_value,
+ off_res.conf_int,
+ off_res.inference_df,
+ )
+
+ def test_conley_auto_is_knob_inert(self, panel):
+ c = _fit(panel, **CONLEY_KW)
+ c_res = _fit(panel, df_convention="residual", **CONLEY_KW)
+ assert c.n_clusters is None
+ assert c.inference_df == c.n_obs - (1 + N_RINGS)
+ assert (c.p_value, c.conf_int, c.inference_df) == (
+ c_res.p_value,
+ c_res.conf_int,
+ c_res.inference_df,
+ )
+
+ def test_conley_with_explicit_cluster_keeps_residual_df(self, panel):
+ """The product-kernel cluster is reported (n_clusters) but never a df source."""
+ cu = _fit(panel, cluster="unit", **CONLEY_KW)
+ cu_res = _fit(panel, cluster="unit", df_convention="residual", **CONLEY_KW)
+ assert cu.n_clusters == panel["unit"].nunique() and cu.cluster_name == "unit"
+ assert cu.inference_df == cu.n_obs - (1 + N_RINGS)
+ assert (cu.p_value, cu.conf_int) == (cu_res.p_value, cu_res.conf_int)
+
+ def test_conley_normal_is_z(self, panel):
+ cz = _fit(panel, df_convention="normal", **CONLEY_KW)
+ assert cz.inference_df is None
+ np.testing.assert_allclose(cz.p_value, _z_tail(cz.t_stat), rtol=1e-12)
+
+
+# --------------------------------------------------------------------------- #
+# (6) Survey df keeps precedence under every value
+# --------------------------------------------------------------------------- #
+
+
+class TestSurveyPrecedence:
+ @pytest.mark.parametrize(
+ "design, expected_df",
+ [
+ (SurveyDesign(weights="w"), lambda df: len(df) - 1), # implicit per-obs PSUs
+ (
+ SurveyDesign(weights="w", strata="stratum", psu="unit"),
+ lambda df: df["unit"].nunique() - df["stratum"].nunique(),
+ ),
+ ],
+ ids=["weights_only", "psu_strata"],
+ )
+ def test_survey_df_wins(self, panel, design, expected_df):
+ fits = {
+ v: _fit(panel, df_convention=v, survey_design=design)
+ for v in ("cluster", "residual", "normal")
+ }
+ exp = expected_df(panel)
+ for v, r in fits.items():
+ assert r.survey_metadata is not None and r.survey_metadata.df_survey == exp, v
+ assert r.inference_df == exp, v
+ base = fits["cluster"]
+ for v in ("residual", "normal"):
+ assert (fits[v].p_value, fits[v].conf_int) == (base.p_value, base.conf_int), v
+ # The PSU design's df (G - n_strata) is distinguishable from G - 1.
+ assert exp != panel["unit"].nunique() - 1 or design.psu is None
+
+
+# --------------------------------------------------------------------------- #
+# (7) Event-study mode
+# --------------------------------------------------------------------------- #
+
+
+def _fit_es(df, **kwargs):
+ est = SpilloverDiD(
+ rings=[0.0, 50.0, 200.0],
+ d_bar=200.0,
+ conley_coords=("lat", "lon"),
+ event_study=True,
+ horizon_max=2,
+ **kwargs,
+ )
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", UserWarning)
+ warnings.simplefilter("ignore", RuntimeWarning)
+ return est.fit(df, outcome="y", unit="unit", time="time", first_treat="first_treat")
+
+
+def _k_effective(res):
+ """Finite stage-2 coefficients - the synthetic 'ATT' lincom entry excluded."""
+ return sum(1 for name, v in res.coefficients.items() if name != "ATT" and np.isfinite(v))
+
+
+@pytest.fixture(scope="module")
+def es_panel():
+ return generate_butts_staggered_dgp(seed=0)
+
+
+class TestEventStudy:
+ def test_cluster_default_is_g_minus_1(self, es_panel):
+ res = _fit_es(es_panel)
+ G = es_panel["unit"].nunique()
+ assert res.n_clusters == G and res.inference_df == G - 1
+ dyn = res.att_dynamic
+ finite = dyn["p_value"].notna() & (dyn["se"] > 0)
+ assert finite.sum() >= 3
+ np.testing.assert_allclose(
+ dyn.loc[finite, "p_value"], _t_tail(dyn.loc[finite, "t_stat"], G - 1), rtol=1e-12
+ )
+ np.testing.assert_allclose(res.p_value, _t_tail(res.t_stat, G - 1), rtol=1e-12)
+
+ def test_residual_uses_n_minus_k(self, es_panel):
+ res = _fit_es(es_panel, df_convention="residual")
+ k = _k_effective(res)
+ assert res.inference_df == res.n_obs - k
+ dyn = res.att_dynamic
+ finite = dyn["p_value"].notna() & (dyn["se"] > 0)
+ np.testing.assert_allclose(
+ dyn.loc[finite, "p_value"],
+ _t_tail(dyn.loc[finite, "t_stat"], res.n_obs - k),
+ rtol=1e-12,
+ )
+
+ def test_normal_is_z_through_the_extractor(self, es_panel):
+ res = _fit_es(es_panel, df_convention="normal")
+ assert res.inference_df is None
+ dyn = res.att_dynamic
+ finite = dyn["p_value"].notna() & (dyn["se"] > 0)
+ np.testing.assert_allclose(
+ dyn.loc[finite, "p_value"], _z_tail(dyn.loc[finite, "t_stat"]), rtol=1e-12
+ )
+ np.testing.assert_allclose(res.p_value, _z_tail(res.t_stat), rtol=1e-12)
+
+
+# --------------------------------------------------------------------------- #
+# (8) The df ladder itself (saturation is unreachable through a fit)
+# --------------------------------------------------------------------------- #
+
+
+KNOBS = ("cluster", "residual", "normal")
+
+
+class TestResolveInferenceDf:
+ @pytest.mark.parametrize("knob", KNOBS)
+ @pytest.mark.parametrize(
+ "n_obs_eff, k_effective, substring",
+ [
+ (4, 4, "inference df = 0 (n_obs=4, effective_rank=4"),
+ (5, 8, "inference df = -3 (n_obs=5, effective_rank=8"),
+ ],
+ )
+ def test_saturated_sample_is_nan_sentinel(self, knob, n_obs_eff, k_effective, substring):
+ with pytest.warns(UserWarning, match=substring.replace("(", r"\(")):
+ df = _resolve_inference_df(
+ df_convention=knob,
+ n_obs_eff=n_obs_eff,
+ k_effective=k_effective,
+ n_clusters=20,
+ vcov_mode="cluster",
+ survey_df=None,
+ has_survey=False,
+ )
+ assert df == 0.0
+
+ @pytest.mark.parametrize("knob", KNOBS)
+ def test_survey_saturated_is_nan_sentinel(self, knob):
+ with pytest.warns(UserWarning, match="df_survey = 0"):
+ df = _resolve_inference_df(
+ df_convention=knob,
+ n_obs_eff=302,
+ k_effective=2,
+ n_clusters=20,
+ vcov_mode="cluster",
+ survey_df=0,
+ has_survey=True,
+ )
+ assert df == 0.0
+
+ @pytest.mark.parametrize("knob", KNOBS)
+ def test_survey_df_beats_every_knob(self, knob):
+ df = _resolve_inference_df(
+ df_convention=knob,
+ n_obs_eff=302,
+ k_effective=2,
+ n_clusters=20,
+ vcov_mode="cluster",
+ survey_df=12,
+ has_survey=True,
+ )
+ assert df == 12.0
+
+ @pytest.mark.parametrize(
+ "vcov_mode, n_clusters, expected",
+ [
+ ("cluster", 20, {"cluster": 19.0, "residual": 300.0, "normal": None}),
+ ("conley", 20, {"cluster": 300.0, "residual": 300.0, "normal": None}),
+ ("hc1", None, {"cluster": 300.0, "residual": 300.0, "normal": None}),
+ ],
+ ids=["cluster_mode", "conley_mode", "hc1_mode"],
+ )
+ def test_knob_branches(self, vcov_mode, n_clusters, expected):
+ for knob, want in expected.items():
+ got = _resolve_inference_df(
+ df_convention=knob,
+ n_obs_eff=302,
+ k_effective=2,
+ n_clusters=n_clusters,
+ vcov_mode=vcov_mode,
+ survey_df=None,
+ has_survey=False,
+ )
+ assert got == want, (vcov_mode, knob)
+
+ def test_zero_sentinel_is_all_nan_in_safe_inference(self):
+ t, p, ci = safe_inference(1.0, 0.5, alpha=0.05, df=0.0)
+ assert np.isnan(t) and np.isnan(p) and np.isnan(ci[0]) and np.isnan(ci[1])
+
+
+# --------------------------------------------------------------------------- #
+# (9) Ledger
+# --------------------------------------------------------------------------- #
+
+
+def test_ledger_row_is_done_and_points_here():
+ from tests.test_v4_matrix import ROWS as LEDGER_ROWS
+
+ row = {r["id"]: r for r in LEDGER_ROWS}["M-154"]
+ assert row["kind"] == "behavior"
+ assert row["status"] == "done"
+ assert row["test_ref"] == THIS_TEST_REF
+ assert row["old"] == "diff_diff:SpilloverDiD[df_convention]"
+ assert row["introduced_in"] == "4.0"
diff --git a/tests/test_variance_conventions.py b/tests/test_variance_conventions.py
index 3612769b3..b1bae41b6 100644
--- a/tests/test_variance_conventions.py
+++ b/tests/test_variance_conventions.py
@@ -78,6 +78,21 @@ def make_panel() -> pd.DataFrame:
return pd.DataFrame(rows)
+def _with_coords(df: pd.DataFrame) -> pd.DataFrame:
+ """Deterministic coordinate overlay for the SpilloverDiD rows (documented in
+ docs/methodology/variance-conventions.md "Shared DGP"): ever-treated units sit
+ in a ~7 km cluster at lat = 0.001 * unit, lon = 0; never-treated units alternate
+ between a near band (lat 0.5, ~55 km, inside the 100 km ring) and a far band
+ (lat 3.0, ~330 km, the Omega_0 far-away controls present in every period)."""
+ out = df.copy()
+ ever = out["first_treat"] > 0
+ out["lat"] = np.where(
+ ever, 0.001 * out["unit"], np.where((out["unit"] // 4) % 2 == 0, 0.5, 3.0)
+ )
+ out["lon"] = 0.0
+ return out
+
+
# ---------------------------------------------------------------------------
# Programmatic discovery of instrumentation targets.
# ---------------------------------------------------------------------------
@@ -545,6 +560,22 @@ def snapshot(self):
"no CR1 sandwich, normal-theory safe_inference gate only"
),
),
+ dict(
+ key="spillover_cluster_unit",
+ fit=lambda df: diff_diff.SpilloverDiD(
+ rings=[0.0, 100.0], conley_coords=("lat", "lon"), df_convention="residual"
+ ).fit(_with_coords(df), outcome="y", unit="unit", time="time", first_treat="first_treat"),
+ cr1_k=(),
+ # Two safe_inference calls (aggregate tau_total + the one ring), both at
+ # the stage-2 residual df n - k = 360 - 2.
+ tail_df=(358.0, 358.0),
+ status="legitimate",
+ reason=(
+ "L3: Wave D Gardner-GMM two-stage sandwich with its own CR1 multiplier "
+ "G/(G-1)*(n-1)/(n-p_2) (two_stage meat, never the shared CR1). Explicit "
+ "df_convention='residual' since the 4.0 default is G-1 (see the _default_df twin)."
+ ),
+ ),
# ---- 4.0 default twins: identical fits WITHOUT the explicit knob, so the
# tail df is G-1 (G = clusters with positive weight on each surface's
# clustered sample). The cr1_k literals are copied from the residual twins
@@ -672,6 +703,20 @@ def snapshot(self):
"same CR1 k as the explicit-'residual' twin, tail df G-1"
),
),
+ dict(
+ key="spillover_cluster_unit_default_df",
+ fit=lambda df: diff_diff.SpilloverDiD(rings=[0.0, 100.0], conley_coords=("lat", "lon")).fit(
+ _with_coords(df), outcome="y", unit="unit", time="time", first_treat="first_treat"
+ ),
+ cr1_k=(),
+ # G-1 with G = 60 units (the 4.0 unit auto-cluster), on both calls.
+ tail_df=(59.0, 59.0),
+ status="legitimate",
+ reason=(
+ "t(G-1): the 4.0 df_convention default (M-154 - SpilloverDiD gained the knob at the "
+ "cut) - no shared-CR1 call, as the explicit-'residual' twin; tail df G-1"
+ ),
+ ),
]
_FAST_KEYS = {
@@ -685,6 +730,8 @@ def snapshot(self):
"wooldridge_hc1_within_default_df",
"lpdid_pre2_post2",
"callaway_santanna_default",
+ "spillover_cluster_unit",
+ "spillover_cluster_unit_default_df",
}