Repository navigation
Conversation
|
@Advueu963 one small thing: the CI run is sitting at |
|
Thanks for approving the run, @Advueu963 — that surfaced a real problem on my side.
The push queued a new run that is sitting at |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
@bongho, @Advueu963 told me about the conversations you guys had and the work you are doing with the lab. We are currently in the midst of rewriting the shapiq_Games and benchmark machinery in #616 and I would like to get the rewrite in first before brining this in. Would you be open to adapt this PR after we merged the other one in? I‘d ping you here? |
|
Thanks @mmschlk — that order makes sense, and yes, I'm happy to adapt this after #616 lands. Looking at #616, it reworks I'll leave this open and wait for your ping. When it comes I'll rebase onto the new layout, re-run the oracle cross-check (numpy closed form vs No rush — the rewrite looks like a big one. |
8b8850e to
d83aa0f
Compare
|
@mmschlk #616 is in, so I rebased this onto the new layout. Thanks for the heads-up on the ordering, reviewing against the old What the adaptation changed:
What I ran on the rebased branch:
No rush on the review, I know v1.8.0 still has items open. |
d83aa0f to
4c0bcd2
Compare
|
Rebased onto v1.8.0. The only conflict was the CHANGELOG, since the release turned
|
Adds LinearGaussianSCM and ConfoundedChainSCM to shapiq_games.synthetic. Both evaluate the marginal, the conditional and the causal value function of Heskes et al. (2020) in closed form on one linear-Gaussian data-generating process, so the three semantics can be compared on the same model and the same point without sampling or a random seed. A linear model on Gaussian features keeps every conditional expectation linear, which is what makes the closed form possible. That also makes the games exact references for benchmarking approximators: no seed, no approximation error, and the Shapley values follow from the eight coalitions of the three-feature example exactly. The gallery example sits with the imputers in technicalities, where the marginal/conditional distinction is already drawn, and extends it with the causal value function: the same feature is +3.00 marginally and -2.33 conditionally.
4c0bcd2 to
00dae92
Compare
Motivation and Context
Closes #599.
Causal Shapley values (Heskes et al., 2020) are not available in a pure-Python stack today. The paper's code supplement was folded into the R package shapr; the Python entry point
pyshapris anrpy2wrapper and needs a working R installation. The same request has been open onshap(shap#3914) since 2024-11 with no assignee.Following @Advueu963's answer on #599, this adds the value function to
shapiq_games, in the shape #551 established. It starts at the synthetic end, because that is where the answer is checkable: on a known SCM all three value functions a Shapley explanation can be built on are available in closed form, so the game is its own ground truth.LinearGaussianSCMtakes a linear model on Gaussian features and evaluates one of three value functions on it:marginalconditionalcausalThe causal structure is taken the way shapr takes it: a
causal_orderingof feature groups plus oneconfoundingflag per group, which together specify the causal chain graph. The non-intervened members of a group are drawn conditional on every preceding group, plus the intervened members of the same group only when that group is not confounded. That is the rule shapr implements inget_S_causal_steps(R/asymmetric_and_causal_Shapley.R).One thing worth noting for review: in the linear-Gaussian case the three value functions are fully determined by the coefficients, the explanation point, the feature mean and covariance, and the chain graph. The structural coefficients are not needed, so the class takes a covariance matrix rather than an SCM object.
ConfoundedChainSCMis a three-feature preset where the three semantics disagree in sign:marginalconditionalcausalX3 is the largest positive contributor under the first row and pushes the prediction down under the other two. Against its unconditional mean of zero, 1 is above average; against what its parents predict for it (2·1 + 1·1 = 3), it came in two units low. Conditional and causal then part ways on X2, because intervening on X1 cuts the confounded path between them while conditioning on it does not.
Public API Changes
Two new names exported from
shapiq_games.synthetic:LinearGaussianSCM(coefficients, x_explain, cov, *, mean=None, causal_ordering=None, confounding=None, mode="causal", normalize=True)ConfoundedChainSCM(*, mode="causal", normalize=True)Nothing existing changes.
docs/source/references.bibgains aheskes2020entry andexamples/causal/README.txtgains one sentence.How Has This Been Tested?
The value functions were derived by hand in closed form and cross-checked two ways that do not share code: a standalone NumPy script enumerating all 8 coalitions, and
shapiq.ExactComputerrun over the game in this PR. Both give the table above to 4 decimal places, and all three satisfy efficiency (sum 6.00 = f(x*)).tests/shapiq_games/tests_unit/test_linear_gaussian_scm.pyadds 17 tests: the three closed-form attributions, efficiency for each, two structural identities that must hold on any correct implementation (marginal == causalon the sink node,conditional == causalonce every ancestor is fixed), the effect of flipping theconfoundingflag, the degenerate single-group case collapsing to the observational value function, and argument validation.pyproject.tomlalready liststests/shapiq_gamesintestpathsand the directory did not exist yet, so these are the first tests in it. I did not touch.github/workflows/ci.yml, which currently runstests/shapiqandtests/shapiq_benchmarkexplicitly — happy to add the path if you want it in CI.uv run pre-commit run --all-filespasses every hook exceptty, which fails identically on a cleanmain(22error[...]diagnostics, none in the files this PR touches; they are the optionalwoodelfimports insrc/shapiq/tree/explainer.pyand friends).The example was run end to end and reproduces the table.
Checklist
CHANGELOG.md(if relevant for users).Three notes for you, none blocking:
On validating against shapr. Add causal Shapley values (Heskes et al., 2020) #599 said I would check the numbers against shapr in R. I dropped that and used the closed form instead. shapr's causal Shapley is a Monte Carlo estimator, so agreement with it would have been agreement to within sampling error; the analytic value is the stronger reference, and it keeps the test suite free of an R dependency. The shapr algorithm is still the source for the rule, just not for the numbers.
On where the causal value function should live long term. While building this I noticed
shapiq.imputeralready hasGaussianImputerandGaussianCopulaImputer, which is the conditional value function on Gaussian features, and thatImputersubclassesGame. So aCausalImputersubclassingGaussianImputerwould reuse that machinery and would plug intoTabularExplainer(imputer=...), giving causal Shapley values on a user's own fitted model rather than only on a synthetic benchmark. That felt like a separate PR, and I did not want to relocate what you had just pointed me at. Would you want that as a follow-up?On the CHANGELOG heading. The file currently starts at
## v1.7.0and I could not find an unreleased-section convention anywhere in its history, so I opened a## Unreleasedheading. Say the word if you would rather the entry went somewhere else, or nowhere until release time.