Skip to content

research: parsimony and regime-contrastive analysis of the bias signal - #29

Merged
hallelx2 merged 1 commit into
mainfrom
halleluyaholudele/hal-4221-parsimony-regime-contrastive-analysis
Oct 25, 2026
Merged

hallelx2 merged 1 commit into
mainfrom
halleluyaholudele/hal-4221-parsimony-regime-contrastive-analysis

Conversation

@hallelx2

Copy link
Copy Markdown
Collaborator

Applies the Anamnesis thesis — conditions, not complexity, are what hide a parsimonious rule — and the Regime-Contrastive Attribution Framework to the rebuilt pipeline's 5,511 features across four regimes.

Parsimony: the complexity was not carrying the signal

Features ranked by worst-case selection frequency across all four regimes, so the ranking is itself regime-filtered. Random subsets at matched size are the control.

k stratified scaffold receptor
ranked / random ranked / random ranked / random
2 0.366 / 0.130 0.335 / 0.131 0.293 / 0.177
11 0.500 / 0.106 0.430 / 0.172 0.284 / 0.137
20 0.536 / 0.322 0.434 / 0.203 0.260 / 0.171
all 5,511 0.529 0.454 0.249

20 features beat all 5,511 under stratified; two beat it under receptor-grouped. The curve is flat from k≈20 in every regime.

The contrastive filter names the shortcuts

14 of the top-50 under the easiest regime are absent from the top-50 under the hardest:

qed  RingCount  NPR2  PMI1  SPS  AvgIpc  BCUT2D_LOGPLOW  EState_VSA2
EState_VSA5  FpDensityMorgan2  PEOE_VSA2  VSA_EState1  morgan_1683
ifp_BW5.39_VdWContact

Coherent pattern: gross size and shape descriptors. Receptor families differ systematically in ligand size, so these encode which receptor and evaporate on an unseen one.

The 11 survivors are charge-distribution surface area (PEOE_VSA1/7/9), electrotopological state (EState_VSA3, VSA_EState3), LogP, SpherocityIndex, and three docking scalars. No receptor contact or BW position survives. vina_pose_diversity_rmsd has selection frequency 1.00 in every regime.

The four-class framing is not supported

One-vs-rest against a 100-draw label-permutation null, receptor-grouped:

target n pos observed null mean ± sd p
ERK 34 0.5219 0.4843 ± 0.0132 0.0495
G protein selectivity 52 0.4764 0.4778 ± 0.0165 0.4158

Selectivity sits below its null — a clean negative. ERK is on the boundary and worth +0.038 over "always negative".

Defining the axis from bias_pathway instead gives 316 clean rows and a usable model: AUC 0.866 stratified, 0.803 scaffold-grouped. Under receptor-grouped the 11 features beat all 5,511 (AUC 0.610 vs 0.578).

A graph network hits the same ceiling

A GIN on molecular graphs, ligand only, reproduces the collapse signature 0.393 → 0.299 → 0.209. Three representations — engineered descriptors, SMILES embedding, molecular graph — one ceiling. That makes the limit look like a property of the data.

Caveats, carried in the document

  • Ranking came from SHAP on the same data, not nested inside folds → absolute ranked figures are optimistic; the random-control contrast is not affected
  • Grouped regimes return sd = 0.0000 across seeds (HAL-4196) → no error bars on grouped deltas
  • 166 of 254 scaffolds are singletons → scaffold-grouping is close to random for most rows
  • 62 duplicate (ligand, receptor) pairs cross the stratified split → stratified is inflated
  • One receptor is 21% of the data → receptor-grouped folds are high-variance

Verification

ruff check scripts/experiments/ clean. Artefacts: rcaf_parsimony.json, binary_erk_splits.json, gnn_arm.json.

Closes HAL-4221

Applies the Anamnesis thesis -- conditions, not complexity, are what hide a
parsimonious rule -- and the Regime-Contrastive Attribution Framework to the
rebuilt pipeline's 5,511 features across four regimes.

Parsimony. Features ranked by worst-case selection frequency across the four
regimes: 20 ranked features beat all 5,511 under stratified (0.536 vs 0.529),
and two beat it under receptor-grouped (0.293 vs 0.249). Random subsets at
matched size give 0.106-0.177 against 0.284-0.500 for ranked, so it is the
selection and not the count. The curve is flat from about k=20 in every
regime: the other 5,491 columns are not carrying signal.

The contrastive filter names the shortcuts. 14 of the top 50 under the
easiest regime are absent from the top 50 under the hardest, and they are
gross size and shape descriptors -- qed, RingCount, NPR2, PMI1, SPS. Those
encode which receptor a ligand came from and evaporate on an unseen one. The
11 that survive all four regimes are ligand charge-distribution and
electrotopological descriptors plus three docking scalars. No receptor
contact and no Ballesteros-Weinstein position survives.

The four-class framing is not supported. Tested one-vs-rest against a
label-permutation null under receptor-grouped CV, G-protein selectivity is
indistinguishable from chance (p = 0.42, observed below the null mean) and
ERK is marginal (p = 0.0495, +0.038 over always-negative). Defining the axis
from bias_pathway instead gives 316 clean rows of G protein against
beta-arrestin and a usable model: AUC 0.866 stratified, 0.803
scaffold-grouped -- and under receptor-grouped the 11 features beat all 5,511.

A graph network reaches the same ceiling from a different direction. A GIN on
molecular graphs alone reproduces the collapse signature, 0.393 -> 0.299 ->
0.209, which makes the limit look like a property of the data rather than of
the representation.

The caveats travel with the numbers, in the document: the ranking was derived
from SHAP on the same data rather than nested inside each fold, so absolute
ranked figures are optimistic; grouped regimes return zero seed variance so
their deltas have no error bars; 166 of 254 scaffolds are singletons, making
scaffold-grouping close to random for most rows; 62 duplicate ligand-receptor
pairs cross the stratified split; and one receptor is 21% of the data.
@hallelx2
hallelx2 merged commit 30c713b into main Oct 25, 2026
2 checks passed
@hallelx2
hallelx2 deleted the halleluyaholudele/hal-4221-parsimony-regime-contrastive-analysis branch October 25, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant