Compact LUAD Bulk Classifier Deconstruction

This project investigates whether ultra-compact lung adenocarcinoma tumor-normal classifiers learned from bulk expression data represent malignant-cell biology or mixed tissue-state signals.

Full manuscript: reports/manuscript_v3.md (title, abstract, introduction, results, methods, figure legends, discussion — no placeholders).

Core idea

Bulk RNA-seq tumor-normal classifiers can achieve very high accuracy, but accuracy alone does not reveal what the model is using. A compact classifier may rely on malignant-cell programs, normal tissue loss, stromal remodeling, immune composition, or a mixture of these signals.

This project builds a full validation pipeline:

1. TCGA-LUAD sparse compact panel discovery
2. Random-panel benchmarking
3. Repeated split stability analysis
4. External validation on GSE32863 and GSE7670
5. Single-cell deconstruction using GSE131907

Main finding

The strongest compact contrast discovered was:

PYCR1 - CD5L

This two-gene contrast transferred directly to two independent GEO cohorts without retraining:

Cohort	AUC	Balanced accuracy
GSE32863	0.994	0.966
GSE7670	0.995	0.983

Single-cell audit showed that this contrast is biologically mixed:

• PYCR1: malignant/tumor-associated epithelial-metabolic compartment
• CD5L: alveolar macrophage / normal-lung myeloid context

Therefore, the strongest compact bulk classifier is not a purely malignant-cell-intrinsic biomarker signature. It captures a cross-compartment tissue-state contrast.

Repository structure

data/
  raw/         (gitignored — re-download per reports/16_reproducibility_package.md)
  processed/   (gitignored — regenerated by running notebooks 01-10 in order)

notebooks/
  01_inspect_expression.ipynb
  02_build_matrix.ipynb
  03_baseline_model.ipynb
  04_gene_importance.ipynb
  05_unconstrained_panel_selection.ipynb
  06_repeated_split_stability.ipynb
  07_external_geo_validation_plan.ipynb
  08_second_geo_validation.ipynb
  09_single_cell_audit.ipynb
  10_make_core_figures.ipynb

reports/
  11_draft_results_section.md
  12_methods_section.md
  13_abstract_and_title.md
  14_figure_legends.md
  15_final_audit_checklist.md
  16_reproducibility_package.md
  manuscript_number_audit.md
  manuscript_v3.md          (final manuscript)

figures/
  fig1_tcga_panel_compression_random_baseline.png
  fig2_gene_stability_k2.png
  fig2_gene_stability_k3.png
  fig2_gene_stability_k5.png
  fig3_external_validation_auc.png
  fig3_external_validation_bal_acc.png
  fig4_single_cell_gene_buckets.png

requirements.txt   (pip environment)
environment.yml     (conda environment)

Reproducibility

Install dependencies:

pip install -r requirements.txt
# or: conda env create -f environment.yml

Run notebooks in order:

1. 01_inspect_expression.ipynb
2. 02_build_matrix.ipynb
3. 03_baseline_model.ipynb
4. 04_gene_importance.ipynb
5. 05_unconstrained_panel_selection.ipynb
6. 06_repeated_split_stability.ipynb
7. 07_external_geo_validation_plan.ipynb
8. 08_second_geo_validation.ipynb
9. 09_single_cell_audit.ipynb
10. 10_make_core_figures.ipynb

Expected final manuscript:

reports/manuscript_v3.md

Expected final figures:

figures/

Download raw datasets into data/raw/ before running from scratch — see data/README.md and reports/16_reproducibility_package.md for paths and a claim-by-claim verification table.

What this project does not claim

This project does not claim to produce a clinical diagnostic test. It evaluates robustness, transferability, and biological interpretation of compact bulk expression signatures.

Key conclusion

Compact bulk expression classifiers should be evaluated using random-panel baselines, repeated split stability, external validation, and single-cell deconstruction before being interpreted as biologically precise biomarkers.