ECCD: Enhanced Cyclic Coordinate Descent Methods for Elastic-Net Penalised Linear Models

Official repository for Enhanced Cyclic Coordinate Descent (ECCD) — 🏆 NeurIPS 2025 poster. ECCD accelerates elastic‑net GLMs by unrolling cyclic CD updates via a second‑order approximation; s=1 recovers CD and s>1 yields speedups. Implemented in C++/Eigen, we observe ~3× average speedups on regularisation‑path benchmarks.

1. Overview

Efficient Coordinate Descent (ECCD) is our re-engineering of the classical coordinate-descent machinery popularised by glmnet. We retain the statistical robustness and model‐path semantics of the original algorithm while optimising the computational core for modern high-dimensional problems. The implementation exposes both a pure R workflow (via the scripts in ablation/ and src/original/) and a C++ backend that is callable from R through Rcpp. Major enhancements include:

• Block-wise updates with user-controllable quota so that memory traffic and cache reuse can be tuned to the feature dimension.
• Warm-start preservation along the regularisation path identical to glmnet, allowing drop-in use for practitioners used to ℓ₁/elastic-net pipelines.
• Extensive instrumentation (convergence monitors, memory trackers, and path visualisers) to enable statistically rigorous comparisons against baselines such as glmnet, ncvreg, biglasso, skglm, and blitzl1.

This repository accompanies our ECCD study and provides all artefacts needed to reproduce ablation experiments and benchmark comparisons.

2. Quick Start

2.1 Repository setup

Clone the repository and create the ECCD conda environment:

git clone https://github.com/Yixiao-Wang-Stats/ECCD.git
cd ECCD
conda env create -f environment.yml
conda activate eccd

Alternatively, install the Python dependencies directly:

pip install -r requirements.txt

2.2 R dependencies

Several R scripts rely on standard statistical packages. Launch R (within the repository) and install:

install.packages(c(
  "Matrix", "matrixStats", "Rcpp", "glmnet",
  "ncvreg", "biglasso", "bigmemory", "sda",
  "bench", "microbenchmark", "entropy", "fdrtool"
))

Note. Some ablation scripts use Bioconductor data; install as needed, e.g. BiocManager::install("airway").

2.3 Smoke test

The most direct assertions live in test/. For a fast run:

Rscript ./test/demo.R

These scripts confirm that the ECCD implementation reproduces the ℓ₁/elastic-net path and achieves the expected runtime profile relative to glmnet.

3. Ablation Studies

The ablation/ directory contains targeted experiments that stress the algorithm under different operating regimes.

3.1 Convergence analysis (ablation/convergence/)

• test_converge.R and test_converge_bcd.R examine path-wise convergence diagnostics, including relative primal decrease and KKT residuals.
• Usage:

  cd ablation/convergence
  Rscript test_converge.R        # ECCD default
  Rscript test_converge_bcd.R    # Blocked coordinate descent variant

  Both scripts emit intermediate res objects so you can inspect iteration histories in R.

3.2 Memory management (ablation/memory_experiment/)

• run_mem.sh orchestrates multiple block-size settings via mem_reset_real.R / mem_reset_fixed.R and logs resident-set usage.
• The C++ helpers (eccd_path_mem_cum.cpp, eccd_path_mem_fixed_s.cpp) expose get_current_rss so you can profile allocation behaviour.
• Usage:

  cd ablation/memory_experiment
  bash run_mem.sh

3.3 Sigmoid microbenchmarks (ablation/sigmoid_speed/)

• testC.R and testC_array.R quantify the cost of scalar vs array sigmoid kernels for tuning the logistic inner loop.
• Usage:

  cd ablation/sigmoid_speed
  Rscript testC.R
  Rscript testC_array.R

Each ablation folder contains a README snippet or inline comments describing additional switches (e.g. fixed vs adaptive block sizes). All scripts assume the ECCD C++ module is compiled in place via Rcpp::sourceCpp.

4. Benchmark Suite

We benchmark ECCD against a diverse set of solvers (single and path-wise):

• glmnet (C-coded coordinate descent)
• ncvreg (non-convex penalties)
• biglasso (memory-mapped path solver)
• skglm (proximal/newton solvers for sparse GLMs)
• blitzl1 (sparse logistic regression in C++)
• Additional baselines via Benchopt (see below)

4.1 Repository benchmarks

• Original path comparisons: src/original/path/logistic/test_perf_w_baselines.R. This script loads curated datasets (e.g. Duke, Colon Cancer), constructs the λ-grid, and reports accuracy, objective alignment, and runtime across all baselines. Results are written to result/.
• Synthetic/real comparisons in Python: the benchmark/ tree contains solver-specific folders (biglasso/, skglm/, blitz/). Each folder exposes dedicated drivers (reb_biglasso.R, run_skglm_logistic_compare.py, etc.) with detailed instructions in the source comments.

Before running the Python benchmarks, ensure the ECCD environment (eccd) is active. All R-based benchmarks reuse the packages listed in §2.2.

4.2 Benchopt integration

Benchopt experiments reside in benchmark/benchopt_exp/<benchmark_name>/. Each benchmark mirrors the official Benchopt structure (datasets folder, solvers folder, objective.py).

To reproduce our Benchopt comparisons:

1. Install Benchopt following the official guide.
2. Place the relevant data files into the datasets/ directory of the specific benchmark (e.g. benchmark/benchopt_exp/benchmark_lasso/datasets/).
3. From the benchmark root, invoke Benchopt (example for the Lasso benchmark):

   cd benchmark/benchopt_exp/benchmark_lasso
   benchopt run .

4. Benchopt handles solver orchestration; ECCD results can then be contrasted with the reference solvers in the Benchopt report.

Reminder. Mesh your environment (eccd) with Benchopt’s installation instructions so that all dependencies (pyarrow, scikit-learn, etc.) are picked up. We recommend using the ECCD environment and installing Benchopt within it to avoid version conflicts.

---

For questions or reproducibility issues, please open a GitHub issue with the script name, command used, and environment information (conda list / sessionInfo() in R). Enjoy exploring ECCD!!