- As outlined in https://arxiv.org/abs/2602.18577, the lasso lambda controls the maximum covariate imbalance. If the lambda sequence contains values that is not feasible for a given dataset, we naturally want to stop and truncate the path.
This is currently done by recognizing when the adelie solver reaches the maximum number of iterations and declares non-convergence:
|
EXPECTED_AD_ERROR <- "adelie_core solver: max coordinate descents reached at lambda index: 0." |
A nice thing with this, is that at the default convergence criteria and tolerances, this gracefully stops the path where we want it without blowing up the coefficients like an unpenalized loss would.
A drawback is that we have to wait for the defaultmaxit=100k number of coordinate descent iterations for the solver to declare non-convergence.
Question: Are there ways to avoid waiting for maxit=100k iterations at default tolerance levels tol=1e-7.
Obviously, a current heuristic would just be to reduce maxit. Note that you'd typically not want to reduce the tol parameter too much as step sizes could get too large and you'd start moving into the regime where the coefficients blow up.
(this added line in the GLM solver is a heuristic to stop when that happens)
The current default, as currently seen in simulations, avoids this completely (props to the adelie solver authors for all the work going into refining all the tolerances to work so well out of the box).
- Additional improvement: There may be better alternatives to this "deviance" based tol scaling workaround for the cbps loss (where deviance isn't well defined)
|
tol * (loss_null - loss_full) / hess_sum, // NOTE: re-defined for balnet (parenthesis always equals 1) |
e.g., suspect balnet could be sped up on larger problems by using a lower effective tolerance.
This is currently done by recognizing when the adelie solver reaches the maximum number of iterations and declares non-convergence:
balnet/r-package/balnet/R/balnet.fit.R
Line 1 in 06bf8b7
A nice thing with this, is that at the default convergence criteria and tolerances, this gracefully stops the path where we want it without blowing up the coefficients like an unpenalized loss would.
A drawback is that we have to wait for the default
maxit=100knumber of coordinate descent iterations for the solver to declare non-convergence.Question: Are there ways to avoid waiting for
maxit=100kiterations at default tolerance levelstol=1e-7.Obviously, a current heuristic would just be to reduce
maxit. Note that you'd typically not want to reduce thetolparameter too much as step sizes could get too large and you'd start moving into the regime where the coefficients blow up.(this added line in the GLM solver is a heuristic to stop when that happens)
The current default, as currently seen in simulations, avoids this completely (props to the adelie solver authors for all the work going into refining all the tolerances to work so well out of the box).
balnet/core/include/adelie_ext/solver_cbps_naive.hpp
Line 450 in 2e81945
e.g., suspect balnet could be sped up on larger problems by using a lower effective tolerance.