optlang interface for the HiGHS solver

[axelvonkamp]

supports LP and QP, but QP performance is not as good as OSQP
possible long-term alternative for GLPK, which seems not to be actively developed any more
no integer variables/MIP support yet, but in principle possible with HiGHS
test script is included

[cdiener]

This is awesome thanks so much. I also was planning this to eventually replace GLPK as reference implementation because it actually supports all solver types (LP, QP, MIP). Will take me a bit to go through everything but I am excited for it.

[axelvonkamp]

This is awesome thanks so much. I also was planning this to eventually replace GLPK as reference implementation because it actually supports all solver types (LP, QP, MIP). Will take me a bit to go through everything but I am excited for it.

Well, you can also thank Claude for it ;-). I mostly directed the implementation towards using the solver object as ground truth and to make it possible to set up everything without the need for symbolic expressions (which still can be used if required).
I also let Claude implement a SCIP interface which in principle works, but it is not as streamlined as the HiGHS interface. If you are interested I can make another pull request for that, but it does not come with a test suite at this stage. I've decided to focus on HiGHS because it has, in my opinion, a cleaner Python interface, and it also drops the GIL when solving.

[cdiener]

Great job. Everything works well. Also passes the cobrapy tests. FVA is quite a bit slower than GLPK but this seems to be an intrinsic Highs thing. Resolve even with basis recycling is slower it seems.

One thing I would like to have is the to_lp and from_lp methods implemented and use them for setstate and getstate. Right now serialization uses JSON which is quite slow. As this is imported for switching solvers (clone) it would be good to have the same strategy as CPLEX (serialize through the .lp).

Thanks again for this. Really cool!

[axelvonkamp]

FVA is quite a bit slower than GLPK but this seems to be an intrinsic Highs thing. Resolve even with basis recycling is slower it seems.

Yes, I noticed that too, but there is a simple explanation for this: GLPK runs a primal simplex by default,, while HiGHS runs a dual simplex. If one sets HiGHS to primal simplex the speed of both is about the same. I think this is something that should taken into account on the cobrapy side.

[axelvonkamp]

One thing I would like to have is the to_lp and from_lp methods implemented and use them for setstate and getstate. Right now serialization uses JSON which is quite slow. As this is imported for switching solvers (clone) it would be good to have the same strategy as CPLEX (serialize through the .lp).

According to interface.py
def clone(cls, model, use_json=True, use_lp=False):
cloning is currently done via JSON by default. In optlang's and cobrapy's code I did not find any active use of cloning via LP, that's why I skipped this, or did I miss something?

[cdiener]

Yes, I noticed that too, but there is a simple explanation for this: GLPK runs a primal simplex by default,, while HiGHS runs a dual simplex. If one sets HiGHS to primal simplex the speed of both is about the same. I think this is something that should taken into account on the cobrapy side.

That's what I thought as well. Might make sense to set simplex_strategy to 0 ("choose") instead. That will pick the primal simplex for simpler models by default. With that it's as fast or faster than GLPK for me.

[cdiener]

According to interface.py def clone(cls, model, use_json=True, use_lp=False): cloning is currently done via JSON by default. In optlang's and cobrapy's code I did not find any active use of cloning via LP, that's why I skipped this, or did I miss something?

It's used when setting the solver via model.solver = "..." in COBRAPY. But you are right it always uses JSON apparently. For GLPK and CPLEX though the serialization via __[s/g]etstate__ uses the LP as well which is faster than JSON. This is used in all model copying in COBRAPY and in also in all multiprocessing code (to broadcast the model object).

Would also be nice to have the [from/to]_lp methods because without a commercial solver optlang currently does not support reading or writing QPs from/to .lp files.

This is just a feature though. We can also add this later.

[axelvonkamp]

According to interface.py def clone(cls, model, use_json=True, use_lp=False): cloning is currently done via JSON by default. In optlang's and cobrapy's code I did not find any active use of cloning via LP, that's why I skipped this, or did I miss something?

It's used when setting the solver via model.solver = "..." in COBRAPY. But you are right it always uses JSON apparently. For GLPK and CPLEX though the serialization via __[s/g]etstate__ uses the LP as well which is faster than JSON. This is used in all model copying in COBRAPY and in also in all multiprocessing code (to broadcast the model object).

Would also be nice to have the [from/to]_lp methods because without a commercial solver optlang currently does not support reading or writing QPs from/to .lp files.

This is just a feature though. We can also add this later.

I will be on vacation for the coming three weeks and will get back to it then.

[cdiener]

I will be on vacation for the coming three weeks and will get back to it then.

Have a great vacation. If you are fine with it, I would already merge it as is and then we can follow up with an additional PR. That way I can start testing it a bit more in COBRAPY and MICOM.