Skip to content

Configure SMT solvers in solvers.toml, generated by configure-solvers - #55

Merged
jkhav merged 8 commits into
masterfrom
feat/solver-config
Oct 27, 2026
Merged

jkhav merged 8 commits into
masterfrom
feat/solver-config

Conversation

@ondrik

@ondrik ondrik commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

The SMT solvers' paths and command-line arguments were hard-coded in Synthesizer.add_solvers, and the repository shipped about 100 MB of solver binaries in solvers/. Now:

  • the solvers are configured in solvers.toml;
  • ./configure-solvers writes that file for the solvers installed on the computer;
  • ./configure-solvers --download first downloads some of them.

Neither the binaries nor solvers.toml are tracked any more.

Solver configuration (solvers.toml)

[solvers.cvc5]
path = "/home/me/cvc5/bin/cvc5"       # a relative path is relative to solvers.toml, a bare name is looked up on PATH
args = ["--produce-models"]           # always passed
incremental_args = ["--incremental"]  # added in the incremental modes (formula sent through stdin)
logics = ["QF_LIA", "QF_NIA", "QF_NRA"]
# portfolio = false                   # keep -a portfolio from running it
  • Loading: src/solver_config.py loads and validates the file. Unknown keys, wrong types, unknown logics, a missing file and invalid TOML each give a SolverConfigError that names the file and table. If the default file is missing, the error tells you to run configure-solvers.
  • Which file is used: --solver-config FILE (CLI) or solver_config= (Python) wins over the MILQ_SOLVER_CONFIG environment variable, which wins over solvers.toml in the repository root.
  • -a NAME accepts any configured solver, so adding a solver needs no code change. portfolio, gurobi and cbc are reserved names.
  • Clearer errors: each of these now gives milq: error: ... and exit code 2:
    • an unknown solver;
    • a solver that doesn't support the formula's logic;
    • a portfolio with no solver for the logic.
  • pySMT:
    • A second pySMT synthesis in the same process reuses the registered solver. If its command or logics differ, it raises a SolverConfigError instead.
    • The pySMT portfolio runs only the configured solvers from its list. Before, a solver that wasn't installed made it crash with "Cannot find solvers".
  • dReal special case removed: selecting dreal used to switch FiveTuple formulas to QF_NRA, which made the encoding fail with "rescaling with wrong logic". Now -a dreal -c FiveTuple reports that dReal does not support QF_LIA.

./configure-solvers

  • Search: it looks for yices-smt2/yices_smt2, cvc5, opensmt, smtinterpol, z3 and dreal, in this order of places:
    1. the -s DIR directories;
    2. the downloaded solvers;
    3. PATH;
    4. for dReal, also /opt/dreal/*/bin.
  • Check: it keeps a solver only if it answers sat to a trivial formula of its first logic. For example, SMTInterpol without Java is reported and left out.
  • Arguments and logics are those of the old solvers.toml. SMTInterpol has no QF_NRA, as decided in Don't register SMTInterpol for QF_NRA #100.
  • Options:
    • -o FILE writes another file, and -o - prints to standard output;
    • an existing file is kept unless --force is given.
  • Downloads: --download [yices2,cvc5,opensmt] first downloads release binaries from GitHub into solvers/ (gitignored):
    • versions: yices2 2.7.0, cvc5 1.3.1 and OpenSMT 2.9.2;
    • each archive is checked against the sha256 GitHub publishes for it, and its executable is linked into solvers/bin;
    • platforms: Linux x86-64 and macOS; on Linux arm64 only cvc5 is available;
    • a solver that is already downloaded is kept;
    • SMTInterpol and dReal have no release binaries to download.

Tests and CI

  • Tests: the tests generate their own configuration the same way, from the solvers that are installed and downloaded. They pass it through MILQ_SOLVER_CONFIG, so they don't depend on the user's solvers.toml.
  • New test files:
    • tests/test_solver_config.py covers loading, file precedence, validation, custom solvers, the portfolio, pySMT and the CLI;
    • tests/test_solver_detection.py and tests/test_solver_download.py cover detection and downloads. They use fake solvers and local file:// archives, so they need no network.
  • CI runs ./configure-solvers --download -o - before the tests, so it runs them with yices2, cvc5, OpenSMT and z3.

Docs

docs/installation.md has a new solver table with install links and a "Solver configuration" section covering configure-solvers and --download. The README, docs/reference.md and docs/index.md are updated too.

🤖 Generated with Claude Code

https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o

@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.66667% with 9 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/solver_download.py 89.9% 5 Missing and 2 partials ⚠️
src/solver_detection.py 98.1% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@ondrik
ondrik marked this pull request as draft October 12, 2026 20:41
@jkhav

jkhav commented Oct 12, 2026

Copy link
Copy Markdown
Member

I was also thinking about swapping everything to configs - we can even extend these configs to full synthesis configs (all milq.py arguments prepared in .toml), gate-set defining + presets, setting custom portfolio of solvers. One open question is whether it would be useful to somehow nest configs -- e.g. portfolio.toml denotes some solvers of the portfolio manually, and some other solvers are added by referring to another config (this would also be useful for the full synthesis configs, which can refer to a gate-set config, solver config etc.)

@ondrik

ondrik commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

I was also thinking about swapping everything to configs - we can even extend these configs to full synthesis configs (all milq.py arguments prepared in .toml), gate-set defining + presets, setting custom portfolio of solvers. One open question is whether it would be useful to somehow nest configs -- e.g. portfolio.toml denotes some solvers of the portfolio manually, and some other solvers are added by referring to another config (this would also be useful for the full synthesis configs, which can refer to a gate-set config, solver config etc.)

good point, I'm still vibe-coding the basic functionality, we can build on top of that - changed to draft PR for now

ondrik and others added 6 commits October 27, 2026 12:03
The paths and command-line arguments of the SMT solvers were hard-coded in
Synthesizer.add_solvers. They now come from solvers.toml in the repository
root (or --solver-config FILE / $MILQ_SOLVER_CONFIG), which also lists the
logics each solver supports and whether the portfolio runs it. A solver
added to the file can be selected with -a NAME.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o
pySMT keeps generic solvers registered for the whole process, so a second
pySMT synthesis in one process failed with SolverRedefinitionError. An
identical registration is now reused, and one with another command or
logics gives a SolverConfigError.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o
Selecting dreal switched the formula to QF_NRA. That only mattered for
FiveTuple (Classic is QF_NRA already), where the encoding then failed
with "rescaling with wrong logic". solvers.toml already lists QF_NRA as
dReal's only logic, so -a dreal -c FiveTuple now reports that dReal does
not support QF_LIA.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o
On Python 3.14, multiprocessing starts processes with forkserver by
default, and pySMT's portfolio processes then lack the generic solvers
registered from solvers.toml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o
The repository no longer ships SMT solver binaries (solvers/, about
100 MB of yices2, cvc5, OpenSMT, SMTInterpol and z3alpha builds from
SMT-COMP), and solvers.toml is no longer tracked. ./configure-solvers
writes it instead: it looks for yices-smt2 (or yices_smt2), cvc5,
opensmt, smtinterpol, z3 and dreal on PATH (dReal also in /opt/dreal),
in directories given with -s, and keeps each one that answers sat to
a trivial formula of its first logic, so e.g. SMTInterpol without Java
is left out. The arguments and logics of each solver are those of the
old solvers.toml. Without the file, milq names the script.

The pySMT portfolio ran a hard-coded list of solvers; it now runs only
those that are configured (z3 is pySMT's own).

The tests generate their own configuration the same way and pass it
through MILQ_SOLVER_CONFIG, so they neither read nor change the user's
solvers.toml. CI installs yices2 2.6.5 and cvc5 1.3.1 from their
releases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o
--download downloads the release binaries of yices2 2.7.0, cvc5 1.3.1
and OpenSMT 2.9.2 for this platform from GitHub into solvers/ (not
tracked), checks them against the sha256 GitHub reports, and links the
executables into solvers/bin, where configure-solvers looks (after the
--search directories, before PATH). --download yices2,cvc5 downloads
only those, --download-dir keeps them elsewhere, and a solver that is
already downloaded is kept. SMTInterpol and dReal have no release
binaries to download.

CI uses it instead of fetching yices2 and cvc5 itself, so the tests run
with OpenSMT too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o
@ondrik
ondrik force-pushed the feat/solver-config branch from b10cd24 to 441add4 Compare October 27, 2026 11:23
@ondrik ondrik changed the title Configure SMT solvers in solvers.toml Configure SMT solvers in solvers.toml, generated by configure-solvers Oct 27, 2026
ondrik and others added 2 commits October 27, 2026 13:27
With yices2 2.7.0, which configure-solvers --download now gets, the
test hung CI: in the incremental search, yices2 2.7.0 does not answer
the depth-1 query of the GHZ2 fidelity encoding within minutes. 2.6.5
answers it in about a second, but only after the depth-0 query; on the
depth-1 query alone it does not answer within a minute either, so it
was luck. z3 and cvc5 take well under a second on the same session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o
The download URLs, versions and sha256 sums were a separate table in
solver_download.py, keyed by platform. They are now part of each
KnownSolver in solver_detection.py (releases, by platform), next to the
executable names, arguments and logics, so everything about a solver is
in one place. solver_download.py keeps only the downloading; a download
links the solver's first executable name. The releases are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0177AhE9SXm3iJ1KP8GTSk6o
@ondrik
ondrik marked this pull request as ready for review October 27, 2026 12:55
@ondrik
ondrik requested a review from jkhav October 27, 2026 12:55
@jkhav
jkhav merged commit b0b623c into master Oct 27, 2026
4 checks passed
@ondrik
ondrik deleted the feat/solver-config branch October 27, 2026 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants