Fix the CI conda cache, the Snakefile f-strings, and blastp calls that fail but exit zero

[mrubash1]

Four small, independent fixes. This replaces #101, which was cut down after #103 landed: the analysis.yml matplotlib and setuptools pins are dropped because #103 took them verbatim, and the python=3.9.16 pins are dropped for the reason below.

The conda env cache restore-keys never matched

The key begins with conda- while the restore-keys begin with snakemake-conda-, so they never matched anything.

Correcting the prefix would be worse than removing them. A restore-keys match still leaves cache-hit false, so the env creation step runs regardless — and that step begins by deleting .snakemake/conda. Making the prefix match would therefore download a stale multi-gigabyte cache only to discard it: strictly slower than the broken prefix that never matched at all.

blastp calls that fail but exit zero

When the remote server refuses to queue a request, blastp -remote reports the error on stderr, writes an empty results file, and still exits zero:

$ blastp -remote -db nr ... ; echo $?
Error: [blastp] bad_request: Could not queue request: DB operation failed.
0

_run_blast_remote printed that stderr but returned the zero exit code unchanged, so the caller treated the failure as a success. The word-size backoff was therefore never attempted — which is the situation it exists for — and the empty results file surfaced later in extract_blast_hits.py as a misleading "no hits were returned".

Failure is now determined by blast_call_failed, which also treats an error on stderr as a failure. An empty results file is deliberately not treated as a failure, because a query that legitimately has no hits produces one too.

This affects the remote backend only; the www backend does not shell out to blastp.

Build the wildcard output paths without f-strings

Nested braces inside an f-string (f"{ANALYSIS_NAME}_{{plotting_mode}}.html") tokenise differently under PEP 701 on Python 3.12+, and snakefmt then rewrites the Snakefile incorrectly. Building the paths without f-strings avoids depending on the interpreter version that happens to run the linter.

On the python pins, and the mamba pin, that are not here

The earlier version of this branch also added python=3.9.16 to analysis.yml, plotting.yml and pandas.yml, and pinned mamba=1.4.2 in cartography_test.yml. Both are dropped, and the reason is worth recording because the first diagnosis was wrong.

The python pins were initially suspected of causing a large regression in env creation time. They were not the cause. The mamba pin was. Pinning mamba=1.4.2 pulls conda 23.3.1 and libmamba 1.4.2 into the outer test environment, and the workflow then runs snakemake --conda-frontend conda against that older conda and its classic solver. Measured on CI:

	main	this branch with the mamba pin
mamba	unpinned, resolves to 2.9.0	1.4.2
analysis.yml creation	35s	over 15 minutes, then the runner was reclaimed
whole env creation step	5m10s, succeeded	failed

analysis.yml is byte-identical between the two, so the ~25x difference is entirely the solver that the outer environment supplies. The same pin was present on the earlier #101 and #102 branches and explains their env-creation failures too, which had been misattributed to the python pins.

The mamba pin was also fixing a path CI does not use: it exists to make --conda-frontend mamba work, and this workflow deliberately uses --conda-frontend conda. If that is worth solving it wants a different fix -- pinning conda rather than mamba -- and its own PR, with the timing above as the evidence.

The python pins are dropped on their own merits rather than as suspects: CI is green on main without them, since #103's targeted matplotlib and setuptools pins address the actual import failure. If they are still wanted for drift prevention, that is a separate PR.

With no envs/ change left in this branch, the conda cache key now matches main, so this PR restores the cache hit instead of forcing a full env creation.

Testing

ruff and snakefmt clean under CI's pinned versions on Python 3.9; unit tests pass, including four new ones covering the exit-zero case.

[ahmedhosny]

Looks good. Three independent, well-justified fixes:

1. Dropping the conda restore-keys is correct: they never matched (snakemake-conda- vs conda-), and a working prefix would still download a stale cache only for the next step to delete .snakemake/conda.
2. blast_call_failed treats blastp -remote stderr Error: as a failure even when the process exits 0, which is what unblocks the word-size backoff. An empty results file is correctly not treated as failure. This only affects the remote backend.
3. Building wildcard output paths without f-strings avoids PEP 701 rewriting the doubled braces under Python 3.12+.

I merged main into this work (conflicts with #107 in blast_utils.py / tests: kept both sides) and opened a follow-up from the resolved branch so it can land on current main.

[ahmedhosny]

Thanks — this LGTM.

After merging #107, this branch conflicted in blast_utils.py and tests/test_blast_utils.py (both sides add after the same spots). I merged main in and kept both: the www timeout/error-page work from #107 and the blast_call_failed / f-string / cache-key fixes from here.

Landing the resolved branch as #109 so it can merge onto current main. Will close this once that is in.

[ahmedhosny]

Landed on main via #109 (this branch conflicted with #107 after that merge; the follow-up kept both sides). Closing this as superseded.