Skip to content

INTERPROSCAN fails with real database: version mismatch and nested data directory #114

Closed

Description

@vagkaratzas

Description of the bug

With the default InterProScan database settings, INTERPROSCAN fails on a real (non-test) run:

Running hmmpress (/usr/local/bin//hmmpress) on data/pirsf/3.10/sf_hmm_all
failed to open SSI index data/pirsf/3.10/sf_hmm_all.h3i
Could not run hmmpress!
...
FATAL - Analysis step 27 : Run pirsf.pl binary for selected proteins for proteins 1 to 405 has failed irretrievably.

CI doesn't catch it because the test profiles use the small interproscan_test.tar.gz.

Causes

1. Container and database versions don't match (confirmed)

  • modules/nf-core/interproscan/main.nf uses the interproscan:5.59_91.0 container.
  • nextflow.config sets interproscan_db_url to the 5.72-103.0 release.

InterProScan reads member database paths from its bundled interproscan.properties. The 5.59 container therefore looks for data/pirsf/3.10, data/panther/17.0, data/tigrfam/15.0 and so on, none of which exist in the 5.72 data. TIGRFAM was also merged into NCBIfam after 5.59, so the default interproscan_applications depends on the version.

2. The staged database directory is probably nested one level too deep (likely, not yet confirmed)

The full EBI release tarball contains a single top-level folder interproscan-X/ holding bin/, data/, interproscan.sh and so on. UNTAR strips that folder, and the module then stages the whole install directory as data (stageAs: 'data'). The HMMs end up at data/data/pirsf/..., while InterProScan looks for data/pirsf/.... The test tarball has no wrapper folder, which would explain why tests pass.

The same failure happens after switching the URL to the matching 5.59 release.

3. The HMMs aren't pre-pressed

Release tarballs need python3 setup.py -f interproscan.properties to be run once. Without it, pirsf.pl runs hmmpress inside the shared UNTAR output while the pipeline runs. With several input FASTAs, parallel INTERPROSCAN tasks can press the same files at the same time and race.

Suggested fix

  • Keep the container and interproscan_db_url on the same version. Either bump both to 5.72 once the container is available (see Bump interproscan in bioconda #20) and replace TIGRFAM with NCBIfam, or point the URL at the 5.59-91.0 release.
  • In subworkflows/local/functional_annotation/main.nf, pass the inner data/ folder when it exists, for both the download branch and the --interproscan_db branch:
    .map { _meta, database -> database.resolve('data').exists() ? database.resolve('data') : database }
  • Press the HMMs once after UNTAR, with a small local step running setup.py, or document that --interproscan_db must point to a database that has already been set up.

Command used and terminal output

Slurm HPC, Singularity, default interproscan_db_url and interproscan_applications (Hamap,PANTHER,PIRSF,TIGRFAM,sfld), --disable-precalc.

System information

  • Pipeline: dev (ffb0473)
  • Executor: Slurm
  • Container engine: Singularity
  • InterProScan container: 5.59-91.0

Activity

  1. vagkaratzas commented on Sep 22, 2026

    @vagkaratzas
    CollaboratorAuthor

    Cause 2 is confirmed. The UNTAR_INTERPROSCAN output for interproscan-5.59-91.0-64-bit.tar.gz is the full install:

    interproscan_db/
    ├── bin/  data/  lib/  src/  work/
    ├── interproscan-5.jar  interproscan.properties  interproscan.sh  setup.py
    └── test_*.fasta / test_*.xml
    

    Staged as data, this puts the member databases at data/data/....

    Workaround until this is fixed: point --interproscan_db at the inner folder, <UNTAR_workdir>/interproscan_db/data. Before that, run python3 setup.py -f interproscan.properties once in interproscan_db/ to press the HMMs.

  2. vagkaratzas commented on Sep 22, 2026

    @vagkaratzas
    CollaboratorAuthor

    Correction: the nested data/data/ layout is not what caused this failure. The root cause is in the interproscan module itself, reported upstream as nf-core/modules#13009.

    interproscan.sh changes into its install directory before it reads INTERPROSCAN_CONF. The module sets that variable to a relative path, so InterProScan loads the container's own interproscan.properties (data.directory=data, relative to the install dir) and uses the small, unpressed, read-only sample database in the container. Whatever is passed to --interproscan_db is ignored, which is why the error stayed exactly the same after switching the URL and the path.

    Once the module is fixed, causes 1 (version mismatch) and 2 (pass the inner data/ folder) still need addressing here.

    Workaround for now: bind a pressed 5.59-91.0 data/ folder over the container's copy:

    process {
        withName: '.*:INTERPROSCAN' {
            containerOptions = '-B /abs/path/interproscan_db/data:/usr/local/share/InterProScan/data'
        }
    }
  3. vagkaratzas commented on Sep 27, 2026

    @vagkaratzas
    CollaboratorAuthor

    Fixed in #115 (now in dev):

    • The interproscan module now uses the staged database (interproscan: staged database is ignored (relative INTERPROSCAN_CONF, wrong bin/data directories) modules#13009), which was the root cause.
    • The default interproscan_db_url now points to the 5.59-91.0 release, which matches the container (cause 1).
    • The inner data/ folder of an untarred release is passed to INTERPROSCAN (cause 2).
    • PIRSF has been removed from the default interproscan_applications, so a default run no longer races on hmmpress (cause 3). To use PIRSF, pre-press the database as described in the usage docs and pass it with --interproscan_db.

    A one-off pressing step after UNTAR, which would let PIRSF go back into the defaults, can be looked at separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions