Skip to content
This repository was archived by the owner on Oct 5, 2026. It is now read-only.
This repository was archived by the owner on Oct 5, 2026. It is now read-only.

Multiple model parameter with same value result in multiple evaluations #220

Description

@Munsio

When running the evaluation and specifying the same model multiple times currently the evaluation is run X amount of times for the model.

Example:

eval-dev-quality evaluate --runtime docker --result-path ./docker-test --runs 5 --model symflower/symbolic-execution --model symflower/symbolic-execution --model symflower/symbolic-execution --repository golang/plain

This runs the symflower/symbolic-execution 3 times with 5 runs as the model was 3 times specified as parameter.

Question:
Do we want this behavior or should we unique the list of models after parsing?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingquestionFurther information is requested

Type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions