A simple evaluator for evaluating a submission on a given task.
Refer to the examples folder for package examples, such as examples/task and examples/submission. Ensure that you zip the contents of these folders without including the folders themselves.
Once you have the packages, follow these steps:
- Install the evaluator, task, and submission packages:
pip install aicon-evaluator.zip task.zip submission.zip
aicon-evaluator.zip is a zip file containing the contents of this folder (not including examples folder).
- Run the evaluation:
aicon_evaluator