Challenge "aido1_test_multistep-v3"

These are the metrics defined:

passed-step1 - passed-step1

1 if the submission passed the first step.

passed-step2 - passed-step2

1 if the submission passed the second step.

Evaluation steps details

• At the beginning execute step step1.

• If step step1 has result success, then execute step step2.

• If step step1 has result failed, then declare the submission FAILED.

• If step step1 has result error, then declare the submission ERROR.

• If step step2 has result success, then declare the submission SUCCESS.

• If step step2 has result failed, then declare the submission FAILED.

• If step step2 has result error, then declare the submission ERROR.

Evaluation step step2

Timeout None

This is the Docker Compose configuration skeleton:

version: '3'
services:
    solution:
        image: SUBMISSION_CONTAINER
        environment: {}
    step2-evaluator:
        image: andreacensi/aido1_test_multistep-v3-step2-evaluator:2018_10_24_13_23_18
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

No particular resources required.

Evaluation step step1

Timeout None

This is the Docker Compose configuration skeleton:

version: '3'
services:
    solution:
        image: SUBMISSION_CONTAINER
        environment: {}
    step1-evaluator:
        image: andreacensi/aido1_test_multistep-v3-step1-evaluator:2018_10_24_13_23_24
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

No particular resources required.