Challenge "aido2-MS-regular"

These are the metrics defined:

passed-step1 - passed-step1

1 if the submission passed the first step.

passed-step2 - passed-step2

1 if the submission passed the second step.

Evaluation steps details

• At the beginning execute step step1.

• If step step1 has result success, then execute step step2.

• If step step1 has result failed, then declare the submission FAILED.

• If step step1 has result error, then declare the submission ERROR.

• If step step2 has result success, then declare the submission SUCCESS.

• If step step2 has result failed, then declare the submission FAILED.

• If step step2 has result error, then declare the submission ERROR.

Evaluation step step1

Timeout 100.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    solution:
        image: SUBMISSION_CONTAINER
        environment: {}
    step1-evaluator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:03c10b7317028189f323630dd52a226b72ab92c3a12fb265dbd3fdc21d539d97
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

No particular resources required.

Evaluation step step2

Timeout 100.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    step2-evaluator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:03c10b7317028189f323630dd52a226b72ab92c3a12fb265dbd3fdc21d539d97
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

No particular resources required.