Challenge "aido1_LF1-v3"

These are the metrics defined:

reward - reward

average reward over the episodes

Evaluation steps details

• At the beginning execute step step1-simulation.

• If step step1-simulation has result success, then execute step step2-scoring.

• If step step1-simulation has result failed, then declare the submission FAILED.

• If step step1-simulation has result error, then declare the submission ERROR.

• If step step2-scoring has result success, then declare the submission SUCCESS.

• If step step2-scoring has result failed, then declare the submission FAILED.

• If step step2-scoring has result error, then declare the submission ERROR.

Evaluation step step1-simulation

Timeout None

This is the Docker Compose configuration skeleton:

version: '3'
services:
    solution:
        image: SUBMISSION_CONTAINER
        environment: {}
    evaluator:
        image: andreacensi/aido1_lf1-v3-evaluator:2018_10_12_12_50_03
        environment:
            DTG_MAP: loop_obstacles
            DTG_EPISODES: '5'
            DTG_CHALLENGE: LF
            DTG_DOMAIN_RAND: '0'
            DTG_ENVIRONMENT: Duckietown-Lf-Lfv-Navv-Silent-v0
            DTG_CAMERA_WIDTH: '160'
            DTG_CAMERA_HEIGHT: '120'
            DTG_STEPS_PER_EPISODE: '500'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Evaluation step step2-scoring

Timeout None

This is the Docker Compose configuration skeleton:

version: '3'
services:
    scorer:
        image: andreacensi/aido1_lf1-v3-evaluator:2018_10_11_23_14_51
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step