AI Driving Olympics Home Challenges Submissions Jobs

Challenge "[old] Lane following (rev 2)"

Challenge description

This challenge is described in the detail in the book here

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

traveled_tiles_median

Median number of tiles traveled.

survival_time_mean

Average survival time in seconds.

good_angle_mean

Integral of the lane angle absolute value.

valid_direction_mean

Checking that the angle was always less than 60.

Details

Technical details

Evaluation steps details

  • At the beginning execute step step1-simulation.

  • If step step1-simulation finishes with status success, then execute step step2-scoring.

  • If step step1-simulation finishes with status failed, then declare the submission FAILED.

  • If step step1-simulation finishes with status error, then declare the submission ERROR.

  • If step step2-scoring finishes with status success, then declare the submission SUCCESS.

  • If step step2-scoring finishes with status failed, then declare the submission FAILED.

  • If step step2-scoring finishes with status error, then declare the submission ERROR.

Evaluation step step1-simulation

Timeout None

This is the Docker Compose configuration skeleton:

version: '3'
services:
    solution:
        image: SUBMISSION_CONTAINER
        environment: {}
    evaluator:
        image: andreacensi/aido1_lf1_r2-v3-evaluator:2018_10_15_20_46_37@sha256:4edc9356c9e16600445e6c6960514f727cb2d10cca79c3fccb47c5da6ec5d64a
        environment:
            launcher_parameters: "agent-info: {challenge: LF}\nchallenge: LF\nenvironment:\
                \ Duckietown-Lf-Lfv-Navv-Silent-v0\nenvironment-constructor: Simulator\n\
                environment-parameters: {accept_start_angle_deg: 4, camera_height:\
                \ 120, camera_width: 160,\n  domain_rand: 0, full_transparency: true,\
                \ map_name: loop_pedestrians, max_steps: 500,\n  seed: 123}\nepisodes:\
                \ 5\nsteps_per_episode: 500\n"
            solution_parameters: '{env: Duckietown-Lf-Lfv-Navv-Silent-v0}

                '

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

RAM free (MB)80
Disk available (MB)100

Evaluation step step2-scoring

Timeout None

This is the Docker Compose configuration skeleton:

version: '3'
services:
    scorer:
        image: andreacensi/aido1_lf1_r2-v3-scorer:2018_10_15_21_15_14@sha256:79cfc9100ecd84ded5270a12ac5b8ea93de6e16a931cb6109a3d869d92f125e1
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

RAM free (MB)80
Disk available (MB)100