AI Driving Olympics Home Challenges Submissions

Challenge "aido5-LFV_multi-real-validation"

Challenge description

(No description.)

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance - driven_lanedir_consec

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time - survival_time

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Lateral deviation - deviation-center-line

This is the median lateral deviation from the center line.

Major infractions - in-drivable-lane

This is the median of the time spent outside of the drivable zones. For example this penalizes driving in the wrong lane.

Dependencies

Dependencies

No dependencies

Details

Technical details

Evaluation steps details

Evaluation step eval0

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval0-evaluator:2020_12_12_18_23_48@sha256:6ccc04e1d810494bcf469cf3b6bb09795bc9fe88f65b1c4f71e06b73d1664bf6
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots3
AIDO 5 Map large loop1

Evaluation step eval1

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval1-evaluator:2020_12_12_18_24_04@sha256:57b67e9a13e9695d3c22667fd1944ce9c4dbfca5b63bdb04c96f56c17c223a72
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots3
AIDO 5 Map large loop1

Evaluation step eval2

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval2-evaluator:2020_12_12_18_24_18@sha256:39ade22960d544aa290caf3f1e305d4f7466239cdf69748288d13e87a59c7b46
        environment: {}

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots3
AIDO 5 Map large loop1

Evaluation step eval0-videos

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval0-videos-evaluator:2020_12_12_18_27_08@sha256:dd96cbeea5a14800e13efdd99fa7c7a4141fecc458bff17f2e7b5fe27b91c679
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: robot,watchtower
            OUTPUT_FRAMERATE: '30'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Cloud simulations1

Evaluation step eval1-videos

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval1-videos-evaluator:2020_12_12_18_27_27@sha256:dd96cbeea5a14800e13efdd99fa7c7a4141fecc458bff17f2e7b5fe27b91c679
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval1
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: robot,watchtower
            OUTPUT_FRAMERATE: '30'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Cloud simulations1

Evaluation step eval2-videos

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval2-videos-evaluator:2020_12_12_18_27_43@sha256:dd96cbeea5a14800e13efdd99fa7c7a4141fecc458bff17f2e7b5fe27b91c679
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval2
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: robot,watchtower
            OUTPUT_FRAMERATE: '30'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Cloud simulations1

Evaluation step eval0-visualize

Timeout 1080.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval0-visualize-evaluator:2020_12_12_18_24_33@sha256:2a8cf3123aae7afca08802ab30696892b99560d4a2842be7de37ca74b529528f
        environment:
            STEP_NAME: eval0

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Cloud simulations1

Evaluation step eval1-visualize

Timeout 1080.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval1-visualize-evaluator:2020_12_12_18_26_40@sha256:2a8cf3123aae7afca08802ab30696892b99560d4a2842be7de37ca74b529528f
        environment:
            STEP_NAME: eval1

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Cloud simulations1

Evaluation step eval2-visualize

Timeout 1080.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido5-lfv_multi-real-validation-eval2-visualize-evaluator:2020_12_12_18_26_54@sha256:2a8cf3123aae7afca08802ab30696892b99560d4a2842be7de37ca74b529528f
        environment:
            STEP_NAME: eval2

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Cloud simulations1