AI Driving Olympics Home Challenges Submissions Jobs

Challenge "LFV 🚗🚗 - Lane following + Vehicles (robotarium 🏎, validation 🏋)"

Challenge description

(No description.)

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Details

Technical details

Evaluation steps details

  • At the beginning execute step eval0.

  • If step eval0 finishes with status success, then execute step eval0-videos-autobots.

  • If step eval0 finishes with status failed, then declare the submission FAILED.

  • If step eval0 finishes with status error, then declare the submission ERROR.

  • If step eval0-videos-autobots finishes with status success, then execute step eval0-videos-watchtowers.

  • If step eval0-videos-autobots finishes with status failed, then declare the submission FAILED.

  • If step eval0-videos-autobots finishes with status error, then declare the submission ERROR.

  • If step eval0-videos-watchtowers finishes with status success, then execute step eval1.

  • If step eval0-videos-watchtowers finishes with status failed, then declare the submission FAILED.

  • If step eval0-videos-watchtowers finishes with status error, then declare the submission ERROR.

  • If step eval1 finishes with status success, then execute step eval1-videos-autobots.

  • If step eval1 finishes with status failed, then declare the submission FAILED.

  • If step eval1 finishes with status error, then declare the submission ERROR.

  • If step eval1-videos-autobots finishes with status success, then execute step eval1-videos-watchtowers.

  • If step eval1-videos-autobots finishes with status failed, then declare the submission FAILED.

  • If step eval1-videos-autobots finishes with status error, then declare the submission ERROR.

  • If step eval1-videos-watchtowers finishes with status success, then declare the submission SUCCESS.

  • If step eval1-videos-watchtowers finishes with status failed, then declare the submission FAILED.

  • If step eval1-videos-watchtowers finishes with status error, then declare the submission ERROR.

Evaluation step hw-evaluation

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-hw-evaluation-evaluator:2019_05_13_17_49_00@sha256:1168d196612a40db2eedce3f412838293600898ed3f6a96dccc2679c82acee56
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots2
AIDO 2 Map LFV public1

Evaluation step eval0

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval0-evaluator:2019_05_19_15_13_25@sha256:da1c347fa3078214042ddcf8a17b732225b632ca33bab1834f43cfe785153115
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots2
AIDO 2 Map LFV public1

Evaluation step eval0-videos-0

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-eval0-videos-0-evaluator:2019_05_16_13_58_37@sha256:45494b8ddc5c6d02c900845a72601852f9b63253b18d19108d7f5ee51ebbeaa5
        environment:
            WORKER_I: '0'
            WORKER_N: '2'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-1

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-eval0-videos-1-evaluator:2019_05_16_13_58_55@sha256:45494b8ddc5c6d02c900845a72601852f9b63253b18d19108d7f5ee51ebbeaa5
        environment:
            WORKER_I: '1'
            WORKER_N: '2'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-extractATs

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval0-extractats-evaluator:2019_05_17_14_48_31@sha256:a12b33e0fb36d6abb88b542b1d0a751467ff24824ac209b75d09db985d931d64
        environment:
            BAGS_PATH: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            ACQ_OUTPUT_BAG: /challenges/challenge-evaluation-output/processed.bag
            ACQ_TEST_STREAM: '0'
            ACQ_DEVICE_THREADS: '6'
            ACQ_OBSERVATIONS_STATISTICS_OUTPUT: /challenges/challenge-evaluation-output/statistics.yaml

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-WesQuackerson

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval0-wesquackerson-evaluator:2019_05_17_14_49_02@sha256:285263e1b8a043e4ef9f7dc25b3d5796c6a0bb09a4d629bd8bba5ec1b07159e5
        environment:
            ATMSGS_BAG: /challenges/previous-steps/eval0-extractATs/challenge-evaluation-output/processed.bag
            TITLE_CARD: '1'
            VIDEO_BAGS: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_FILE: /challenges/challenge-evaluation-output/overheadVideo.mp4
            POSES_TOPIC: /poses_acquisition/poses
            VIDEO_TOPIC: /camera_node/image/compressed
            MIN_SHOT_LENGTH: '12'
            OUTPUT_FRAMERATE: '12'
            PREVIOUS_RESULTS: /challenges/previous-steps/eval0/challenge-evaluation-output/challenge-results/challenge_results.yaml
            CAMERA_RESOLUTION_WIDTH: '972'
            CAMERA_RESOLUTION_HEIGHT: '1296'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos

Timeout 7200.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval0-videos-evaluator:2019_05_17_13_24_37@sha256:90e7c45aff3c2936cca59c2eb16668e882cc06a377ce9315900496a8efaaeb61
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-autobots

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval0-videos-autobots-evaluator:2019_05_19_15_13_30@sha256:34794ab6feac586ef8ad82b467cf7feaf4ee7286422eca5fdce3ed0a9ab00d97
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: autobot
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-watchtowers

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval0-videos-watchtowers-evaluator:2019_05_19_15_13_33@sha256:34794ab6feac586ef8ad82b467cf7feaf4ee7286422eca5fdce3ed0a9ab00d97
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '0'
            BAG_NAME_FILTER: watchtower
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval1

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval1-evaluator:2019_05_19_15_13_27@sha256:da1c347fa3078214042ddcf8a17b732225b632ca33bab1834f43cfe785153115
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots2
AIDO 2 Map LFV public1

Evaluation step eval1-videos-autobots

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval1-videos-autobots-evaluator:2019_05_19_15_13_37@sha256:34794ab6feac586ef8ad82b467cf7feaf4ee7286422eca5fdce3ed0a9ab00d97
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: autobot
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval1-videos-watchtowers

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfv-real-validation-eval1-videos-watchtowers-evaluator:2019_05_19_15_13_40@sha256:34794ab6feac586ef8ad82b467cf7feaf4ee7286422eca5fdce3ed0a9ab00d97
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '0'
            BAG_NAME_FILTER: watchtower
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1