AI Driving Olympics Home Challenges Submissions Jobs

Challenge "LFVI 🚗🚗🚦 - Lane following + Vehicles + Intersections (robotarium 🏎, validation 🏋)"

Challenge description

(No description.)

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Details

Technical details

Evaluation steps details

  • At the beginning execute step eval0.

  • If step eval0 finishes with status success, then execute step eval0-videos-autobots.

  • If step eval0 finishes with status failed, then declare the submission FAILED.

  • If step eval0 finishes with status error, then declare the submission ERROR.

  • If step eval0-videos-autobots finishes with status success, then execute step eval0-videos-watchtowers.

  • If step eval0-videos-autobots finishes with status failed, then declare the submission FAILED.

  • If step eval0-videos-autobots finishes with status error, then declare the submission ERROR.

  • If step eval0-videos-watchtowers finishes with status success, then execute step eval1.

  • If step eval0-videos-watchtowers finishes with status failed, then declare the submission FAILED.

  • If step eval0-videos-watchtowers finishes with status error, then declare the submission ERROR.

  • If step eval1 finishes with status success, then execute step eval1-videos-autobots.

  • If step eval1 finishes with status failed, then declare the submission FAILED.

  • If step eval1 finishes with status error, then declare the submission ERROR.

  • If step eval1-videos-autobots finishes with status success, then execute step eval1-videos-watchtowers.

  • If step eval1-videos-autobots finishes with status failed, then declare the submission FAILED.

  • If step eval1-videos-autobots finishes with status error, then declare the submission ERROR.

  • If step eval1-videos-watchtowers finishes with status success, then declare the submission SUCCESS.

  • If step eval1-videos-watchtowers finishes with status failed, then declare the submission FAILED.

  • If step eval1-videos-watchtowers finishes with status error, then declare the submission ERROR.

Evaluation step hw-evaluation

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfvi-real-validation-hw-evaluation-evaluator:2019_05_13_17_49_23@sha256:1168d196612a40db2eedce3f412838293600898ed3f6a96dccc2679c82acee56
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots2
AIDO 2 Map LFVI public1

Evaluation step eval0

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lfvi-real-validation-eval0-evaluator:2019_05_21_17_31_48@sha256:fc01933476dfd30edd7a0218c8626286dfd4132494502a2020f4843f66390c55
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots2
AIDO 2 Map LFVI public1

Evaluation step eval0-videos-0

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfvi-real-validation-eval0-videos-0-evaluator:2019_05_16_14_00_25@sha256:45494b8ddc5c6d02c900845a72601852f9b63253b18d19108d7f5ee51ebbeaa5
        environment:
            WORKER_I: '0'
            WORKER_N: '2'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-1

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfvi-real-validation-eval0-videos-1-evaluator:2019_05_16_14_00_44@sha256:45494b8ddc5c6d02c900845a72601852f9b63253b18d19108d7f5ee51ebbeaa5
        environment:
            WORKER_I: '1'
            WORKER_N: '2'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-extractATs

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfvi-real-validation-eval0-extractats-evaluator:2019_05_17_14_50_17@sha256:a12b33e0fb36d6abb88b542b1d0a751467ff24824ac209b75d09db985d931d64
        environment:
            BAGS_PATH: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            ACQ_OUTPUT_BAG: /challenges/challenge-evaluation-output/processed.bag
            ACQ_TEST_STREAM: '0'
            ACQ_DEVICE_THREADS: '6'
            ACQ_OBSERVATIONS_STATISTICS_OUTPUT: /challenges/challenge-evaluation-output/statistics.yaml

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-WesQuackerson

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lfvi-real-validation-eval0-wesquackerson-evaluator:2019_05_17_14_50_48@sha256:285263e1b8a043e4ef9f7dc25b3d5796c6a0bb09a4d629bd8bba5ec1b07159e5
        environment:
            ATMSGS_BAG: /challenges/previous-steps/eval0-extractATs/challenge-evaluation-output/processed.bag
            TITLE_CARD: '1'
            VIDEO_BAGS: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_FILE: /challenges/challenge-evaluation-output/overheadVideo.mp4
            POSES_TOPIC: /poses_acquisition/poses
            VIDEO_TOPIC: /camera_node/image/compressed
            MIN_SHOT_LENGTH: '12'
            OUTPUT_FRAMERATE: '12'
            PREVIOUS_RESULTS: /challenges/previous-steps/eval0/challenge-evaluation-output/challenge-results/challenge_results.yaml
            CAMERA_RESOLUTION_WIDTH: '972'
            CAMERA_RESOLUTION_HEIGHT: '1296'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido2-lfvi-real-validation-eval0-videos-evaluator:2019_05_17_11_05_53@sha256:ff28da3480a099039a1abcbd1e04ec22c5b2dc336160e47842400edcc968fae0
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-autobots

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lfvi-real-validation-eval0-videos-autobots-evaluator:2019_05_20_11_21_04@sha256:d9aa83c8a6ece96a85d38fa99609ac9b0cee0686edcf9eaaca2a8a14175715c0
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: autobot
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-watchtowers

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lfvi-real-validation-eval0-videos-watchtowers-evaluator:2019_05_20_11_21_23@sha256:d9aa83c8a6ece96a85d38fa99609ac9b0cee0686edcf9eaaca2a8a14175715c0
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '0'
            BAG_NAME_FILTER: watchtower
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval1

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lfvi-real-validation-eval1-evaluator:2019_05_21_17_32_00@sha256:fc01933476dfd30edd7a0218c8626286dfd4132494502a2020f4843f66390c55
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots2
AIDO 2 Map LFVI public1

Evaluation step eval1-videos-autobots

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lfvi-real-validation-eval1-videos-autobots-evaluator:2019_05_20_11_21_41@sha256:d9aa83c8a6ece96a85d38fa99609ac9b0cee0686edcf9eaaca2a8a14175715c0
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: autobot
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval1-videos-watchtowers

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lfvi-real-validation-eval1-videos-watchtowers-evaluator:2019_05_20_11_21_59@sha256:d9aa83c8a6ece96a85d38fa99609ac9b0cee0686edcf9eaaca2a8a14175715c0
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '0'
            BAG_NAME_FILTER: watchtower
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1