AI Driving Olympics Home Challenges Submissions Jobs

Challenge "LFV 🚗🚗 - Lane following + Vehicles (robotarium 🏎, validation 🏋)"

Challenge description

(No description.)

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Details

Technical details

Evaluation steps details

  • At the beginning execute step eval0.

  • If step eval0 finishes with status success, then execute step eval0-videos-autobots.

  • If step eval0 finishes with status failed, then declare the submission FAILED.

  • If step eval0 finishes with status error, then declare the submission ERROR.

  • If step eval0-videos-autobots finishes with status success, then execute step eval0-videos-watchtowers.

  • If step eval0-videos-autobots finishes with status failed, then execute step eval0-extractATs.

  • If step eval0-videos-autobots finishes with status error, then execute step eval0-extractATs.

  • If step eval0-videos-watchtowers finishes with status success, then execute step eval0-extractATs.

  • If step eval0-videos-watchtowers finishes with status failed, then execute step eval0-extractATs.

  • If step eval0-videos-watchtowers finishes with status error, then execute step eval0-extractATs.

  • If step eval0-extractATs finishes with status success, then execute step eval0-WesQuackerson.

  • If step eval0-extractATs finishes with status failed, then declare the submission FAILED.

  • If step eval0-extractATs finishes with status error, then declare the submission ERROR.

  • If step eval0-WesQuackerson finishes with status success, then execute step eval0-localization.

  • If step eval0-WesQuackerson finishes with status failed, then execute step eval0-localization.

  • If step eval0-WesQuackerson finishes with status error, then execute step eval0-localization.

  • If step eval0-localization finishes with status success, then declare the submission SUCCESS.

  • If step eval0-localization finishes with status failed, then declare the submission FAILED.

  • If step eval0-localization finishes with status error, then declare the submission ERROR.

Evaluation step eval0

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-offseason-eval0-evaluator:2019_06_19_16_33_33@sha256:1cc1f34ccf9060b7ef286d8dfcf0e1815d5a220d5850a342090537ed313f7ea3
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots2
AIDO 2 Map LFV public1

Evaluation step eval1

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-offseason-eval1-evaluator:2019_06_19_16_33_46@sha256:1cc1f34ccf9060b7ef286d8dfcf0e1815d5a220d5850a342090537ed313f7ea3
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots2
AIDO 2 Map LFV public1

Evaluation step eval0-videos-autobots

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-offseason-eval0-videos-autobots-evaluator:2019_06_19_16_33_58@sha256:a7425ff06a130654f37ddf2d5ad75760d65fc29c60bab935e96a88db7413c7f8
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: autobot
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-watchtowers

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-offseason-eval0-videos-watchtowers-evaluator:2019_06_19_16_34_15@sha256:a7425ff06a130654f37ddf2d5ad75760d65fc29c60bab935e96a88db7413c7f8
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '0'
            BAG_NAME_FILTER: watchtower
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-extractATs

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-offseason-eval0-extractats-evaluator:2019_06_19_16_34_39@sha256:e8754d9dda791175a48761aed021863c57c3fdd503e06bbf7ad5da3edcae8140
        environment:
            BAGS_PATH: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            ACQ_OUTPUT_BAG: /challenges/challenge-evaluation-output/processed.bag
            ACQ_TEST_STREAM: '0'
            ACQ_DEVICE_THREADS: '6'
            ACQ_OBSERVATIONS_STATISTICS_OUTPUT: /challenges/challenge-evaluation-output/statistics.yaml

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-WesQuackerson

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-offseason-eval0-wesquackerson-evaluator:2019_06_19_16_34_55@sha256:f08d21880f67096f78a3948136327d0067f5df3e037b15c391b51aa6f4827acb
        environment:
            ATMSGS_BAG: /challenges/previous-steps/eval0-extractATs/challenge-evaluation-output/processed.bag
            TITLE_CARD: '1'
            VIDEO_BAGS: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_FILE: /challenges/challenge-evaluation-output/overheadVideo.mp4
            POSES_TOPIC: /poses_acquisition/poses
            VIDEO_TOPIC: /camera_node/image/compressed
            MIN_SHOT_LENGTH: '12'
            OUTPUT_FRAMERATE: '12'
            PREVIOUS_RESULTS: /challenges/previous-steps/eval0/challenge-evaluation-output/challenge-results/challenge_results.yaml
            CAMERA_RESOLUTION_WIDTH: '972'
            CAMERA_RESOLUTION_HEIGHT: '1296'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-localization

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfv-real-validation-offseason-eval0-localization-evaluator:2019_06_19_16_35_09@sha256:ee4b5676af8bfd33fe2f3bfad1f866414295f5f03ffa9e468f54ac7eaa14bad1
        environment:
            ATMSGS_BAG: /challenges/previous-steps/eval0-extractATs/challenge-evaluation-output/processed.bag
            OUTPUT_DIR: /challenges/challenge-evaluation-output

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1