Duckietown Challenges Home Challenges Submissions

Challenge "aido2-LF-real-validation"

Challenge description

(No description.)

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance - driven_lanedir_consec_median

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time - survival_time_median

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Dependencies

Dependencies

No dependencies

Details

Technical details

Evaluation steps details

Evaluation step hw-evaluation

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lf-real-validation-hw-evaluation-evaluator:2019_05_13_17_48_37@sha256:1168d196612a40db2eedce3f412838293600898ed3f6a96dccc2679c82acee56
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots1
AIDO 2 Map LF public1

Evaluation step eval0

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-validation-eval0-evaluator:2019_06_11_19_00_11@sha256:04dad2af97033eef32f70bfea1f9fad34663a5ad17f7e998c7984bb68012de88
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots1
AIDO 2 Map LF public1

Evaluation step eval0-videos

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lf-real-validation-eval0-videos-evaluator:2019_05_17_13_23_57@sha256:90e7c45aff3c2936cca59c2eb16668e882cc06a377ce9315900496a8efaaeb61
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-extractATs

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lf-real-validation-eval0-extractats-evaluator:2019_05_17_14_45_03@sha256:a12b33e0fb36d6abb88b542b1d0a751467ff24824ac209b75d09db985d931d64
        environment:
            BAGS_PATH: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            ACQ_OUTPUT_BAG: /challenges/challenge-evaluation-output/processed.bag
            ACQ_TEST_STREAM: '0'
            ACQ_DEVICE_THREADS: '6'
            ACQ_OBSERVATIONS_STATISTICS_OUTPUT: /challenges/challenge-evaluation-output/statistics.yaml

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-WesQuackerson

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lf-real-validation-eval0-wesquackerson-evaluator:2019_05_17_14_45_11@sha256:285263e1b8a043e4ef9f7dc25b3d5796c6a0bb09a4d629bd8bba5ec1b07159e5
        environment:
            ATMSGS_BAG: /challenges/previous-steps/eval0-extractATs/challenge-evaluation-output/processed.bag
            TITLE_CARD: '1'
            VIDEO_BAGS: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_FILE: /challenges/challenge-evaluation-output/overheadVideo.mp4
            POSES_TOPIC: /poses_acquisition/poses
            VIDEO_TOPIC: /camera_node/image/compressed
            MIN_SHOT_LENGTH: '12'
            OUTPUT_FRAMERATE: '12'
            PREVIOUS_RESULTS: /challenges/previous-steps/eval0/challenge-evaluation-output/challenge-results/challenge_results.yaml
            CAMERA_RESOLUTION_WIDTH: '972'
            CAMERA_RESOLUTION_HEIGHT: '1296'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-0

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lf-real-validation-eval0-videos-0-evaluator:2019_05_16_13_56_53@sha256:45494b8ddc5c6d02c900845a72601852f9b63253b18d19108d7f5ee51ebbeaa5
        environment:
            WORKER_I: '0'
            WORKER_N: '2'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-1

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lf-real-validation-eval0-videos-1-evaluator:2019_05_16_13_57_12@sha256:45494b8ddc5c6d02c900845a72601852f9b63253b18d19108d7f5ee51ebbeaa5
        environment:
            WORKER_I: '1'
            WORKER_N: '2'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-autobots

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-validation-eval0-videos-autobots-evaluator:2019_06_11_19_00_51@sha256:5876da21ebc74f8c77acec009b39b23c52b9f374eef76e182936e1ad8d05ae86
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: autobot
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-watchtowers

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-validation-eval0-videos-watchtowers-evaluator:2019_06_11_19_03_32@sha256:5876da21ebc74f8c77acec009b39b23c52b9f374eef76e182936e1ad8d05ae86
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '0'
            BAG_NAME_FILTER: watchtower
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval1

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-validation-eval1-evaluator:2019_06_11_19_00_39@sha256:04dad2af97033eef32f70bfea1f9fad34663a5ad17f7e998c7984bb68012de88
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots1
AIDO 2 Map LF public1

Evaluation step eval1-videos-autobots

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-validation-eval1-videos-autobots-evaluator:2019_06_11_19_03_47@sha256:5876da21ebc74f8c77acec009b39b23c52b9f374eef76e182936e1ad8d05ae86
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: autobot
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval1-videos-watchtowers

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-validation-eval1-videos-watchtowers-evaluator:2019_06_11_19_04_02@sha256:5876da21ebc74f8c77acec009b39b23c52b9f374eef76e182936e1ad8d05ae86
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '0'
            BAG_NAME_FILTER: watchtower
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1