Duckietown Challenges Home Challenges Submissions

Challenge "aido2-LF-real-testing"

Challenge description

(No description.)

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance - driven_lanedir_consec_median

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time - survival_time_median

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Dependencies

Dependencies

No dependencies

Details

Technical details

Evaluation steps details

Evaluation step hw-evaluation

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lf-real-testing-hw-evaluation-evaluator:2019_05_13_17_48_19@sha256:1168d196612a40db2eedce3f412838293600898ed3f6a96dccc2679c82acee56
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots1
AIDO 2 Map LF private1

Evaluation step eval0

Timeout 18000.0

Evaluation in the robotarium.

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-testing-eval0-evaluator:2019_06_11_19_07_10@sha256:04dad2af97033eef32f70bfea1f9fad34663a5ad17f7e998c7984bb68012de88
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

# Duckiebots1
AIDO 2 Map LF private1

Evaluation step eval0-videos

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/aido2-lf-real-testing-eval0-videos-evaluator:2019_05_15_16_22_04@sha256:440645bb2e89727086a010c45c71b3d069b60489343501e05eca8f5c9108ee3c
        environment:
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-0

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lf-real-testing-eval0-videos-0-evaluator:2019_05_16_07_22_45@sha256:90e7c45aff3c2936cca59c2eb16668e882cc06a377ce9315900496a8efaaeb61
        environment:
            WORKER_I: '0'
            WORKER_N: '2'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-1

Timeout 1800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/aleksandarpetrov/aido2-lf-real-testing-eval0-videos-1-evaluator:2019_05_16_07_33_00@sha256:90e7c45aff3c2936cca59c2eb16668e882cc06a377ce9315900496a8efaaeb61
        environment:
            WORKER_I: '1'
            WORKER_N: '2'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-extractATs

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-testing-eval0-extractats-evaluator:2019_05_20_11_23_06@sha256:419dbf15fc26c5a361678e984c0a9c852298fc27e6b6e2444633f561f60f006a
        environment:
            BAGS_PATH: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            ACQ_OUTPUT_BAG: /challenges/challenge-evaluation-output/processed.bag
            ACQ_TEST_STREAM: '0'
            ACQ_DEVICE_THREADS: '6'
            ACQ_OBSERVATIONS_STATISTICS_OUTPUT: /challenges/challenge-evaluation-output/statistics.yaml

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-WesQuackerson

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-testing-eval0-wesquackerson-evaluator:2019_05_20_11_23_38@sha256:410e1c87425a36638404166049a50dada92d2a819e18f859858bd3eed0a59978
        environment:
            ATMSGS_BAG: /challenges/previous-steps/eval0-extractATs/challenge-evaluation-output/processed.bag
            TITLE_CARD: '1'
            VIDEO_BAGS: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_FILE: /challenges/challenge-evaluation-output/overheadVideo.mp4
            POSES_TOPIC: /poses_acquisition/poses
            VIDEO_TOPIC: /camera_node/image/compressed
            MIN_SHOT_LENGTH: '12'
            OUTPUT_FRAMERATE: '12'
            PREVIOUS_RESULTS: /challenges/previous-steps/eval0/challenge-evaluation-output/challenge-results/challenge_results.yaml
            CAMERA_RESOLUTION_WIDTH: '972'
            CAMERA_RESOLUTION_HEIGHT: '1296'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-autobots

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-testing-eval0-videos-autobots-evaluator:2019_06_11_19_07_22@sha256:5876da21ebc74f8c77acec009b39b23c52b9f374eef76e182936e1ad8d05ae86
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '1'
            BAG_NAME_FILTER: autobot
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1

Evaluation step eval0-videos-watchtowers

Timeout 10800.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lf-real-testing-eval0-videos-watchtowers-evaluator:2019_06_11_19_07_37@sha256:5876da21ebc74f8c77acec009b39b23c52b9f374eef76e182936e1ad8d05ae86
        environment:
            WORKER_I: '0'
            WORKER_N: '1'
            INPUT_DIR: /challenges/previous-steps/eval0/challenge-evaluation-output/raw_logs/bags
            OUTPUT_DIR: /challenges/challenge-evaluation-output
            DEBUG_OVERLAY: '0'
            BAG_NAME_FILTER: watchtower
            OUTPUT_FRAMERATE: '12'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

IPFS mountpoint /ipfs available1