Duckietown Challenges | Home | Challenges | Submissions |
(No description.)
These are the metrics defined:
driven_lanedir_consec
This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)
This is discretized to tiles.
survival_time
This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.
deviation-center-line
This is the median lateral deviation from the center line.
in-drivable-lane
This is the median of the time spent outside of the drivable zones. For example this penalizes driving in the wrong lane.
No dependencies
At the beginning execute step eval0
.
If step eval0
has result success, then execute step eval0-visualize
.
If step eval0
has result failed, then declare the submission FAILED
.
If step eval0
has result error, then declare the submission ERROR
.
If step eval0
has result success, then execute step eval0-videos
.
If step eval1
has result success, then execute step eval1-videos
.
If step eval2
has result success, then execute step eval2-videos
.
If step eval0
has result success, then execute step eval1
.
If step eval0-visualize
has result failed, then declare the submission FAILED
.
If step eval0-visualize
has result error, then declare the submission ERROR
.
If step eval1
has result success, then execute step eval1-visualize
.
If step eval1
has result failed, then declare the submission FAILED
.
If step eval1
has result error, then declare the submission ERROR
.
If step eval1
has result success, then execute step eval2
.
If step eval1-visualize
has result failed, then declare the submission FAILED
.
If step eval1-visualize
has result error, then declare the submission ERROR
.
If step eval2
has result success, then execute step eval2-visualize
.
If step eval2
has result failed, then declare the submission FAILED
.
If step eval2
has result error, then declare the submission ERROR
.
If step eval2-visualize
has result success, then declare the submission SUCCESS
.
If step eval2-visualize
has result failed, then declare the submission FAILED
.
If step eval2-visualize
has result error, then declare the submission ERROR
.
eval0
Timeout 18000.0
Evaluation in the robotarium.
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:0bea3cba7422f9e44468d7698e19cef710e82972774e5c0ed67eb7855e93f0fb environment: {}
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
# Duckiebots | 3 |
AIDO 5 Map large loop | 1 |
eval1
Timeout 18000.0
Evaluation in the robotarium.
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:20c7991d50ffe21634964eb3ed3a33f6f8b7fb57807970b804c5c2041c82ef22 environment: {}
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
# Duckiebots | 3 |
AIDO 5 Map large loop | 1 |
eval2
Timeout 18000.0
Evaluation in the robotarium.
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:95091e550bc85b2e1075f772ae86e21cc52e9ee308e7a8d2d0be896a87e5d477 environment: {}
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
# Duckiebots | 3 |
AIDO 5 Map large loop | 1 |
eval0-videos
Timeout 10800.0
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:fb6881d1a62f8d99eb94c0195ef51436e56f3853bb7244cc097adce352d4272f environment: WORKER_I: '0' WORKER_N: '1' INPUT_DIR: /challenges/previous-steps/eval0 OUTPUT_DIR: /challenges/challenge-evaluation-output DEBUG_OVERLAY: '1' BAG_NAME_FILTER: robot,watchtower OUTPUT_FRAMERATE: '30'
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
Cloud simulations | 1 |
eval1-videos
Timeout 10800.0
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:fb6881d1a62f8d99eb94c0195ef51436e56f3853bb7244cc097adce352d4272f environment: WORKER_I: '0' WORKER_N: '1' INPUT_DIR: /challenges/previous-steps/eval1 OUTPUT_DIR: /challenges/challenge-evaluation-output DEBUG_OVERLAY: '1' BAG_NAME_FILTER: robot,watchtower OUTPUT_FRAMERATE: '30'
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
Cloud simulations | 1 |
eval2-videos
Timeout 10800.0
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:fb6881d1a62f8d99eb94c0195ef51436e56f3853bb7244cc097adce352d4272f environment: WORKER_I: '0' WORKER_N: '1' INPUT_DIR: /challenges/previous-steps/eval2 OUTPUT_DIR: /challenges/challenge-evaluation-output DEBUG_OVERLAY: '1' BAG_NAME_FILTER: robot,watchtower OUTPUT_FRAMERATE: '30'
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
Cloud simulations | 1 |
eval0-visualize
Timeout 1080.0
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:23255ec65e3f8de4a648ac5b29bc25c22c03c9bd13d978c987d70fe27b62b696 environment: STEP_NAME: eval0
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
Cloud simulations | 1 |
eval1-visualize
Timeout 1080.0
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:23255ec65e3f8de4a648ac5b29bc25c22c03c9bd13d978c987d70fe27b62b696 environment: STEP_NAME: eval1
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
Cloud simulations | 1 |
eval2-visualize
Timeout 1080.0
This is the Docker Compose configuration skeleton:
version: '3' services: evaluator: image: docker.io/andreacensi/duckietown-challenges@sha256:23255ec65e3f8de4a648ac5b29bc25c22c03c9bd13d978c987d70fe27b62b696 environment: STEP_NAME: eval2
The text SUBMISSION_CONTAINER
will be replaced with the user containter.
Cloud simulations | 1 |