Challenge "aido2-LFVI-real-testing"

These are the metrics defined:

Traveled distance - driven_lanedir_consec_median

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time - survival_time_median

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Evaluation steps details

• At the beginning execute step eval0.

• If step eval0 has result success, then declare the submission SUCCESS.

• If step eval0 has result failed, then declare the submission FAILED.

• If step eval0 has result error, then declare the submission ERROR.

Evaluation step hw-evaluation

Timeout 18000.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/amaurx/aido2-lfvi-real-testing-hw-evaluation-evaluator:2019_05_13_17_49_12@sha256:1168d196612a40db2eedce3f412838293600898ed3f6a96dccc2679c82acee56
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Evaluation step eval0

Timeout 18000.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/gian1717/aido2-lfvi-real-testing-eval0-evaluator:2019_06_11_19_08_27@sha256:04dad2af97033eef32f70bfea1f9fad34663a5ad17f7e998c7984bb68012de88
        environment: {}
        ports:
        - 8005:8005

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step