Duckietown Challenges Home Challenges Submissions

Challenge "aido2_LFVI_r3-v4-z2"

Challenge description

(No description.)

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance - driven_lanedir_consec_median

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time - survival_time_median

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Lateral deviation - deviation-center-line_median

This is the median lateral deviation from the center line.

Major infractions - in-drivable-lane_median

This is the median of the time spent outside of the drivable zones. For example this penalizes driving in the wrong lane.

Dependencies

Dependencies

No dependencies

Details

Technical details

Evaluation steps details

Evaluation step step1-simulation

Timeout None

This is the Docker Compose configuration skeleton:

version: '3'
services:
    solution:
        image: SUBMISSION_CONTAINER
        environment:
            AIDONODE_DATA_IN: /fifos/agent-in
            AIDONODE_DATA_OUT: fifo:/fifos/agent-out
            AIDONODE_ENCODING: cbor
    evaluator:
        image: docker.io/andreacensi/aido2_lfvi_r3-v4-z2-step1-simulation-evaluator:2019_04_13_20_59_32@sha256:b0b9c003bc7984aab8ed9d009d9735472156eac2617b398e359a56979eb7142d
        environment:
            AIDONODE_ENCODING: cbor
            DISABLE_CONTRACTS: '1'
            experiment_manager_parameters: 'episodes_per_scenario: 1

                episode_length_s: 15.0

                min_episode_length_s: 0

                seed: 42

                physics_dt: 0.05

                max_failures: 2

                agent_in: /fifos/agent-in

                agent_out: /fifos/agent-out

                sim_in: /fifos/simulator-in

                sim_out: /fifos/simulator-out

                sm_in: /fifos/scenario_maker-in

                sm_out: /fifos/scenario_maker-out

                '
    simulator:
        image: docker.io/andreacensi/aido2_lfvi_r3-v4-z2-step1-simulation-simulator:2019_04_13_21_00_46@sha256:b87f78b077ad5daa111b761a94f20906cf3ecce20b4e212b763f3d9b1c857f4c
        environment:
            AIDONODE_CONFIG: "env_constructor: Simulator\nenv_parameters:\n  max_steps:\
                \ 500001 # we don't want the gym to reset itself\n  domain_rand: 0\n\
                \  camera_width: 640\n  camera_height: 480\n  distortion: true\n"
            AIDONODE_DATA_IN: /fifos/simulator-in
            AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
            AIDONODE_ENCODING: cbor
    scenario_maker:
        image: docker.io/andreacensi/aido2_lfvi_r3-v4-z2-step1-simulation-scenario_maker:2019_04_13_21_00_08@sha256:59f5ff9eae4cd6fec75e4358b23a16d51cac57f6b688fd442f3bf4b25bef2b12
        environment:
            AIDONODE_CONFIG: 'maps:

                - udem1

                scenarios_per_map: 5

                robots_npcs: 4

                '
            AIDONODE_DATA_IN: /fifos/scenario_maker-in
            AIDONODE_DATA_OUT: fifo:/fifos/scenario_maker-out
            AIDONODE_ENCODING: cbor
            DISABLE_CONTRACTS: '1'

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

No particular resources required.