Duckietown Challenges Home Challenges Submissions

Challenge "gb-eDI"

Challenge description

Dynamics and image randomization

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance - driven_lanedir_consec_median

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time - survival_time_median

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Lateral deviation - deviation-center-line_median

This is the median lateral deviation from the center line.

Major infractions - in-drivable-lane_median

This is the median of the time spent outside of the drivable zones. For example this penalizes driving in the wrong lane.

Dependencies

Dependencies

No dependencies

Details

Technical details

Evaluation steps details

Evaluation step step1-simulation

Timeout 7200.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    solution:
        image: SUBMISSION_CONTAINER
        environment:
            AIDONODE_DATA_IN: /fifos/agent-in
            AIDONODE_DATA_OUT: fifo:/fifos/agent-out
    evaluator:
        image: docker.io/gibernas/gb-edi-step1-simulation-evaluator:2020_02_21_08_49_34@sha256:893ff1bb91c96bfaa1e91f4778366e3f3efb371a27219f858ef73b3bf4b7c82b
        environment:
            experiment_manager_parameters: 'episodes_per_scenario: 1

                episode_length_s: 20.0

                min_episode_length_s: 0.0

                seed: 43

                physics_dt: 0.05

                max_failures: 4

                agent_in: /fifos/agent-in

                agent_out: /fifos/agent-out

                sim_in: /fifos/simulator-in

                sim_out: /fifos/simulator-out

                sm_in: /fifos/scenario_maker-in

                sm_out: /fifos/scenario_maker-out

                timeout_initialization: 120

                timeout_regular: 120

                '
    simulator:
        image: docker.io/gibernas/gb-edi-step1-simulation-simulator:2020_02_21_08_49_59@sha256:1b1590ecf2b0606d1c46f35ea4d556feab8c961f6b51adeb7b0deee10099ce79
        environment:
            AIDONODE_CONFIG: "env_constructor: Simulator\nenv_parameters:\n  max_steps:\
                \ 500001 # we don't want the gym to reset itself\n  domain_rand: 0\n\
                \  camera_width: 640\n  camera_height: 480\n  distortion: true\n \
                \ dynamics_rand: true\n  domain_rand: true\n  camera_rand: true\n\
                \  seed: 43\n"
            AIDONODE_DATA_IN: /fifos/simulator-in
            AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
    scenario_maker:
        image: docker.io/gibernas/gb-edi-step1-simulation-scenario_maker:2020_02_21_08_52_19@sha256:a2f2d5f26f8e27f82ddcbafba465f9d79c9a5ebab9b23d8e9191d89a65bb51ad
        environment:
            AIDONODE_CONFIG: 'maps:

                - ETHZ_autolab_technical_track

                scenarios_per_map: 10

                robots_npcs: 0

                '
            AIDONODE_DATA_IN: /fifos/scenario_maker-in
            AIDONODE_DATA_OUT: fifo:/fifos/scenario_maker-out

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Cloud simulations1