AI Driving Olympics Home Challenges Submissions Jobs

Challenge "LFVI 🚗🚗🚦 - Lane following + Vehicles + Intersections (simulation 👾, validation 🏋) "

Challenge description

(No description.)

Leaderboard

Submissions

Challenge logistics

Scoring

Scoring criteria

These are the metrics defined:

Traveled distance

This is the median distance traveled, along a lane. (That is, going in circles will not make this metric increase.)

This is discretized to tiles.

Survival time

This is the median survival time. The simulation is terminated when the car goes outside of the road or it crashes with an obstacle.

Lateral deviation

This is the median lateral deviation from the center line.

Major infractions

This is the median of the time spent outside of the drivable zones. For example this penalizes driving in the wrong lane.

Details

Technical details

Evaluation steps details

  • At the beginning execute step step1-simulation.

  • If step step1-simulation finishes with status success, then declare the submission SUCCESS.

  • If step step1-simulation finishes with status failed, then declare the submission FAILED.

  • If step step1-simulation finishes with status error, then declare the submission ERROR.

Evaluation step step1-simulation

Timeout 3600.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    solution:
        image: SUBMISSION_CONTAINER
        environment:
            AIDONODE_DATA_IN: /fifos/agent-in
            AIDONODE_DATA_OUT: fifo:/fifos/agent-out
    evaluator:
        image: docker.io/andreacensi/aido2-lfvi-sim-validation-offseason-step1-simulation-evaluator:2019_05_29_23_04_35@sha256:213ad50470dbc8eb46358923697d514115571d52805b7e533455505237267396
        environment:
            experiment_manager_parameters: 'episodes_per_scenario: 1

                episode_length_s: 15.0

                min_episode_length_s: 0.0

                seed: 42

                physics_dt: 0.05

                max_failures: 2

                agent_in: /fifos/agent-in

                agent_out: /fifos/agent-out

                sim_in: /fifos/simulator-in

                sim_out: /fifos/simulator-out

                sm_in: /fifos/scenario_maker-in

                sm_out: /fifos/scenario_maker-out

                timeout_initialization: 15

                timeout_regular: 30

                '
    simulator:
        image: docker.io/andreacensi/aido2-lfvi-sim-validation-offseason-step1-simulation-simulator:2019_05_29_23_04_57@sha256:5c2becae6a23fe73d1fc37d537bd33f5aee5580538eb2a50d77f719fb78d7c6b
        environment:
            AIDONODE_CONFIG: "env_constructor: Simulator\nenv_parameters:\n  max_steps:\
                \ 500001 # we don't want the gym to reset itself\n  domain_rand: 0\n\
                \  camera_width: 640\n  camera_height: 480\n  distortion: true\n"
            AIDONODE_DATA_IN: /fifos/simulator-in
            AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
    scenario_maker:
        image: docker.io/andreacensi/aido2-lfvi-sim-validation-offseason-step1-simulation-scenario_maker:2019_05_29_23_05_22@sha256:076739bd069dc5eb706f3fd656abc0c62bf4b82c19a0a26bce8802afc62a028b
        environment:
            AIDONODE_CONFIG: 'maps:

                - udem1

                scenarios_per_map: 5

                robots_npcs: 4

                '
            AIDONODE_DATA_IN: /fifos/scenario_maker-in
            AIDONODE_DATA_OUT: fifo:/fifos/scenario_maker-out

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

No particular resources required.