Challenge "mooc-BV1"

These are the metrics defined:

Distance from starting point. - distance-from-start_mean

Distance from starting point

Evaluation steps details

• At the beginning execute step sim-0of5.

• If step sim-0of5 has result failed, then declare the submission FAILED.

• If step sim-0of5 has result error, then declare the submission ERROR.

• At the beginning execute step sim-1of5.

• If step sim-1of5 has result failed, then declare the submission FAILED.

• If step sim-1of5 has result error, then declare the submission ERROR.

• At the beginning execute step sim-2of5.

• If step sim-2of5 has result failed, then declare the submission FAILED.

• If step sim-2of5 has result error, then declare the submission ERROR.

• At the beginning execute step sim-3of5.

• If step sim-3of5 has result failed, then declare the submission FAILED.

• If step sim-3of5 has result error, then declare the submission ERROR.

• At the beginning execute step sim-4of5.

• If step sim-4of5 has result failed, then declare the submission FAILED.

• If step sim-4of5 has result error, then declare the submission ERROR.

• If (step sim-0of5 has result success) and (step sim-1of5 has result success) and (step sim-2of5 has result success) and (step sim-3of5 has result success) and (step sim-4of5 has result success), then declare the submission SUCCESS.

Evaluation step sim-0of5

Timeout 10000.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:96e899b76f9e102b9cabc69f0bfef166bf63a19f9906d6f135b55a7c4c8fa9ba
        environment:
            replica: '{"index": 0, "total": 5}'
            experiment_manager_parameters: 'episodes_per_scenario: 3

                episode_length_s: 60.0

                min_episode_length_s: 0.0

                seed: 20200922

                physics_dt: 0.05

                max_failures: 2

                fifo_dir: /fifos

                sim_in: /fifos/simulator-in

                sim_out: /fifos/simulator-out

                sm_in: /fifos/scenario_maker-in

                sm_out: /fifos/scenario_maker-out

                timeout_initialization: 120

                timeout_regular: 120

                port: 10123

                scenarios:

                - /scenarios

                '
        ports:
        - '10123'
    simulator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:f6e97655c6dbe5966e76cbba2afb40fb8b138b1c95181b8e862852ae64b20356
        environment:
            replica: '{"index": 0, "total": 5}'
            AIDONODE_CONFIG: "env_constructor: Simulator\nenv_parameters:\n  max_steps:\
                \ 500001 # we don't want the gym to reset itself\n  domain_rand: 0\n\
                \  camera_width: 640\n  camera_height: 480\n  distortion: true\n \
                \ num_tris_distractors: 0\n  #color_ground: [0, 0, 0] # black\n  #color_sky:\
                \ [0, 0, 0.1] # dark blue\n  enable_leds: true\n\nterminate_on_ool:\
                \ false\nterminate_on_out_of_tile: true\nterminate_on_collision: true\n\
                topdown_resolution: 300\nmax_pixel_mov: 1000\ndebug_profile: False\n\
                debug_profile_summary: True\n"
            AIDONODE_DATA_IN: /fifos/simulator-in
            AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
    solution-ego0:
        image: SUBMISSION_CONTAINER
        environment:
            replica: '{"index": 0, "total": 5}'
            AIDONODE_NAME: ego0
            AIDONODE_DATA_IN: /fifos/ego0-in
            AIDONODE_DATA_OUT: fifo:/fifos/ego0-out

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Evaluation step sim-1of5

Timeout 10000.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:96e899b76f9e102b9cabc69f0bfef166bf63a19f9906d6f135b55a7c4c8fa9ba
        environment:
            replica: '{"index": 1, "total": 5}'
            experiment_manager_parameters: 'episodes_per_scenario: 3

                episode_length_s: 60.0

                min_episode_length_s: 0.0

                seed: 20200922

                physics_dt: 0.05

                max_failures: 2

                fifo_dir: /fifos

                sim_in: /fifos/simulator-in

                sim_out: /fifos/simulator-out

                sm_in: /fifos/scenario_maker-in

                sm_out: /fifos/scenario_maker-out

                timeout_initialization: 120

                timeout_regular: 120

                port: 10123

                scenarios:

                - /scenarios

                '
        ports:
        - '10123'
    simulator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:f6e97655c6dbe5966e76cbba2afb40fb8b138b1c95181b8e862852ae64b20356
        environment:
            replica: '{"index": 1, "total": 5}'
            AIDONODE_CONFIG: "env_constructor: Simulator\nenv_parameters:\n  max_steps:\
                \ 500001 # we don't want the gym to reset itself\n  domain_rand: 0\n\
                \  camera_width: 640\n  camera_height: 480\n  distortion: true\n \
                \ num_tris_distractors: 0\n  #color_ground: [0, 0, 0] # black\n  #color_sky:\
                \ [0, 0, 0.1] # dark blue\n  enable_leds: true\n\nterminate_on_ool:\
                \ false\nterminate_on_out_of_tile: true\nterminate_on_collision: true\n\
                topdown_resolution: 300\nmax_pixel_mov: 1000\ndebug_profile: False\n\
                debug_profile_summary: True\n"
            AIDONODE_DATA_IN: /fifos/simulator-in
            AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
    solution-ego0:
        image: SUBMISSION_CONTAINER
        environment:
            replica: '{"index": 1, "total": 5}'
            AIDONODE_NAME: ego0
            AIDONODE_DATA_IN: /fifos/ego0-in
            AIDONODE_DATA_OUT: fifo:/fifos/ego0-out

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Evaluation step sim-2of5

Timeout 10000.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:96e899b76f9e102b9cabc69f0bfef166bf63a19f9906d6f135b55a7c4c8fa9ba
        environment:
            replica: '{"index": 2, "total": 5}'
            experiment_manager_parameters: 'episodes_per_scenario: 3

                episode_length_s: 60.0

                min_episode_length_s: 0.0

                seed: 20200922

                physics_dt: 0.05

                max_failures: 2

                fifo_dir: /fifos

                sim_in: /fifos/simulator-in

                sim_out: /fifos/simulator-out

                sm_in: /fifos/scenario_maker-in

                sm_out: /fifos/scenario_maker-out

                timeout_initialization: 120

                timeout_regular: 120

                port: 10123

                scenarios:

                - /scenarios

                '
        ports:
        - '10123'
    simulator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:f6e97655c6dbe5966e76cbba2afb40fb8b138b1c95181b8e862852ae64b20356
        environment:
            replica: '{"index": 2, "total": 5}'
            AIDONODE_CONFIG: "env_constructor: Simulator\nenv_parameters:\n  max_steps:\
                \ 500001 # we don't want the gym to reset itself\n  domain_rand: 0\n\
                \  camera_width: 640\n  camera_height: 480\n  distortion: true\n \
                \ num_tris_distractors: 0\n  #color_ground: [0, 0, 0] # black\n  #color_sky:\
                \ [0, 0, 0.1] # dark blue\n  enable_leds: true\n\nterminate_on_ool:\
                \ false\nterminate_on_out_of_tile: true\nterminate_on_collision: true\n\
                topdown_resolution: 300\nmax_pixel_mov: 1000\ndebug_profile: False\n\
                debug_profile_summary: True\n"
            AIDONODE_DATA_IN: /fifos/simulator-in
            AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
    solution-ego0:
        image: SUBMISSION_CONTAINER
        environment:
            replica: '{"index": 2, "total": 5}'
            AIDONODE_NAME: ego0
            AIDONODE_DATA_IN: /fifos/ego0-in
            AIDONODE_DATA_OUT: fifo:/fifos/ego0-out

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Evaluation step sim-3of5

Timeout 10000.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:96e899b76f9e102b9cabc69f0bfef166bf63a19f9906d6f135b55a7c4c8fa9ba
        environment:
            replica: '{"index": 3, "total": 5}'
            experiment_manager_parameters: 'episodes_per_scenario: 3

                episode_length_s: 60.0

                min_episode_length_s: 0.0

                seed: 20200922

                physics_dt: 0.05

                max_failures: 2

                fifo_dir: /fifos

                sim_in: /fifos/simulator-in

                sim_out: /fifos/simulator-out

                sm_in: /fifos/scenario_maker-in

                sm_out: /fifos/scenario_maker-out

                timeout_initialization: 120

                timeout_regular: 120

                port: 10123

                scenarios:

                - /scenarios

                '
        ports:
        - '10123'
    simulator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:f6e97655c6dbe5966e76cbba2afb40fb8b138b1c95181b8e862852ae64b20356
        environment:
            replica: '{"index": 3, "total": 5}'
            AIDONODE_CONFIG: "env_constructor: Simulator\nenv_parameters:\n  max_steps:\
                \ 500001 # we don't want the gym to reset itself\n  domain_rand: 0\n\
                \  camera_width: 640\n  camera_height: 480\n  distortion: true\n \
                \ num_tris_distractors: 0\n  #color_ground: [0, 0, 0] # black\n  #color_sky:\
                \ [0, 0, 0.1] # dark blue\n  enable_leds: true\n\nterminate_on_ool:\
                \ false\nterminate_on_out_of_tile: true\nterminate_on_collision: true\n\
                topdown_resolution: 300\nmax_pixel_mov: 1000\ndebug_profile: False\n\
                debug_profile_summary: True\n"
            AIDONODE_DATA_IN: /fifos/simulator-in
            AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
    solution-ego0:
        image: SUBMISSION_CONTAINER
        environment:
            replica: '{"index": 3, "total": 5}'
            AIDONODE_NAME: ego0
            AIDONODE_DATA_IN: /fifos/ego0-in
            AIDONODE_DATA_OUT: fifo:/fifos/ego0-out

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step

Evaluation step sim-4of5

Timeout 10000.0

This is the Docker Compose configuration skeleton:

version: '3'
services:
    evaluator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:96e899b76f9e102b9cabc69f0bfef166bf63a19f9906d6f135b55a7c4c8fa9ba
        environment:
            replica: '{"index": 4, "total": 5}'
            experiment_manager_parameters: 'episodes_per_scenario: 3

                episode_length_s: 60.0

                min_episode_length_s: 0.0

                seed: 20200922

                physics_dt: 0.05

                max_failures: 2

                fifo_dir: /fifos

                sim_in: /fifos/simulator-in

                sim_out: /fifos/simulator-out

                sm_in: /fifos/scenario_maker-in

                sm_out: /fifos/scenario_maker-out

                timeout_initialization: 120

                timeout_regular: 120

                port: 10123

                scenarios:

                - /scenarios

                '
        ports:
        - '10123'
    simulator:
        image: docker.io/andreacensi/duckietown-challenges@sha256:f6e97655c6dbe5966e76cbba2afb40fb8b138b1c95181b8e862852ae64b20356
        environment:
            replica: '{"index": 4, "total": 5}'
            AIDONODE_CONFIG: "env_constructor: Simulator\nenv_parameters:\n  max_steps:\
                \ 500001 # we don't want the gym to reset itself\n  domain_rand: 0\n\
                \  camera_width: 640\n  camera_height: 480\n  distortion: true\n \
                \ num_tris_distractors: 0\n  #color_ground: [0, 0, 0] # black\n  #color_sky:\
                \ [0, 0, 0.1] # dark blue\n  enable_leds: true\n\nterminate_on_ool:\
                \ false\nterminate_on_out_of_tile: true\nterminate_on_collision: true\n\
                topdown_resolution: 300\nmax_pixel_mov: 1000\ndebug_profile: False\n\
                debug_profile_summary: True\n"
            AIDONODE_DATA_IN: /fifos/simulator-in
            AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
    solution-ego0:
        image: SUBMISSION_CONTAINER
        environment:
            replica: '{"index": 4, "total": 5}'
            AIDONODE_NAME: ego0
            AIDONODE_DATA_IN: /fifos/ego0-in
            AIDONODE_DATA_OUT: fifo:/fifos/ego0-out

The text SUBMISSION_CONTAINER will be replaced with the user containter.

Resources required for evaluating this step