Submission 9312

Submission

9312

Competing

yes

Challenge

aido5-LF-sim-validation

User

Jerome Labonte 🇨🇦

Date submitted

2020-10-25 18:51:49+00:00

Last status update

2021-12-04 20:52:33+00:00

Complete

complete

Details

Evaluation is complete.

Sisters

Result

💚

Jobs

LFv-sim: 58248

User label

exercise_ros_template

Admin priority

Blessing

n/a

User priority

58248

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID

step

status

up to date

date started

date completed

duration

message

58248

LFv-sim

success

yes

2020-12-03 22:30:20+00:00

2020-12-03 22:55:39+00:00

0:25:19

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	6.031820652770268
survival_time_median	49.9999999999993
deviation-center-line_median	2.890422675124041
in-drivable-lane_median	3.4499999999998687

other stats

agent_compute-ego0_max	0.012406478362732317
agent_compute-ego0_mean	0.012276380741659668
agent_compute-ego0_median	0.012284028484818316
agent_compute-ego0_min	0.01213098763426972
complete-iteration_max	0.1979052769832122
complete-iteration_mean	0.17552726892415405
complete-iteration_median	0.17495263801231964
complete-iteration_min	0.15429852268876482
deviation-center-line_max	3.758763972217636
deviation-center-line_mean	2.5023341170384743
deviation-center-line_min	0.4697271456881783
deviation-heading_max	12.82412797422542
deviation-heading_mean	8.260324347972649
deviation-heading_median	9.078803948993649
deviation-heading_min	2.0595615196778807
driven_any_max	11.679181502401502
driven_any_mean	8.223934584604503
driven_any_median	9.015676038053552
driven_any_min	3.185204759909409
driven_lanedir_consec_max	7.104699727992231
driven_lanedir_consec_mean	5.146472905121384
driven_lanedir_consec_min	1.4175505869527705
driven_lanedir_max	11.29941124583484
driven_lanedir_mean	7.387417596565012
driven_lanedir_median	8.41635427673622
driven_lanedir_min	1.4175505869527705
get_duckie_state_max	2.30358800324274e-06
get_duckie_state_mean	2.202856139607719e-06
get_duckie_state_median	2.22553352004613e-06
get_duckie_state_min	2.056769515095876e-06
get_robot_state_max	0.003805351098510844
get_robot_state_mean	0.003676816666338264
get_robot_state_median	0.003668067316877763
get_robot_state_min	0.003565780933086689
get_state_dump_max	0.004711381600957231
get_state_dump_mean	0.004611894442696353
get_state_dump_median	0.0045920555990656385
get_state_dump_min	0.004552084971696902
get_ui_image_max	0.03427312679779835
get_ui_image_mean	0.030095626159444507
get_ui_image_median	0.030282848125690973
get_ui_image_min	0.02554368158859774
in-drivable-lane_max	11.500000000000124
in-drivable-lane_mean	4.774999999999968
in-drivable-lane_min	0.70000000000001
per-episodes	details {"LF-norm-loop-000-ego0": {"driven_any": 11.679181502401502, "get_ui_image": 0.028943001281014093, "step_physics": 0.10325940344156968, "survival_time": 59.99999999999873, "driven_lanedir": 11.29941124583484, "get_state_dump": 0.004711381600957231, "get_robot_state": 0.003805351098510844, "sim_render-ego0": 0.003842394119694668, "get_duckie_state": 2.30358800324274e-06, "in-drivable-lane": 0.70000000000001, "deviation-heading": 11.522751178524354, "agent_compute-ego0": 0.012376527206586064, "complete-iteration": 0.17183745850333565, "set_robot_commands": 0.002266838786802522, "deviation-center-line": 3.751344232044408, "driven_lanedir_consec": 6.253869021466859, "sim_compute_sim_state": 0.010468882386829336, "sim_compute_performance-ego0": 0.002075689420612726}, "LF-norm-zigzag-000-ego0": {"driven_any": 3.185204759909409, "get_ui_image": 0.03427312679779835, "step_physics": 0.1253918115909283, "survival_time": 19.45000000000014, "driven_lanedir": 1.4175505869527705, "get_state_dump": 0.004552084971696902, "get_robot_state": 0.003565780933086689, "sim_render-ego0": 0.003664032006875063, "get_duckie_state": 2.1659410916841942e-06, "in-drivable-lane": 11.500000000000124, "deviation-heading": 2.0595615196778807, "agent_compute-ego0": 0.012191529763050568, "complete-iteration": 0.1979052769832122, "set_robot_commands": 0.0020778521513327573, "deviation-center-line": 0.4697271456881783, "driven_lanedir_consec": 1.4175505869527705, "sim_compute_sim_state": 0.010203690406603691, "sim_compute_performance-ego0": 0.0019014523579524113}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.531352203733553, "get_ui_image": 0.03162269497036785, "step_physics": 0.10669178790069848, "survival_time": 39.999999999999865, "driven_lanedir": 6.810000834221187, "get_state_dump": 0.004589244816335995, "get_robot_state": 0.0036260673913467542, "sim_render-ego0": 0.0036889935254157706, "get_duckie_state": 2.056769515095876e-06, "in-drivable-lane": 4.89999999999978, "deviation-heading": 6.634856719462939, "agent_compute-ego0": 0.012406478362732317, "complete-iteration": 0.17806781752130363, "set_robot_commands": 0.0021248685286732647, "deviation-center-line": 2.029501118203675, "driven_lanedir_consec": 5.809772284073675, "sim_compute_sim_state": 0.01130362336852875, "sim_compute_performance-ego0": 0.0019284964500741568}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.499999872373552, "get_ui_image": 0.02554368158859774, "step_physics": 0.09404513798188806, "survival_time": 59.99999999999873, "driven_lanedir": 10.022707719251253, "get_state_dump": 0.0045948663817952816, "get_robot_state": 0.003710067242408771, "sim_render-ego0": 0.003754696977029335, "get_duckie_state": 2.285125948408065e-06, "in-drivable-lane": 1.9999999999999576, "deviation-heading": 12.82412797422542, "agent_compute-ego0": 0.01213098763426972, "complete-iteration": 0.15429852268876482, "set_robot_commands": 0.002183860585056276, "deviation-center-line": 3.758763972217636, "driven_lanedir_consec": 7.104699727992231, "sim_compute_sim_state": 0.006277977874336592, "sim_compute_performance-ego0": 0.001972915925749335}}
set_robot_commands_max	0.002266838786802522
set_robot_commands_mean	0.002163355012966205
set_robot_commands_median	0.00215436455686477
set_robot_commands_min	0.0020778521513327573
sim_compute_performance-ego0_max	0.002075689420612726
sim_compute_performance-ego0_mean	0.001969638538597157
sim_compute_performance-ego0_median	0.0019507061879117456
sim_compute_performance-ego0_min	0.0019014523579524113
sim_compute_sim_state_max	0.01130362336852875
sim_compute_sim_state_mean	0.009563543509074592
sim_compute_sim_state_median	0.010336286396716514
sim_compute_sim_state_min	0.006277977874336592
sim_render-ego0_max	0.003842394119694668
sim_render-ego0_mean	0.003737529157253709
sim_render-ego0_median	0.003721845251222553
sim_render-ego0_min	0.003664032006875063
simulation-passed	1
step_physics_max	0.1253918115909283
step_physics_mean	0.10734703522877112
step_physics_median	0.10497559567113408
step_physics_min	0.09404513798188806
survival_time_max	59.99999999999873
survival_time_mean	44.862499999999365
survival_time_min	19.45000000000014

No reset possible

58246

LFv-sim

host-error

yes

2020-12-03 22:29:20+00:00

2020-12-03 23:00:31+00:00

0:31:11

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 59, in get_services_id
    raise ZValueError(container_ids=container_ids, services=services, res=res, names=names)
zuper_commons.types.exceptions.ZValueError: 

│ container_ids: [791a9a3ef6aceb8d2aba4ccff4f9b1973088d56feedaba8354b8b2b169111171,
│                 e4a7c844e8f516a7dddb16e05f69cf540c45ebec690951b763316664626eb17a]
│      services: dict[3]
│                │ evaluator:
│                │ dict[7]
│                │ │ image: docker.io/andreacensi/aido5-lf-sim-validation-lfv-sim-evaluator@sha256:6d0af9441525e1ed05be582f41e00fc178083c86797d28cd1a255c7025d0fd50
│                │ │ environment:
│                │ │ dict[10]
│                │ │ │ experiment_manager_parameters:
│                │ │ │ |episodes_per_scenario: 1
│                │ │ │ |episode_length_s: 60.0
│                │ │ │ |min_episode_length_s: 0.0
│                │ │ │ |seed: 888
│                │ │ │ |physics_dt: 0.05
│                │ │ │ |max_failures: 2
│                │ │ │ |fifo_dir: /fifos
│                │ │ │ |sim_in: /fifos/simulator-in
│                │ │ │ |sim_out: /fifos/simulator-out
│                │ │ │ |sm_in: /fifos/scenario_maker-in
│                │ │ │ |sm_out: /fifos/scenario_maker-out
│                │ │ │ |timeout_initialization: 120
│                │ │ │ |timeout_regular: 120
│                │ │ │ |port: 10123
│                │ │ │ |scenarios:
│                │ │ │ |- /scenarios
│                │ │ │ |
│                │ │ │ challenge_name: aido5-LF-sim-validation
│                │ │ │ challenge_step_name: LFv-sim
│                │ │ │ submission_id: 9312
│                │ │ │ submitter_name: jeromelabonte
│                │ │ │ SUBMISSION_CONTAINER: docker.io/jeromelabonte/aido-submissions:2020_10_25_18_51_25@sha256:04c7e311c0eaf6eb7705d26af59a86294addff2361cf6d4c9fb9c8ea981aa3c6
│                │ │ │ username: ubuntu
│                │ │ │ uid: 0
│                │ │ │ USER: ubuntu
│                │ │ │ HOME: /fake-home/ubuntu
│                │ │ ports: [10123]
│                │ │ labels: {org.duckietown.created_by_runner: true, org.duckietown.runner_name: nogpu-prod-07_27d69f530ab0}
│                │ │ user: 0:0
│                │ │ volumes:
│                │ │ [
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission9312/LFv-sim-nogpu-prod-07_27d69f530ab0-job58246-a-wd:/challenges:rw,
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission9312/LFv-sim-nogpu-prod-07_27d69f530ab0-job58246-a-fifos:/fifos:rw,
│                │ │  /tmp/duckietown/dt-challenges-runner/20_12_03_22_29_20-39644/fake-ubuntu-home:/fake-home/ubuntu:rw]
│                │ │ networks: {evaluation: {aliases: [evaluation]} }
│                │ simulator:
│                │ dict[6]
│                │ │ image: docker.io/duckietown/challenge-aido_lf-simulator-gym@sha256:c0096866077db3574e425d40603d8f5fc8ebbd164da7c0578df94ff4ede58d95
│                │ │ environment:
│                │ │ dict[12]
│                │ │ │ AIDONODE_CONFIG:
│                │ │ │ |env_constructor: Simulator
│                │ │ │ |env_parameters:
│                │ │ │ |  max_steps: 500001 # we don't want the gym to reset itself
│                │ │ │ |  domain_rand: 0
│                │ │ │ |  camera_width: 640
│                │ │ │ |  camera_height: 480
│                │ │ │ |  distortion: true
│                │ │ │ |  num_tris_distractors: 0
│                │ │ │ |  color_ground: [0, 0.3, 0] # green
│                │ │ │ |  enable_leds: true
│                │ │ │ |
│                │ │ │ AIDONODE_DATA_IN: /fifos/simulator-in
│                │ │ │ AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
│                │ │ │ challenge_name: aido5-LF-sim-validation
│                │ │ │ challenge_step_name: LFv-sim
│                │ │ │ submission_id: 9312
│                │ │ │ submitter_name: jeromelabonte
│                │ │ │ SUBMISSION_CONTAINER: docker.io/jeromelabonte/aido-submissions:2020_10_25_18_51_25@sha256:04c7e311c0eaf6eb7705d26af59a86294addff2361cf6d4c9fb9c8ea981aa3c6
│                │ │ │ username: ubuntu
│                │ │ │ uid: 0
│                │ │ │ USER: ubuntu
│                │ │ │ HOME: /fake-home/ubuntu
│                │ │ labels: {org.duckietown.created_by_runner: true, org.duckietown.runner_name: nogpu-prod-07_27d69f530ab0}
│                │ │ user: 0:0
│                │ │ volumes:
│                │ │ [
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission9312/LFv-sim-nogpu-prod-07_27d69f530ab0-job58246-a-wd:/challenges:rw,
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission9312/LFv-sim-nogpu-prod-07_27d69f530ab0-job58246-a-fifos:/fifos:rw,
│                │ │  /tmp/duckietown/dt-challenges-runner/20_12_03_22_29_20-39644/fake-ubuntu-home:/fake-home/ubuntu:rw]
│                │ │ networks: {evaluation: {aliases: [evaluation]} }
│                │ solution-ego0:
│                │ dict[6]
│                │ │ image: docker.io/jeromelabonte/aido-submissions@sha256:04c7e311c0eaf6eb7705d26af59a86294addff2361cf6d4c9fb9c8ea981aa3c6
│                │ │ environment:
│                │ │ dict[13]
│                │ │ │ AIDONODE_NAME: ego0
│                │ │ │ AIDONODE_DATA_IN: /fifos/ego0-in
│                │ │ │ AIDO_REQUIRE_GPU: 1
│                │ │ │ AIDONODE_DATA_OUT: fifo:/fifos/ego0-out
│                │ │ │ challenge_name: aido5-LF-sim-validation
│                │ │ │ challenge_step_name: LFv-sim
│                │ │ │ submission_id: 9312
│                │ │ │ submitter_name: jeromelabonte
│                │ │ │ SUBMISSION_CONTAINER: docker.io/jeromelabonte/aido-submissions:2020_10_25_18_51_25@sha256:04c7e311c0eaf6eb7705d26af59a86294addff2361cf6d4c9fb9c8ea981aa3c6
│                │ │ │ username: ubuntu
│                │ │ │ uid: 0
│                │ │ │ USER: ubuntu
│                │ │ │ HOME: /fake-home/ubuntu
│                │ │ labels: {org.duckietown.created_by_runner: true, org.duckietown.runner_name: nogpu-prod-07_27d69f530ab0}
│                │ │ user: 0:0
│                │ │ volumes:
│                │ │ [
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission9312/LFv-sim-nogpu-prod-07_27d69f530ab0-job58246-a-wd:/challenges:rw,
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission9312/LFv-sim-nogpu-prod-07_27d69f530ab0-job58246-a-fifos:/fifos:rw,
│                │ │  /tmp/duckietown/dt-challenges-runner/20_12_03_22_29_20-39644/fake-ubuntu-home:/fake-home/ubuntu:rw]
│                │ │ networks: {evaluation: {aliases: [evaluation]} }
│           res: dict[2]
│                │ evaluator: 791a9a3ef6aceb8d2aba4ccff4f9b1973088d56feedaba8354b8b2b169111171
│                │ solution-ego0: e4a7c844e8f516a7dddb16e05f69cf540c45ebec690951b763316664626eb17a
│         names: dict[2]
│                │ 791a9a3ef6aceb8d2aba4ccff4f9b1973088d56feedaba8354b8b2b169111171: nogpu-prod-07_27d69f530ab0-job58246-666783_evaluator_1
│                │ e4a7c844e8f516a7dddb16e05f69cf540c45ebec690951b763316664626eb17a: nogpu-prod-07_27d69f530ab0-job58246-666783_solution-ego0_1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 959, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: |791a9a3ef6aceb8d2aba4ccff4f9b1973088d56feedaba8354b8b2b169111171
│         |e4a7c844e8f516a7dddb16e05f69cf540c45ebec690951b763316664626eb17a
│         |
│  names: dict[2]
│         │ 791a9a3ef6aceb8d2aba4ccff4f9b1973088d56feedaba8354b8b2b169111171: nogpu-prod-07_27d69f530ab0-job58246-666783_evaluator_1
│         │ e4a7c844e8f516a7dddb16e05f69cf540c45ebec690951b763316664626eb17a: nogpu-prod-07_27d69f530ab0-job58246-666783_solution-ego0_1

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52420

LFv-sim

error

2020-12-02 11:36:51+00:00

2020-12-02 11:46:44+00:00

0:09:53

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139866483119536
- M:video_aido:cmdline(in:/;out:/) 139866483120544
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52411

LFv-sim

error

2020-12-02 11:35:21+00:00

2020-12-02 11:39:08+00:00

0:03:47

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139751068943744
- M:video_aido:cmdline(in:/;out:/) 139751070121216
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52410

LFv-sim

timeout

2020-12-02 11:34:50+00:00

2020-12-02 11:48:12+00:00

0:13:22

Timeout because eval [...]

Timeout because evaluator contacted us

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52326

LFv-sim

host-error

2020-12-02 11:23:22+00:00

2020-12-02 11:30:29+00:00

0:07:07

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

41742

LFv-sim

success

2020-11-08 13:35:19+00:00

2020-11-08 13:44:51+00:00

0:09:32

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

41741

LFv-sim

success

2020-11-08 13:34:48+00:00

2020-11-08 13:44:13+00:00

0:09:25

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

41736

LFv-sim

success

2020-11-08 13:24:27+00:00

2020-11-08 13:34:18+00:00

0:09:51

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

41734

LFv-sim

success

2020-11-08 13:23:51+00:00

2020-11-08 13:34:06+00:00

0:10:15

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

38198

LFv-sim

error

2020-11-04 00:40:13+00:00

2020-11-04 00:40:48+00:00

0:00:35

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation/submission9312/LFv-sim-mont01-6ef51bb8a9d6-1-job38198-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

38196

LFv-sim

error

2020-11-04 00:39:38+00:00

2020-11-04 00:40:18+00:00

0:00:40

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation/submission9312/LFv-sim-mont02-80325a328f54-1-job38196-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

38195

LFv-sim

error

2020-11-04 00:39:26+00:00

2020-11-04 00:40:07+00:00

0:00:41

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation/submission9312/LFv-sim-mont04-e828c68b6a88-1-job38195-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

36332

LFv-sim

error

2020-11-02 04:29:46+00:00

2020-11-02 04:30:36+00:00

0:00:50

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

No results found! Something very wrong. 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1088, in run_one
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/challenge_results.py", line 90, in read_challenge_results
    raise NoResultsFound(msg)
duckietown_challenges.challenge_results.NoResultsFound: File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9312/LFv-sim-Sandy1-sandy-1-job36332-a-wd/challenge-results/challenge_results.yaml' does not exist.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

36330

LFv-sim

success

2020-11-02 04:29:32+00:00

2020-11-02 04:39:25+00:00

0:09:53

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35763

LFv-sim

success

2020-11-01 04:08:52+00:00

2020-11-01 04:10:01+00:00

0:01:09

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35761

LFv-sim

success

2020-11-01 04:08:21+00:00

2020-11-01 04:09:17+00:00

0:00:56

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35367

LFv-sim

error

2020-10-31 02:49:17+00:00

2020-10-31 03:11:45+00:00

0:22:28

The result file is n [...]

The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9312/LFv-sim-reg02-1b92df2e7e91-1-job35367:

File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9312/LFv-sim-reg02-1b92df2e7e91-1-job35367/challenge-results/challenge_results.yaml' does not exist.

This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.

List of all files:

 -/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9312/LFv-sim-reg02-1b92df2e7e91-1-job35367/docker-compose.original.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9312/LFv-sim-reg02-1b92df2e7e91-1-job35367/docker-compose.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9312/LFv-sim-reg02-1b92df2e7e91-1-job35367/logs/challenges-runner/stdout.log
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9312/LFv-sim-reg02-1b92df2e7e91-1-job35367/logs/challenges-runner/stderr.log

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35000

LFv-sim

success

2020-10-29 03:34:50+00:00

2020-10-29 03:59:04+00:00

0:24:14

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

34659

LFv-sim

success

2020-10-25 18:52:54+00:00

2020-10-25 19:18:18+00:00

0:25:24

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

34658

LFv-sim

success

2020-10-25 18:52:15+00:00

2020-10-25 19:17:10+00:00

0:24:55

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible