Submission 6846

Submission

6846

Competing

yes

Challenge

aido5-LF-sim-validation

User

Daniil Lisus

Date submitted

2020-10-18 21:13:11+00:00

Last status update

2021-12-04 20:52:37+00:00

Complete

complete

Details

Evaluation is complete.

Sisters

Result

💚

Jobs

LFv-sim: 58537

User label

exercise_ros_template

Admin priority

Blessing

n/a

User priority

58537

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID

step

status

up to date

date started

date completed

duration

message

58537

LFv-sim

success

yes

2020-12-04 00:17:09+00:00

2020-12-04 00:44:24+00:00

0:27:15

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	2.504840754270347
survival_time_median	59.99999999999873
deviation-center-line_median	1.216546680069829
in-drivable-lane_median	6.350000000000046

other stats

agent_compute-ego0_max	0.03360922191661164
agent_compute-ego0_mean	0.018005855233487494
agent_compute-ego0_median	0.012885236833698755
agent_compute-ego0_min	0.012643725349940824
complete-iteration_max	0.21255385055743828
complete-iteration_mean	0.18694583498150025
complete-iteration_median	0.17924625937091024
complete-iteration_min	0.17673697062674212
deviation-center-line_max	1.8696151790215392
deviation-center-line_mean	1.1388641753608
deviation-center-line_min	0.2527481622820023
deviation-heading_max	9.376568774460342
deviation-heading_mean	5.133420742076749
deviation-heading_median	4.814821146762741
deviation-heading_min	1.5274719003211694
driven_any_max	6.19228006790988
driven_any_mean	3.903836869838584
driven_any_median	4.3920875509474016
driven_any_min	0.6388923095496536
driven_lanedir_consec_max	5.856121864907712
driven_lanedir_consec_mean	2.77182132963326
driven_lanedir_consec_min	0.22148194508463437
driven_lanedir_max	5.856121864907712
driven_lanedir_mean	2.77182132963326
driven_lanedir_median	2.504840754270347
driven_lanedir_min	0.22148194508463437
get_duckie_state_max	1.4773614202113473e-06
get_duckie_state_mean	1.3903732847332704e-06
get_duckie_state_median	1.3721711922682586e-06
get_duckie_state_min	1.3397893341852168e-06
get_robot_state_max	0.0041073293709735096
get_robot_state_mean	0.0038982782520961393
get_robot_state_median	0.0038548519371153418
get_robot_state_min	0.003776079763180364
get_state_dump_max	0.005022576210600053
get_state_dump_mean	0.004929371341841128
get_state_dump_median	0.004921446476181225
get_state_dump_min	0.004852016204402012
get_ui_image_max	0.03636138275186851
get_ui_image_mean	0.031205173904510337
get_ui_image_median	0.031151778890528743
get_ui_image_min	0.02615575508511533
in-drivable-lane_max	52.799999999998704
in-drivable-lane_mean	17.737499999999663
in-drivable-lane_min	5.449999999999857
per-episodes	details {"LF-norm-loop-000-ego0": {"driven_any": 6.19228006790988, "get_ui_image": 0.02914650235743844, "step_physics": 0.10618111076799658, "survival_time": 59.99999999999873, "driven_lanedir": 5.856121864907712, "get_state_dump": 0.005022576210600053, "get_robot_state": 0.0041073293709735096, "sim_render-ego0": 0.004138564587036438, "get_duckie_state": 1.4773614202113473e-06, "in-drivable-lane": 5.449999999999857, "deviation-heading": 7.665145585116451, "agent_compute-ego0": 0.012643725349940824, "complete-iteration": 0.17673697062674212, "set_robot_commands": 0.0024342616332956994, "deviation-center-line": 1.8696151790215392, "driven_lanedir_consec": 5.856121864907712, "sim_compute_sim_state": 0.010667463226381884, "sim_compute_performance-ego0": 0.0022955018217418713}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6388923095496536, "get_ui_image": 0.03636138275186851, "step_physics": 0.13582155565736154, "survival_time": 9.4, "driven_lanedir": 0.22148194508463437, "get_state_dump": 0.004863843716010846, "get_robot_state": 0.003776079763180364, "sim_render-ego0": 0.0038604698483906096, "get_duckie_state": 1.385098411923363e-06, "in-drivable-lane": 6.350000000000001, "deviation-heading": 1.5274719003211694, "agent_compute-ego0": 0.012670108250209264, "complete-iteration": 0.21255385055743828, "set_robot_commands": 0.002219241762918139, "deviation-center-line": 0.2527481622820023, "driven_lanedir_consec": 0.22148194508463437, "sim_compute_sim_state": 0.010841932246293972, "sim_compute_performance-ego0": 0.0020434957332712}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.124149694540122, "get_ui_image": 0.03315705542361905, "step_physics": 0.1051024731549494, "survival_time": 59.99999999999873, "driven_lanedir": 4.524801239057554, "get_state_dump": 0.004979049236351604, "get_robot_state": 0.0038856671910599606, "sim_render-ego0": 0.003977253276243695, "get_duckie_state": 1.3397893341852168e-06, "in-drivable-lane": 6.35000000000009, "deviation-heading": 9.376568774460342, "agent_compute-ego0": 0.013100365417188252, "complete-iteration": 0.18151831884963823, "set_robot_commands": 0.002339146317887763, "deviation-center-line": 1.8414822001524016, "driven_lanedir_consec": 4.524801239057554, "sim_compute_sim_state": 0.012729516136556936, "sim_compute_performance-ego0": 0.0021456703357553603}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.66002540735468, "get_ui_image": 0.02615575508511533, "step_physics": 0.094111645052971, "survival_time": 59.99999999999873, "driven_lanedir": 0.4848802694831395, "get_state_dump": 0.004852016204402012, "get_robot_state": 0.003824036683170722, "sim_render-ego0": 0.0038447719529506865, "get_duckie_state": 1.3592439726131543e-06, "in-drivable-lane": 52.799999999998704, "deviation-heading": 1.9644967084090297, "agent_compute-ego0": 0.03360922191661164, "complete-iteration": 0.17697419989218224, "set_robot_commands": 0.0022697605558676484, "deviation-center-line": 0.5916111599872568, "driven_lanedir_consec": 0.4848802694831395, "sim_compute_sim_state": 0.006226941806688396, "sim_compute_performance-ego0": 0.001987554945616202}}
set_robot_commands_max	0.0024342616332956994
set_robot_commands_mean	0.002315602567492312
set_robot_commands_median	0.0023044534368777056
set_robot_commands_min	0.002219241762918139
sim_compute_performance-ego0_max	0.0022955018217418713
sim_compute_performance-ego0_mean	0.0021180557090961583
sim_compute_performance-ego0_median	0.00209458303451328
sim_compute_performance-ego0_min	0.001987554945616202
sim_compute_sim_state_max	0.012729516136556936
sim_compute_sim_state_mean	0.010116463353980296
sim_compute_sim_state_median	0.010754697736337929
sim_compute_sim_state_min	0.006226941806688396
sim_render-ego0_max	0.004138564587036438
sim_render-ego0_mean	0.003955264916155357
sim_render-ego0_median	0.003918861562317152
sim_render-ego0_min	0.0038447719529506865
simulation-passed	1
step_physics_max	0.13582155565736154
step_physics_mean	0.11030419615831964
step_physics_median	0.105641791961473
step_physics_min	0.094111645052971
survival_time_max	59.99999999999873
survival_time_mean	47.34999999999904
survival_time_min	9.4

No reset possible

58535

LFv-sim

success

yes

2020-12-04 00:16:39+00:00

2020-12-04 00:42:53+00:00

0:26:14

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

58532

LFv-sim

success

yes

2020-12-04 00:16:09+00:00

2020-12-04 00:42:27+00:00

0:26:18

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

58530

LFv-sim

success

yes

2020-12-04 00:15:34+00:00

2020-12-04 00:41:30+00:00

0:25:56

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

58529

LFv-sim

success

yes

2020-12-04 00:15:01+00:00

2020-12-04 00:47:48+00:00

0:32:47

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52477

LFv-sim

error

2020-12-02 11:45:03+00:00

2020-12-02 11:51:01+00:00

0:05:58

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140099056048304
- M:video_aido:cmdline(in:/;out:/) 140099038266752
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52474

LFv-sim

error

2020-12-02 11:44:21+00:00

2020-12-02 11:53:18+00:00

0:08:57

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140057197104624
- M:video_aido:cmdline(in:/;out:/) 140057197102224
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52473

LFv-sim

error

2020-12-02 11:44:14+00:00

2020-12-02 11:53:26+00:00

0:09:12

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140150798530016
- M:video_aido:cmdline(in:/;out:/) 140150798818608
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

52458

LFv-sim

error

2020-12-02 11:42:45+00:00

2020-12-02 11:54:52+00:00

0:12:07

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140490020962800
- M:video_aido:cmdline(in:/;out:/) 140490020965728
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

41793

LFv-sim

success

2020-11-08 14:52:31+00:00

2020-11-08 15:02:03+00:00

0:09:32

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

38346

LFv-sim

success

2020-11-04 02:32:14+00:00

2020-11-04 02:48:55+00:00

0:16:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

38345

LFv-sim

success

2020-11-04 02:31:18+00:00

2020-11-04 02:40:43+00:00

0:09:25

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

36445

LFv-sim

success

2020-11-02 05:40:04+00:00

2020-11-02 05:50:20+00:00

0:10:16

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35865

LFv-sim

success

2020-11-01 04:32:13+00:00

2020-11-01 04:33:15+00:00

0:01:02

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35863

LFv-sim

success

2020-11-01 04:31:39+00:00

2020-11-01 04:32:40+00:00

0:01:01

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35443

LFv-sim

error

2020-10-31 07:38:02+00:00

2020-10-31 08:00:45+00:00

0:22:43

The result file is n [...]

The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission6846/LFv-sim-reg05-b2dee9d94ee0-1-job35443:

File '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission6846/LFv-sim-reg05-b2dee9d94ee0-1-job35443/challenge-results/challenge_results.yaml' does not exist.

This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.

List of all files:

 -/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission6846/LFv-sim-reg05-b2dee9d94ee0-1-job35443/docker-compose.original.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission6846/LFv-sim-reg05-b2dee9d94ee0-1-job35443/docker-compose.yaml
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission6846/LFv-sim-reg05-b2dee9d94ee0-1-job35443/logs/challenges-runner/stdout.log
-/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission6846/LFv-sim-reg05-b2dee9d94ee0-1-job35443/logs/challenges-runner/stderr.log

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

35127

LFv-sim

success

2020-10-29 13:22:39+00:00

2020-10-29 13:46:10+00:00

0:23:31

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

33462

LFv-sim

success

2020-10-19 20:42:18+00:00

2020-10-19 21:09:06+00:00

0:26:48

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

33441

LFv-sim

success

2020-10-18 21:14:22+00:00

2020-10-18 21:33:30+00:00

0:19:08

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

33440

LFv-sim

success

2020-10-18 21:14:03+00:00

2020-10-18 21:34:52+00:00

0:20:49

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible