Duckietown Challenges Home Challenges Submissions

Submission 11277

Submission11277
Competingyes
Challengeaido5-LF-sim-validation
UserJacopo Tani
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 55277
Next
User labeltemplate-random
Admin priority50
Blessingn/a
User priority50

55277

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
55277LFv-simsuccessyes0:05:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5040495128145347
survival_time_median4.6499999999999915
deviation-center-line_median0.10620228737371736
in-drivable-lane_median2.674999999999992


other stats
agent_compute-ego0_max0.011867049667570326
agent_compute-ego0_mean0.010935102942127056
agent_compute-ego0_median0.010726242550188982
agent_compute-ego0_min0.01042087700055993
complete-iteration_max0.20008949093196704
complete-iteration_mean0.17993790905535156
complete-iteration_median0.18142132088541985
complete-iteration_min0.1568195035185995
deviation-center-line_max0.16455763165774434
deviation-center-line_mean0.1092777488823554
deviation-center-line_min0.06014878912424249
deviation-heading_max1.819139548495243
deviation-heading_mean0.860079214882594
deviation-heading_median0.6517793617099584
deviation-heading_min0.31761858761521594
driven_any_max3.078535752374254
driven_any_mean1.8526223552312917
driven_any_median1.6219947547238958
driven_any_min1.0879641591031215
driven_lanedir_consec_max0.6643282900394009
driven_lanedir_consec_mean0.4884432809313369
driven_lanedir_consec_min0.28134580805687714
driven_lanedir_max0.6643282900394009
driven_lanedir_mean0.4924956463812869
driven_lanedir_median0.5121542437144349
driven_lanedir_min0.28134580805687714
get_duckie_state_max1.3642840915256077e-06
get_duckie_state_mean1.2522697856956496e-06
get_duckie_state_median1.2462192927997193e-06
get_duckie_state_min1.152356465657552e-06
get_robot_state_max0.003818561633427938
get_robot_state_mean0.003678061218409118
get_robot_state_median0.0036799683087113976
get_robot_state_min0.0035337466227857374
get_state_dump_max0.0049455927477942575
get_state_dump_mean0.004666623297165506
get_state_dump_median0.004642317786749908
get_state_dump_min0.004436264867367951
get_ui_image_max0.03479878021323163
get_ui_image_mean0.02990970699953656
get_ui_image_median0.029149940030442342
get_ui_image_min0.02654016772402993
in-drivable-lane_max6.549999999999982
in-drivable-lane_mean3.3499999999999894
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.078535752374254, "get_ui_image": 0.02654016772402993, "step_physics": 0.09487897534913654, "survival_time": 7.84999999999998, "driven_lanedir": 0.28134580805687714, "get_state_dump": 0.004453674147400675, "get_robot_state": 0.0035337466227857374, "sim_render-ego0": 0.003675257103352607, "get_duckie_state": 1.2796136397349684e-06, "in-drivable-lane": 6.549999999999982, "deviation-heading": 0.891006057417327, "agent_compute-ego0": 0.010515695885766911, "complete-iteration": 0.1568195035185995, "set_robot_commands": 0.002066904985452, "deviation-center-line": 0.13094886067496936, "driven_lanedir_consec": 0.28134580805687714, "sim_compute_sim_state": 0.009131998955448972, "sim_compute_performance-ego0": 0.0019425168822083292}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5806581102909054, "get_ui_image": 0.03479878021323163, "step_physics": 0.13015729188919067, "survival_time": 4.549999999999992, "driven_lanedir": 0.3891269085945981, "get_state_dump": 0.004436264867367951, "get_robot_state": 0.003614062848298446, "sim_render-ego0": 0.003654137901637865, "get_duckie_state": 1.2128249458644702e-06, "in-drivable-lane": 2.449999999999993, "deviation-heading": 1.819139548495243, "agent_compute-ego0": 0.01042087700055993, "complete-iteration": 0.20008949093196704, "set_robot_commands": 0.0021688393924547277, "deviation-center-line": 0.16455763165774434, "driven_lanedir_consec": 0.37291744679479777, "sim_compute_sim_state": 0.008850304976753567, "sim_compute_performance-ego0": 0.0019074626590894616}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0879641591031215, "get_ui_image": 0.03057069910897149, "step_physics": 0.13235809405644736, "survival_time": 3.5499999999999954, "driven_lanedir": 0.6643282900394009, "get_state_dump": 0.0049455927477942575, "get_robot_state": 0.003745873769124349, "sim_render-ego0": 0.0037884116172790527, "get_duckie_state": 1.3642840915256077e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.41255266600258983, "agent_compute-ego0": 0.011867049667570326, "complete-iteration": 0.19919786850611368, "set_robot_commands": 0.002221733331680298, "deviation-center-line": 0.08145571407246535, "driven_lanedir_consec": 0.6643282900394009, "sim_compute_sim_state": 0.0076276858647664385, "sim_compute_performance-ego0": 0.001985341310501098}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6633313991568863, "get_ui_image": 0.027729180951913197, "step_physics": 0.10282010833422343, "survival_time": 4.749999999999991, "driven_lanedir": 0.6351815788342716, "get_state_dump": 0.004830961426099141, "get_robot_state": 0.003818561633427938, "sim_render-ego0": 0.003998669485251109, "get_duckie_state": 1.152356465657552e-06, "in-drivable-lane": 2.89999999999999, "deviation-heading": 0.31761858761521594, "agent_compute-ego0": 0.010936789214611052, "complete-iteration": 0.163644773264726, "set_robot_commands": 0.0023340235153834024, "deviation-center-line": 0.06014878912424249, "driven_lanedir_consec": 0.6351815788342716, "sim_compute_sim_state": 0.005004676679770152, "sim_compute_performance-ego0": 0.002087761958440145}}
set_robot_commands_max0.0023340235153834024
set_robot_commands_mean0.002197875306242607
set_robot_commands_median0.0021952863620675125
set_robot_commands_min0.002066904985452
sim_compute_performance-ego0_max0.002087761958440145
sim_compute_performance-ego0_mean0.0019807707025597584
sim_compute_performance-ego0_median0.001963929096354714
sim_compute_performance-ego0_min0.0019074626590894616
sim_compute_sim_state_max0.009131998955448972
sim_compute_sim_state_mean0.007653666619184782
sim_compute_sim_state_median0.008238995420760002
sim_compute_sim_state_min0.005004676679770152
sim_render-ego0_max0.003998669485251109
sim_render-ego0_mean0.0037791190268801583
sim_render-ego0_median0.00373183436031583
sim_render-ego0_min0.003654137901637865
simulation-passed1
step_physics_max0.13235809405644736
step_physics_mean0.11505361740724952
step_physics_median0.11648870011170706
step_physics_min0.09487897534913654
survival_time_max7.84999999999998
survival_time_mean5.17499999999999
survival_time_min3.5499999999999954
No reset possible
50231LFv-simerrorno0:03:55
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139877341678608
- M:video_aido:cmdline(in:/;out:/) 139877341678224
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50204LFv-simerrorno0:04:13
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139668524045360
- M:video_aido:cmdline(in:/;out:/) 139668524045312
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50189LFv-simerrorno0:04:49
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140590052247536
- M:video_aido:cmdline(in:/;out:/) 140590052247776
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40863LFv-simsuccessno0:04:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40862LFv-simsuccessno0:04:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible