Duckietown Challenges Home Challenges Submissions

Submission 11632

Submission11632
Competingyes
Challengeaido5-LF-sim-validation
UserMo Kleit 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54214
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54214

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54214LFv-simsuccessyes0:23:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.447054789472756
survival_time_median40.67499999999945
deviation-center-line_median1.3090631962496042
in-drivable-lane_median20.599999999999262


other stats
agent_compute-ego0_max0.012859270951458227
agent_compute-ego0_mean0.012315842856324505
agent_compute-ego0_median0.012322529164760891
agent_compute-ego0_min0.011759042144318008
complete-iteration_max0.2680914067776404
complete-iteration_mean0.23672776733038176
complete-iteration_median0.24180390605418856
complete-iteration_min0.19521185043550948
deviation-center-line_max2.430598536274395
deviation-center-line_mean1.3032202464430382
deviation-center-line_min0.16415605699854974
deviation-heading_max14.488716140264115
deviation-heading_mean7.438955331266209
deviation-heading_median6.81292289032485
deviation-heading_min1.6412594041510191
driven_any_max7.920398547596092
driven_any_mean4.83694429871109
driven_any_median5.304790486467768
driven_any_min0.8177976743127328
driven_lanedir_consec_max2.28662509580533
driven_lanedir_consec_mean1.4137934241066676
driven_lanedir_consec_min0.47443902167582896
driven_lanedir_max3.8274238332254016
driven_lanedir_mean1.9873430773283651
driven_lanedir_median1.8237547272061152
driven_lanedir_min0.47443902167582896
get_duckie_state_max1.5619759247681805e-06
get_duckie_state_mean1.3803096353097871e-06
get_duckie_state_median1.331077188398012e-06
get_duckie_state_min1.2971082396749457e-06
get_robot_state_max0.004008180070146222
get_robot_state_mean0.003836841489397784
get_robot_state_median0.003849209919467056
get_robot_state_min0.003640766048510803
get_state_dump_max0.005155836310342094
get_state_dump_mean0.004889259867327439
get_state_dump_median0.004870536408890823
get_state_dump_min0.004660130341186015
get_ui_image_max0.035785454717175716
get_ui_image_mean0.030970816545103672
get_ui_image_median0.03107283073865429
get_ui_image_min0.025952149985930404
in-drivable-lane_max33.49999999999916
in-drivable-lane_mean19.36249999999942
in-drivable-lane_min2.7499999999999902
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.920398547596092, "get_ui_image": 0.02801156540298144, "step_physics": 0.14928570913335465, "survival_time": 59.99999999999873, "driven_lanedir": 3.8274238332254016, "get_state_dump": 0.004991368588361017, "get_robot_state": 0.003851761329581001, "sim_render-ego0": 0.003905388437441048, "get_duckie_state": 1.3582513890198925e-06, "in-drivable-lane": 28.74999999999842, "deviation-heading": 9.553856784384935, "agent_compute-ego0": 0.012237645307250264, "complete-iteration": 0.2163448468731603, "set_robot_commands": 0.0023109883094806654, "deviation-center-line": 2.2125954373597803, "driven_lanedir_consec": 1.9607805951103852, "sim_compute_sim_state": 0.009581979565775266, "sim_compute_performance-ego0": 0.0020805352137150315}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.8177976743127328, "get_ui_image": 0.035785454717175716, "step_physics": 0.19191218902324808, "survival_time": 7.199999999999982, "driven_lanedir": 0.47443902167582896, "get_state_dump": 0.004749704229420629, "get_robot_state": 0.003846658509353111, "sim_render-ego0": 0.0037457663437415814, "get_duckie_state": 1.3039029877761317e-06, "in-drivable-lane": 2.7499999999999902, "deviation-heading": 1.6412594041510191, "agent_compute-ego0": 0.01240741302227152, "complete-iteration": 0.26726296523521686, "set_robot_commands": 0.002312686525542161, "deviation-center-line": 0.16415605699854974, "driven_lanedir_consec": 0.47443902167582896, "sim_compute_sim_state": 0.010421639475329168, "sim_compute_performance-ego0": 0.001992912950186894}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6895851963008117, "get_ui_image": 0.03413409607432713, "step_physics": 0.1934751879389041, "survival_time": 21.35000000000017, "driven_lanedir": 0.9333289838351266, "get_state_dump": 0.005155836310342094, "get_robot_state": 0.004008180070146222, "sim_render-ego0": 0.004109538047113151, "get_duckie_state": 1.5619759247681805e-06, "in-drivable-lane": 12.450000000000108, "deviation-heading": 4.071988996264764, "agent_compute-ego0": 0.012859270951458227, "complete-iteration": 0.2680914067776404, "set_robot_commands": 0.0024161583909364506, "deviation-center-line": 0.4055309551394284, "driven_lanedir_consec": 0.9333289838351266, "sim_compute_sim_state": 0.009627244182836227, "sim_compute_performance-ego0": 0.002205371856689453}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.919995776634725, "get_ui_image": 0.025952149985930404, "step_physics": 0.13513721971090986, "survival_time": 59.99999999999873, "driven_lanedir": 2.714180470577104, "get_state_dump": 0.004660130341186015, "get_robot_state": 0.003640766048510803, "sim_render-ego0": 0.003715303120863229, "get_duckie_state": 1.2971082396749457e-06, "in-drivable-lane": 33.49999999999916, "deviation-heading": 14.488716140264115, "agent_compute-ego0": 0.011759042144318008, "complete-iteration": 0.19521185043550948, "set_robot_commands": 0.0022239887545646778, "deviation-center-line": 2.430598536274395, "driven_lanedir_consec": 2.28662509580533, "sim_compute_sim_state": 0.0060952078988411144, "sim_compute_performance-ego0": 0.00194470451634492}}
set_robot_commands_max0.0024161583909364506
set_robot_commands_mean0.002315955495130989
set_robot_commands_median0.002311837417511413
set_robot_commands_min0.0022239887545646778
sim_compute_performance-ego0_max0.002205371856689453
sim_compute_performance-ego0_mean0.0020558811342340747
sim_compute_performance-ego0_median0.0020367240819509627
sim_compute_performance-ego0_min0.00194470451634492
sim_compute_sim_state_max0.010421639475329168
sim_compute_sim_state_mean0.008931517780695446
sim_compute_sim_state_median0.009604611874305748
sim_compute_sim_state_min0.0060952078988411144
sim_render-ego0_max0.004109538047113151
sim_render-ego0_mean0.003868998987289752
sim_render-ego0_median0.003825577390591314
sim_render-ego0_min0.003715303120863229
simulation-passed1
step_physics_max0.1934751879389041
step_physics_mean0.1674525764516042
step_physics_median0.17059894907830137
step_physics_min0.13513721971090986
survival_time_max59.99999999999873
survival_time_mean37.137499999999406
survival_time_min7.199999999999982
No reset possible
54206LFv-simsuccessyes0:16:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54203LFv-simsuccessyes0:17:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49316LFv-simerrorno0:09:31
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139777821217888
- M:video_aido:cmdline(in:/;out:/) 139777999315248
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49304LFv-simerrorno0:03:04
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140207430600928
- M:video_aido:cmdline(in:/;out:/) 140207432169840
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43203LFv-simsuccessno0:06:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43202LFv-simsuccessno0:08:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible