Duckietown Challenges Home Challenges Submissions

Submission 10759

Submission10759
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57922
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57922

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57922LFv-simsuccessyes0:09:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9867306319733088
survival_time_median13.575000000000058
deviation-center-line_median0.24420743547193435
in-drivable-lane_median6.3250000000000455


other stats
agent_compute-ego0_max0.0139010793284366
agent_compute-ego0_mean0.012988223747203225
agent_compute-ego0_median0.012775231997172036
agent_compute-ego0_min0.012501351666032222
complete-iteration_max0.22479266166687012
complete-iteration_mean0.1895397046745489
complete-iteration_median0.185831688370621
complete-iteration_min0.16170278029008345
deviation-center-line_max0.8127384461972961
deviation-center-line_mean0.36320992860110063
deviation-center-line_min0.15168639726323777
deviation-heading_max3.634379566479617
deviation-heading_mean1.956516168506468
deviation-heading_median1.7917631944913612
deviation-heading_min0.6081587185635318
driven_any_max5.283466383620802
driven_any_mean3.202275528115158
driven_any_median2.9948640273804092
driven_any_min1.5359076740790103
driven_lanedir_consec_max3.462549769917823
driven_lanedir_consec_mean1.5023177959763154
driven_lanedir_consec_min0.573260150040821
driven_lanedir_max3.462549769917823
driven_lanedir_mean1.5023177959763154
driven_lanedir_median0.9867306319733088
driven_lanedir_min0.573260150040821
get_duckie_state_max1.3582634203361743e-06
get_duckie_state_mean1.312120204146779e-06
get_duckie_state_median1.3167502587301689e-06
get_duckie_state_min1.2567168787906045e-06
get_robot_state_max0.0036517209128329622
get_robot_state_mean0.003569790804119962
get_robot_state_median0.00354901113008198
get_robot_state_min0.003529420043482925
get_state_dump_max0.004597518318577817
get_state_dump_mean0.004537394240712435
get_state_dump_median0.004536528640576717
get_state_dump_min0.00447900136311849
get_ui_image_max0.035842498143514
get_ui_image_mean0.030866332245215275
get_ui_image_median0.03067301354910198
get_ui_image_min0.02627680373914314
in-drivable-lane_max12.300000000000086
in-drivable-lane_mean7.35000000000004
in-drivable-lane_min4.449999999999984
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.283466383620802, "get_ui_image": 0.028319641163474636, "step_physics": 0.10574992759185924, "survival_time": 22.75000000000019, "driven_lanedir": 3.462549769917823, "get_state_dump": 0.00458107496562757, "get_robot_state": 0.0036517209128329622, "sim_render-ego0": 0.0037582067021152423, "get_duckie_state": 1.2888197313275254e-06, "in-drivable-lane": 7.550000000000107, "deviation-heading": 3.634379566479617, "agent_compute-ego0": 0.012501351666032222, "complete-iteration": 0.17351341770406356, "set_robot_commands": 0.0022168337253102087, "deviation-center-line": 0.8127384461972961, "driven_lanedir_consec": 3.462549769917823, "sim_compute_sim_state": 0.01067308741703368, "sim_compute_performance-ego0": 0.001979465547360872}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5359076740790103, "get_ui_image": 0.035842498143514, "step_physics": 0.15070663452148436, "survival_time": 7.449999999999981, "driven_lanedir": 0.573260150040821, "get_state_dump": 0.00447900136311849, "get_robot_state": 0.0035344362258911133, "sim_render-ego0": 0.0036566940943400066, "get_duckie_state": 1.3446807861328124e-06, "in-drivable-lane": 4.449999999999984, "deviation-heading": 1.1690300227414114, "agent_compute-ego0": 0.01270792007446289, "complete-iteration": 0.22479266166687012, "set_robot_commands": 0.002080128987630208, "deviation-center-line": 0.1756996914610268, "driven_lanedir_consec": 0.573260150040821, "sim_compute_sim_state": 0.00983016808827718, "sim_compute_performance-ego0": 0.001870698928833008}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.270330269764891, "get_ui_image": 0.03302638593472933, "step_physics": 0.12385045540960211, "survival_time": 18.950000000000134, "driven_lanedir": 1.3314231659677562, "get_state_dump": 0.004597518318577817, "get_robot_state": 0.0035635860342728464, "sim_render-ego0": 0.00379884870428788, "get_duckie_state": 1.2567168787906045e-06, "in-drivable-lane": 12.300000000000086, "deviation-heading": 2.4144963662413113, "agent_compute-ego0": 0.0139010793284366, "complete-iteration": 0.19814995903717844, "set_robot_commands": 0.0021345050711380808, "deviation-center-line": 0.3127151794828419, "driven_lanedir_consec": 1.3314231659677562, "sim_compute_sim_state": 0.011239405054795114, "sim_compute_performance-ego0": 0.001956752099488911}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.719397784995928, "get_ui_image": 0.02627680373914314, "step_physics": 0.10164015220873283, "survival_time": 8.199999999999982, "driven_lanedir": 0.6420380979788614, "get_state_dump": 0.004491982315525864, "get_robot_state": 0.003529420043482925, "sim_render-ego0": 0.003675677559592507, "get_duckie_state": 1.3582634203361743e-06, "in-drivable-lane": 5.0999999999999845, "deviation-heading": 0.6081587185635318, "agent_compute-ego0": 0.012842543919881185, "complete-iteration": 0.16170278029008345, "set_robot_commands": 0.0020867997949773616, "deviation-center-line": 0.15168639726323777, "driven_lanedir_consec": 0.6420380979788614, "sim_compute_sim_state": 0.005176583203402432, "sim_compute_performance-ego0": 0.0018981268911650688}}
set_robot_commands_max0.0022168337253102087
set_robot_commands_mean0.002129566894763965
set_robot_commands_median0.002110652433057721
set_robot_commands_min0.002080128987630208
sim_compute_performance-ego0_max0.001979465547360872
sim_compute_performance-ego0_mean0.0019262608667119647
sim_compute_performance-ego0_median0.00192743949532699
sim_compute_performance-ego0_min0.001870698928833008
sim_compute_sim_state_max0.011239405054795114
sim_compute_sim_state_mean0.009229810940877103
sim_compute_sim_state_median0.01025162775265543
sim_compute_sim_state_min0.005176583203402432
sim_render-ego0_max0.00379884870428788
sim_render-ego0_mean0.003722356765083909
sim_render-ego0_median0.0037169421308538746
sim_render-ego0_min0.0036566940943400066
simulation-passed1
step_physics_max0.15070663452148436
step_physics_mean0.12048679243291964
step_physics_median0.11480019150073068
step_physics_min0.10164015220873283
survival_time_max22.75000000000019
survival_time_mean14.337500000000071
survival_time_min7.449999999999981
No reset possible
51784LFv-simerrorno0:02:28
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140377592973776
- M:video_aido:cmdline(in:/;out:/) 140377592973728
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51781LFv-simerrorno0:03:03
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140011985207456
- M:video_aido:cmdline(in:/;out:/) 140011985066304
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51766LFv-simhost-errorno0:03:41
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51764LFv-simhost-errorno0:05:16
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40997LFv-simsuccessno0:07:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40993LFv-simsuccessno0:08:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37119LFv-simsuccessno0:10:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37118LFv-simsuccessno0:08:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible