Duckietown Challenges Home Challenges Submissions

Submission 10950

Submission10950
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57360
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57360

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57360LFv-simsuccessyes0:24:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.525730693386784
survival_time_median33.87500000000014
deviation-center-line_median1.7228346055931645
in-drivable-lane_median6.575000000000069


other stats
agent_compute-ego0_max0.034505917208038105
agent_compute-ego0_mean0.01766893461144623
agent_compute-ego0_median0.012276718489635048
agent_compute-ego0_min0.01161638425847672
complete-iteration_max0.20409221150887707
complete-iteration_mean0.1785965363301596
complete-iteration_median0.17863765705859125
complete-iteration_min0.15301861969457875
deviation-center-line_max2.4064858575818904
deviation-center-line_mean1.5212832737446096
deviation-center-line_min0.2329780262102198
deviation-heading_max10.3979120796542
deviation-heading_mean6.541766622623263
deviation-heading_median7.39076209189082
deviation-heading_min0.9876302270572148
driven_any_max8.325496194863941
driven_any_mean4.770651550106515
driven_any_median4.524635689089345
driven_any_min1.707838627383428
driven_lanedir_consec_max7.61508142925005
driven_lanedir_consec_mean3.8302949423404047
driven_lanedir_consec_min0.654636953338003
driven_lanedir_max7.61508142925005
driven_lanedir_mean3.8302949423404047
driven_lanedir_median3.525730693386784
driven_lanedir_min0.654636953338003
get_duckie_state_max1.3967670460857411e-06
get_duckie_state_mean1.31887196123168e-06
get_duckie_state_median1.301029890593617e-06
get_duckie_state_min1.2766610176537457e-06
get_robot_state_max0.0037354836388239785
get_robot_state_mean0.003590749537692862
get_robot_state_median0.003555459826506095
get_robot_state_min0.0035165948589352797
get_state_dump_max0.004615206007754971
get_state_dump_mean0.004539616070640952
get_state_dump_median0.00452748114791942
get_state_dump_min0.004488295978969998
get_ui_image_max0.03513398498454422
get_ui_image_mean0.029687084937278833
get_ui_image_median0.02900980737310689
get_ui_image_min0.025594740018357327
in-drivable-lane_max8.449999999999712
in-drivable-lane_mean6.1624999999999615
in-drivable-lane_min3.049999999999997
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.325496194863941, "get_ui_image": 0.026457179694449672, "step_physics": 0.09003195059885093, "survival_time": 59.99999999999873, "driven_lanedir": 7.61508142925005, "get_state_dump": 0.004615206007754971, "get_robot_state": 0.0035250689167464207, "sim_render-ego0": 0.00364643667857911, "get_duckie_state": 1.2766610176537457e-06, "in-drivable-lane": 3.049999999999997, "deviation-heading": 10.3979120796542, "agent_compute-ego0": 0.01161638425847672, "complete-iteration": 0.15301861969457875, "set_robot_commands": 0.0021117593128417156, "deviation-center-line": 2.4064858575818904, "driven_lanedir_consec": 7.61508142925005, "sim_compute_sim_state": 0.00907586079453747, "sim_compute_performance-ego0": 0.0018565388742235677}, "LF-norm-zigzag-000-ego0": {"driven_any": 5.052186584345446, "get_ui_image": 0.03513398498454422, "step_physics": 0.12950018505570748, "survival_time": 37.74999999999999, "driven_lanedir": 3.7804120828958343, "get_state_dump": 0.004488295978969998, "get_robot_state": 0.0037354836388239785, "sim_render-ego0": 0.003822165822225904, "get_duckie_state": 1.3967670460857411e-06, "in-drivable-lane": 8.449999999999712, "deviation-heading": 8.346488276008976, "agent_compute-ego0": 0.012299591586703346, "complete-iteration": 0.20409221150887707, "set_robot_commands": 0.002197013960944282, "deviation-center-line": 1.897898177376698, "driven_lanedir_consec": 3.7804120828958343, "sim_compute_sim_state": 0.010867418435515553, "sim_compute_performance-ego0": 0.001960661045457951}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.997084793833243, "get_ui_image": 0.03156243505176411, "step_physics": 0.11652384899221918, "survival_time": 30.00000000000029, "driven_lanedir": 3.2710493038777333, "get_state_dump": 0.004557515142761332, "get_robot_state": 0.0035858507362657695, "sim_render-ego0": 0.003741564250825447, "get_duckie_state": 1.3055499897225329e-06, "in-drivable-lane": 4.800000000000068, "deviation-heading": 6.435035907772663, "agent_compute-ego0": 0.012253845392566753, "complete-iteration": 0.18693762928396215, "set_robot_commands": 0.0021753656288946727, "deviation-center-line": 1.5477710338096304, "driven_lanedir_consec": 3.2710493038777333, "sim_compute_sim_state": 0.010530774486243427, "sim_compute_performance-ego0": 0.001918190925966285}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.707838627383428, "get_ui_image": 0.025594740018357327, "step_physics": 0.08915010333931359, "survival_time": 13.65000000000006, "driven_lanedir": 0.654636953338003, "get_state_dump": 0.004497447153077508, "get_robot_state": 0.0035165948589352797, "sim_render-ego0": 0.003683699308520686, "get_duckie_state": 1.2965097914647012e-06, "in-drivable-lane": 8.350000000000069, "deviation-heading": 0.9876302270572148, "agent_compute-ego0": 0.034505917208038105, "complete-iteration": 0.17033768483322032, "set_robot_commands": 0.0021454208958758053, "deviation-center-line": 0.2329780262102198, "driven_lanedir_consec": 0.654636953338003, "sim_compute_sim_state": 0.005076317891587306, "sim_compute_performance-ego0": 0.00208166219892293}}
set_robot_commands_max0.002197013960944282
set_robot_commands_mean0.002157389949639119
set_robot_commands_median0.002160393262385239
set_robot_commands_min0.0021117593128417156
sim_compute_performance-ego0_max0.00208166219892293
sim_compute_performance-ego0_mean0.0019542632611426834
sim_compute_performance-ego0_median0.001939425985712118
sim_compute_performance-ego0_min0.0018565388742235677
sim_compute_sim_state_max0.010867418435515553
sim_compute_sim_state_mean0.008887592901970939
sim_compute_sim_state_median0.009803317640390448
sim_compute_sim_state_min0.005076317891587306
sim_render-ego0_max0.003822165822225904
sim_render-ego0_mean0.0037234665150377862
sim_render-ego0_median0.0037126317796730662
sim_render-ego0_min0.00364643667857911
simulation-passed1
step_physics_max0.12950018505570748
step_physics_mean0.1063015219965228
step_physics_median0.10327789979553506
step_physics_min0.08915010333931359
survival_time_max59.99999999999873
survival_time_mean35.34999999999977
survival_time_min13.65000000000006
No reset possible
51121LFv-simerrorno0:04:34
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139641201132496
- M:video_aido:cmdline(in:/;out:/) 139641201767472
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51113LFv-simerrorno0:05:10
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139730103863760
- M:video_aido:cmdline(in:/;out:/) 139731216137328
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40560LFv-simsuccessno0:11:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39847LFv-simsuccessno0:13:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38530LFv-simsuccessno0:11:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38529LFv-simsuccessno0:08:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible