Duckietown Challenges Home Challenges Submissions

Submission 11729

Submission11729
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 53908
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

53908

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
53908LFv-simsuccessyes0:26:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.869064475659838
survival_time_median49.77499999999931
deviation-center-line_median0.914007491848232
in-drivable-lane_median26.37499999999945


other stats
agent_compute-ego0_max0.013176634150877484
agent_compute-ego0_mean0.012746709374061615
agent_compute-ego0_median0.012678755625446824
agent_compute-ego0_min0.012452692094475332
complete-iteration_max0.2834378955810051
complete-iteration_mean0.22389618576170983
complete-iteration_median0.2099152074864663
complete-iteration_min0.1923164324929016
deviation-center-line_max2.3311101415752726
deviation-center-line_mean1.101240158517741
deviation-center-line_min0.24583550879922733
deviation-heading_max8.478659451594144
deviation-heading_mean4.936405601220808
deviation-heading_median5.177647373470299
deviation-heading_min0.9116682063484914
driven_any_max10.705656314114764
driven_any_mean7.0273229015088665
driven_any_median8.397747940884118
driven_any_min0.6081394101524684
driven_lanedir_consec_max4.098245300329343
driven_lanedir_consec_mean2.0491899572097827
driven_lanedir_consec_min0.36038557719011255
driven_lanedir_max4.980166353734681
driven_lanedir_mean2.5830801760747732
driven_lanedir_median2.495884386687149
driven_lanedir_min0.36038557719011255
get_duckie_state_max1.58317083125309e-06
get_duckie_state_mean1.4338703943621616e-06
get_duckie_state_median1.395217461572861e-06
get_duckie_state_min1.3618758230498342e-06
get_robot_state_max0.004289998102942474
get_robot_state_mean0.004062208853497657
get_robot_state_median0.003996828608199502
get_robot_state_min0.003965180094649152
get_state_dump_max0.0053470567501554085
get_state_dump_mean0.005191946120158086
get_state_dump_median0.005168106706795288
get_state_dump_min0.005084514316886363
get_ui_image_max0.037640823581354406
get_ui_image_mean0.03289914561020159
get_ui_image_median0.03273372864544541
get_ui_image_min0.02848830156856113
in-drivable-lane_max53.19999999999884
in-drivable-lane_mean27.34999999999943
in-drivable-lane_min3.4499999999999877
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.705656314114764, "get_ui_image": 0.029836376541163105, "step_physics": 0.12373394314990652, "survival_time": 59.99999999999873, "driven_lanedir": 4.025075920749424, "get_state_dump": 0.00518555605441307, "get_robot_state": 0.004026932283603182, "sim_render-ego0": 0.004105998316374945, "get_duckie_state": 1.58317083125309e-06, "in-drivable-lane": 39.89999999999907, "deviation-heading": 5.476723420684913, "agent_compute-ego0": 0.012582716199380967, "complete-iteration": 0.19312001167983436, "set_robot_commands": 0.0024159675236050037, "deviation-center-line": 1.31480178236387, "driven_lanedir_consec": 2.8857201492533227, "sim_compute_sim_state": 0.008915302854692013, "sim_compute_performance-ego0": 0.0022202282523632448}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6081394101524684, "get_ui_image": 0.037640823581354406, "step_physics": 0.20379653015756996, "survival_time": 6.099999999999986, "driven_lanedir": 0.36038557719011255, "get_state_dump": 0.005150657359177504, "get_robot_state": 0.003965180094649152, "sim_render-ego0": 0.004022780472670144, "get_duckie_state": 1.3859291386798146e-06, "in-drivable-lane": 3.4499999999999877, "deviation-heading": 0.9116682063484914, "agent_compute-ego0": 0.01277479505151268, "complete-iteration": 0.2834378955810051, "set_robot_commands": 0.0023935771569973084, "deviation-center-line": 0.24583550879922733, "driven_lanedir_consec": 0.36038557719011255, "sim_compute_sim_state": 0.011454630673416262, "sim_compute_performance-ego0": 0.0021458776985726707}, "LF-norm-techtrack-000-ego0": {"driven_any": 9.93112313379931, "get_ui_image": 0.035631080749727706, "step_physics": 0.1469905013148731, "survival_time": 59.99999999999873, "driven_lanedir": 0.9666928526248748, "get_state_dump": 0.0053470567501554085, "get_robot_state": 0.004289998102942474, "sim_render-ego0": 0.004311088320615389, "get_duckie_state": 1.4045057844659074e-06, "in-drivable-lane": 53.19999999999884, "deviation-heading": 4.878571326255685, "agent_compute-ego0": 0.013176634150877484, "complete-iteration": 0.22671040329309824, "set_robot_commands": 0.002556927297434938, "deviation-center-line": 0.5132132013325938, "driven_lanedir_consec": 0.8524088020663536, "sim_compute_sim_state": 0.011886709238666976, "sim_compute_performance-ego0": 0.0024219645151587746}, "LF-norm-small_loop-000-ego0": {"driven_any": 6.864372747968925, "get_ui_image": 0.02848830156856113, "step_physics": 0.12691100558849297, "survival_time": 39.54999999999989, "driven_lanedir": 4.980166353734681, "get_state_dump": 0.005084514316886363, "get_robot_state": 0.003966724932795823, "sim_render-ego0": 0.004070744972036342, "get_duckie_state": 1.3618758230498342e-06, "in-drivable-lane": 12.849999999999824, "deviation-heading": 8.478659451594144, "agent_compute-ego0": 0.012452692094475332, "complete-iteration": 0.1923164324929016, "set_robot_commands": 0.002369453509648641, "deviation-center-line": 2.3311101415752726, "driven_lanedir_consec": 4.098245300329343, "sim_compute_sim_state": 0.006756340313439417, "sim_compute_performance-ego0": 0.002124960675384059}}
set_robot_commands_max0.002556927297434938
set_robot_commands_mean0.0024339813719214728
set_robot_commands_median0.002404772340301156
set_robot_commands_min0.002369453509648641
sim_compute_performance-ego0_max0.0024219645151587746
sim_compute_performance-ego0_mean0.0022282577853696877
sim_compute_performance-ego0_median0.0021830529754679577
sim_compute_performance-ego0_min0.002124960675384059
sim_compute_sim_state_max0.011886709238666976
sim_compute_sim_state_mean0.009753245770053667
sim_compute_sim_state_median0.010184966764054138
sim_compute_sim_state_min0.006756340313439417
sim_render-ego0_max0.004311088320615389
sim_render-ego0_mean0.004127653020424205
sim_render-ego0_median0.004088371644205644
sim_render-ego0_min0.004022780472670144
simulation-passed1
step_physics_max0.20379653015756996
step_physics_mean0.15035799505271064
step_physics_median0.13695075345168303
step_physics_min0.12373394314990652
survival_time_max59.99999999999873
survival_time_mean41.41249999999933
survival_time_min6.099999999999986
No reset possible
48797LFv-simerrorno0:05:21
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139855594073296
- M:video_aido:cmdline(in:/;out:/) 139855594057584
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43748LFv-simsuccessno0:07:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43747LFv-simsuccessno0:06:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43746LFv-simsuccessno0:10:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible