Duckietown Challenges Home Challenges Submissions

Submission 11361

Submission11361
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54774
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

54774

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54774LFv-simsuccessyes0:11:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.3648764731352072
survival_time_median16.700000000000102
deviation-center-line_median0.42360461985215225
in-drivable-lane_median10.475000000000067


other stats
agent_compute-ego0_max0.012269513607025146
agent_compute-ego0_mean0.011982090776651404
agent_compute-ego0_median0.0119613933893053
agent_compute-ego0_min0.01173606272096987
complete-iteration_max0.19415080943496563
complete-iteration_mean0.1755437136642381
complete-iteration_median0.17705808573298984
complete-iteration_min0.15390787375600715
deviation-center-line_max1.027827776387302
deviation-center-line_mean0.5042019097031415
deviation-center-line_min0.14177062272095947
deviation-heading_max6.1614440939068045
deviation-heading_mean2.6919260665570026
deviation-heading_median2.01402981261738
deviation-heading_min0.5782005470864454
driven_any_max5.070364245218005
driven_any_mean3.1394702160710453
driven_any_median2.8956926835830243
driven_any_min1.696131251900129
driven_lanedir_consec_max2.1062163378509595
driven_lanedir_consec_mean1.3752234964373655
driven_lanedir_consec_min0.664924701628089
driven_lanedir_max2.1062163378509595
driven_lanedir_mean1.3795575372656532
driven_lanedir_median1.373544554791782
driven_lanedir_min0.664924701628089
get_duckie_state_max1.4331605699327257e-06
get_duckie_state_mean1.347763355421798e-06
get_duckie_state_median1.3440001046909666e-06
get_duckie_state_min1.2698926423725328e-06
get_robot_state_max0.003690125346183777
get_robot_state_mean0.003652832633662764
get_robot_state_median0.00364862954948684
get_robot_state_min0.0036239460894936007
get_state_dump_max0.00487422438906759
get_state_dump_mean0.004697189760784122
get_state_dump_median0.004650905234539485
get_state_dump_min0.004612724184989929
get_ui_image_max0.0348317860836709
get_ui_image_mean0.030086113762032348
get_ui_image_median0.02979865613910887
get_ui_image_min0.02591535668624075
in-drivable-lane_max18.450000000000127
in-drivable-lane_mean10.537500000000076
in-drivable-lane_min2.750000000000039
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.5228973422936005, "get_ui_image": 0.02811223930782742, "step_physics": 0.1009559640177974, "survival_time": 13.450000000000056, "driven_lanedir": 2.1062163378509595, "get_state_dump": 0.0046199948699386035, "get_robot_state": 0.003657266828748915, "sim_render-ego0": 0.0037617674580326783, "get_duckie_state": 1.4331605699327257e-06, "in-drivable-lane": 2.750000000000039, "deviation-heading": 1.6629935815362058, "agent_compute-ego0": 0.01173606272096987, "complete-iteration": 0.1666954861746894, "set_robot_commands": 0.00228169494205051, "deviation-center-line": 0.5516647753163715, "driven_lanedir_consec": 2.1062163378509595, "sim_compute_sim_state": 0.009497753779093425, "sim_compute_performance-ego0": 0.001981019973754883}, "LF-norm-zigzag-000-ego0": {"driven_any": 5.070364245218005, "get_ui_image": 0.0348317860836709, "step_physics": 0.1197605241101314, "survival_time": 33.05000000000026, "driven_lanedir": 1.977873541775945, "get_state_dump": 0.00487422438906759, "get_robot_state": 0.003639992270224764, "sim_render-ego0": 0.0038015143748856743, "get_duckie_state": 1.355240352204199e-06, "in-drivable-lane": 18.450000000000127, "deviation-heading": 6.1614440939068045, "agent_compute-ego0": 0.012104931194616588, "complete-iteration": 0.19415080943496563, "set_robot_commands": 0.002271039968530937, "deviation-center-line": 1.027827776387302, "driven_lanedir_consec": 1.9605373784627955, "sim_compute_sim_state": 0.010788595928526356, "sim_compute_performance-ego0": 0.001988452726980711}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.2684880248724477, "get_ui_image": 0.03148507297039032, "step_physics": 0.11707866072654724, "survival_time": 19.95000000000015, "driven_lanedir": 0.769215567807619, "get_state_dump": 0.004612724184989929, "get_robot_state": 0.003690125346183777, "sim_render-ego0": 0.0038772493600845335, "get_duckie_state": 1.3327598571777343e-06, "in-drivable-lane": 14.800000000000136, "deviation-heading": 2.365066043698554, "agent_compute-ego0": 0.012269513607025146, "complete-iteration": 0.18742068529129027, "set_robot_commands": 0.0022604089975357057, "deviation-center-line": 0.295544464387933, "driven_lanedir_consec": 0.769215567807619, "sim_compute_sim_state": 0.010050771236419676, "sim_compute_performance-ego0": 0.0020043933391571045}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.696131251900129, "get_ui_image": 0.02591535668624075, "step_physics": 0.09497395440151817, "survival_time": 9.45, "driven_lanedir": 0.664924701628089, "get_state_dump": 0.004681815599140368, "get_robot_state": 0.0036239460894936007, "sim_render-ego0": 0.0037590767207898593, "get_duckie_state": 1.2698926423725328e-06, "in-drivable-lane": 6.150000000000003, "deviation-heading": 0.5782005470864454, "agent_compute-ego0": 0.011817855583994012, "complete-iteration": 0.15390787375600715, "set_robot_commands": 0.0021511567266363847, "deviation-center-line": 0.14177062272095947, "driven_lanedir_consec": 0.664924701628089, "sim_compute_sim_state": 0.004979679459019711, "sim_compute_performance-ego0": 0.0019191478428087737}}
set_robot_commands_max0.00228169494205051
set_robot_commands_mean0.0022410751586883844
set_robot_commands_median0.0022657244830333215
set_robot_commands_min0.0021511567266363847
sim_compute_performance-ego0_max0.0020043933391571045
sim_compute_performance-ego0_mean0.001973253470675368
sim_compute_performance-ego0_median0.001984736350367797
sim_compute_performance-ego0_min0.0019191478428087737
sim_compute_sim_state_max0.010788595928526356
sim_compute_sim_state_mean0.008829200100764793
sim_compute_sim_state_median0.009774262507756553
sim_compute_sim_state_min0.004979679459019711
sim_render-ego0_max0.0038772493600845335
sim_render-ego0_mean0.0037999019784481862
sim_render-ego0_median0.003781640916459177
sim_render-ego0_min0.0037590767207898593
simulation-passed1
step_physics_max0.1197605241101314
step_physics_mean0.10819227581399855
step_physics_median0.10901731237217233
step_physics_min0.09497395440151817
survival_time_max33.05000000000026
survival_time_mean18.97500000000012
survival_time_min9.45
No reset possible
49972LFv-simerrorno0:10:40
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139939458576832
- M:video_aido:cmdline(in:/;out:/) 139939458579664
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49946LFv-simerrorno0:04:25
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140440623418672
- M:video_aido:cmdline(in:/;out:/) 140440623415984
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42053LFv-simsuccessno0:08:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42052LFv-simsuccessno0:08:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42050LFv-simsuccessno0:09:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42049LFv-simsuccessno0:09:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42047LFv-simsuccessno0:09:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible