Duckietown Challenges Home Challenges Submissions

Submission 10919

Submission10919
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57475
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57475

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57475LFv-simsuccessyes0:13:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.0205158783526302
survival_time_median9.374999999999998
deviation-center-line_median0.3959608457174308
in-drivable-lane_median4.950000000000007


other stats
agent_compute-ego0_max0.03808654515967579
agent_compute-ego0_mean0.022715707526345615
agent_compute-ego0_median0.020336190284188115
agent_compute-ego0_min0.012103904377330431
complete-iteration_max0.25623914480209353
complete-iteration_mean0.20866116530570572
complete-iteration_median0.20377537508011773
complete-iteration_min0.1708547662604939
deviation-center-line_max2.9429110939472567
deviation-center-line_mean0.966230480907686
deviation-center-line_min0.1300891382486256
deviation-heading_max14.848183249575785
deviation-heading_mean4.753725676190616
deviation-heading_median1.566921267916356
deviation-heading_min1.0328769193539653
driven_any_max13.253724983973957
driven_any_mean4.42288344767309
driven_any_median1.8820778836877805
driven_any_min0.6736530393428456
driven_lanedir_consec_max10.06228504916547
driven_lanedir_consec_mean3.08845630991674
driven_lanedir_consec_min0.2505084337962318
driven_lanedir_max10.062243453410844
driven_lanedir_mean3.088445910978084
driven_lanedir_median1.0205158783526302
driven_lanedir_min0.2505084337962318
get_duckie_state_max1.4142556623979048e-06
get_duckie_state_mean1.3695062649789952e-06
get_duckie_state_median1.3888890470438337e-06
get_duckie_state_min1.2859913034304096e-06
get_robot_state_max0.003916659758458683
get_robot_state_mean0.0036857125323559456
get_robot_state_median0.003664090355468928
get_robot_state_min0.003498009660027244
get_state_dump_max0.004816657868190784
get_state_dump_mean0.004670733617034098
get_state_dump_median0.004645687460800095
get_state_dump_min0.00457490167834542
get_ui_image_max0.03689300775527954
get_ui_image_mean0.031288385534358684
get_ui_image_median0.03107694179924943
get_ui_image_min0.02610665078365634
in-drivable-lane_max15.399999999999718
in-drivable-lane_mean6.999999999999931
in-drivable-lane_min2.6999999999999904
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 1.8027020583485216, "get_ui_image": 0.0279060344804417, "step_physics": 0.10630778832869096, "survival_time": 8.74999999999999, "driven_lanedir": 0.939086770821679, "get_state_dump": 0.00457490167834542, "get_robot_state": 0.003498009660027244, "sim_render-ego0": 0.003744478930126537, "get_duckie_state": 1.4142556623979048e-06, "in-drivable-lane": 4.549999999999999, "deviation-heading": 1.3854866078043155, "agent_compute-ego0": 0.012103904377330431, "complete-iteration": 0.1708547662604939, "set_robot_commands": 0.0020784234458749943, "deviation-center-line": 0.4411826543740469, "driven_lanedir_consec": 0.939086770821679, "sim_compute_sim_state": 0.0086368742314252, "sim_compute_performance-ego0": 0.0019226114858280528}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6736530393428456, "get_ui_image": 0.03689300775527954, "step_physics": 0.16465172052383423, "survival_time": 4.94999999999999, "driven_lanedir": 0.2505084337962318, "get_state_dump": 0.004705202579498291, "get_robot_state": 0.003716902732849121, "sim_render-ego0": 0.003987674713134766, "get_duckie_state": 1.3875961303710938e-06, "in-drivable-lane": 2.6999999999999904, "deviation-heading": 1.0328769193539653, "agent_compute-ego0": 0.0272412371635437, "complete-iteration": 0.25623914480209353, "set_robot_commands": 0.002262399196624756, "deviation-center-line": 0.1300891382486256, "driven_lanedir_consec": 0.2505084337962318, "sim_compute_sim_state": 0.010590815544128418, "sim_compute_performance-ego0": 0.0021012425422668457}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.9614537090270392, "get_ui_image": 0.03424784911805717, "step_physics": 0.14321017265319824, "survival_time": 10.000000000000009, "driven_lanedir": 1.1019449858835817, "get_state_dump": 0.004816657868190784, "get_robot_state": 0.003916659758458683, "sim_render-ego0": 0.004122159967375039, "get_duckie_state": 1.3901819637165733e-06, "in-drivable-lane": 5.350000000000016, "deviation-heading": 1.7483559280283962, "agent_compute-ego0": 0.013431143404832526, "complete-iteration": 0.21833240214865007, "set_robot_commands": 0.002356416550441761, "deviation-center-line": 0.35073903706081466, "driven_lanedir_consec": 1.1019449858835817, "sim_compute_sim_state": 0.010010142824543057, "sim_compute_performance-ego0": 0.0021308310589387047}, "LF-norm-small_loop-000-ego0": {"driven_any": 13.253724983973957, "get_ui_image": 0.02610665078365634, "step_physics": 0.10274384103150888, "survival_time": 59.99999999999873, "driven_lanedir": 10.062243453410844, "get_state_dump": 0.004586172342101898, "get_robot_state": 0.0036112779780887345, "sim_render-ego0": 0.003788148235222581, "get_duckie_state": 1.2859913034304096e-06, "in-drivable-lane": 15.399999999999718, "deviation-heading": 14.848183249575785, "agent_compute-ego0": 0.03808654515967579, "complete-iteration": 0.1892183480115854, "set_robot_commands": 0.00223427827312587, "deviation-center-line": 2.9429110939472567, "driven_lanedir_consec": 10.06228504916547, "sim_compute_sim_state": 0.006055071590147248, "sim_compute_performance-ego0": 0.001921591214792218}}
set_robot_commands_max0.002356416550441761
set_robot_commands_mean0.002232879366516845
set_robot_commands_median0.0022483387348753133
set_robot_commands_min0.0020784234458749943
sim_compute_performance-ego0_max0.0021308310589387047
sim_compute_performance-ego0_mean0.0020190690754564554
sim_compute_performance-ego0_median0.002011927014047449
sim_compute_performance-ego0_min0.001921591214792218
sim_compute_sim_state_max0.010590815544128418
sim_compute_sim_state_mean0.00882322604756098
sim_compute_sim_state_median0.009323508527984128
sim_compute_sim_state_min0.006055071590147248
sim_render-ego0_max0.004122159967375039
sim_render-ego0_mean0.00391061546146473
sim_render-ego0_median0.003887911474178673
sim_render-ego0_min0.003744478930126537
simulation-passed1
step_physics_max0.16465172052383423
step_physics_mean0.12922838063430808
step_physics_median0.1247589804909446
step_physics_min0.10274384103150888
survival_time_max59.99999999999873
survival_time_mean20.924999999999677
survival_time_min4.94999999999999
No reset possible
57462LFv-simsuccessyes0:30:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57459LFv-simtimeoutyes0:23:40
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57456LFv-simsuccessyes0:19:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51242LFv-simerrorno0:02:53
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139668624215584
- M:video_aido:cmdline(in:/;out:/) 139668624214432
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51238LFv-simerrorno0:09:17
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140474617101280
- M:video_aido:cmdline(in:/;out:/) 140474617101376
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51227LFv-simerrorno0:02:15
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140337793666496
- M:video_aido:cmdline(in:/;out:/) 140337793666304
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40607LFv-simsuccessno0:11:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40606LFv-simsuccessno0:11:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40605LFv-simsuccessno0:09:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39891LFv-simsuccessno0:10:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38470LFv-simsuccessno0:07:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38469LFv-simsuccessno0:07:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible