Duckietown Challenges Home Challenges Submissions

Submission 11043

Submission11043
Competingyes
Challengeaido5-LF-sim-validation
UserBhavya Patwa 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 56978
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

56978

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
56978LFv-simsuccessyes0:24:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.007649848251626
survival_time_median43.44999999999949
deviation-center-line_median2.230262288450494
in-drivable-lane_median15.100000000000064


other stats
agent_compute-ego0_max0.013471111273154234
agent_compute-ego0_mean0.01269643763879629
agent_compute-ego0_median0.012661606137879035
agent_compute-ego0_min0.01199142700627285
complete-iteration_max0.1988029291487148
complete-iteration_mean0.17462190831304236
complete-iteration_median0.17504621200667564
complete-iteration_min0.14959228009010334
deviation-center-line_max3.682005120147053
deviation-center-line_mean2.1529569750084687
deviation-center-line_min0.4692982029858335
deviation-heading_max15.067072115098089
deviation-heading_mean7.316215017861106
deviation-heading_median6.183697182521184
deviation-heading_min1.8303935913039668
driven_any_max9.120682832596817
driven_any_mean6.599802347842903
driven_any_median6.708498913566562
driven_any_min3.861528731641671
driven_lanedir_consec_max6.618478447442525
driven_lanedir_consec_mean4.568968780377495
driven_lanedir_consec_min1.6420969775642025
driven_lanedir_max6.618478447442525
driven_lanedir_mean4.568968780377495
driven_lanedir_median5.007649848251626
driven_lanedir_min1.6420969775642025
get_duckie_state_max1.29750527311225e-06
get_duckie_state_mean1.2357322676760135e-06
get_duckie_state_median1.2578325758704265e-06
get_duckie_state_min1.129758645850951e-06
get_robot_state_max0.003783094577300243
get_robot_state_mean0.00360044057351456
get_robot_state_median0.003547665826224346
get_robot_state_min0.0035233360643093037
get_state_dump_max0.004859626802623781
get_state_dump_mean0.004658762327438383
get_state_dump_median0.004639592791847995
get_state_dump_min0.00449623692343376
get_ui_image_max0.03340108745997395
get_ui_image_mean0.029463901579080368
get_ui_image_median0.029764761975857923
get_ui_image_min0.02492499490463168
in-drivable-lane_max22.699999999999896
in-drivable-lane_mean15.61250000000004
in-drivable-lane_min9.550000000000136
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.070749526092123, "get_ui_image": 0.027067668805095837, "step_physics": 0.09448922719937752, "survival_time": 26.900000000000247, "driven_lanedir": 3.832261375010652, "get_state_dump": 0.004635258375603104, "get_robot_state": 0.003524496294350704, "sim_render-ego0": 0.003718839726775387, "get_duckie_state": 1.2522504590659062e-06, "in-drivable-lane": 9.550000000000136, "deviation-heading": 2.373760469431734, "agent_compute-ego0": 0.012934702004483987, "complete-iteration": 0.16052899776451662, "set_robot_commands": 0.002073679430365341, "deviation-center-line": 0.90085460684676, "driven_lanedir_consec": 3.832261375010652, "sim_compute_sim_state": 0.010104234674202487, "sim_compute_performance-ego0": 0.001893546452991153}, "LF-norm-zigzag-000-ego0": {"driven_any": 8.346248301041001, "get_ui_image": 0.03340108745997395, "step_physics": 0.11654675274864024, "survival_time": 59.99999999999873, "driven_lanedir": 6.618478447442525, "get_state_dump": 0.004643927208092886, "get_robot_state": 0.0035233360643093037, "sim_render-ego0": 0.003694888455583888, "get_duckie_state": 1.29750527311225e-06, "in-drivable-lane": 14.299999999999931, "deviation-heading": 15.067072115098089, "agent_compute-ego0": 0.01238851027127408, "complete-iteration": 0.18956342624883468, "set_robot_commands": 0.002039447215871946, "deviation-center-line": 3.682005120147053, "driven_lanedir_consec": 6.618478447442525, "sim_compute_sim_state": 0.011333775460769692, "sim_compute_performance-ego0": 0.0019043833885859889}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.861528731641671, "get_ui_image": 0.032461855146620006, "step_physics": 0.12411641921752538, "survival_time": 23.350000000000197, "driven_lanedir": 1.6420969775642025, "get_state_dump": 0.004859626802623781, "get_robot_state": 0.003783094577300243, "sim_render-ego0": 0.004123487533667149, "get_duckie_state": 1.2634146926749463e-06, "in-drivable-lane": 15.900000000000194, "deviation-heading": 1.8303935913039668, "agent_compute-ego0": 0.013471111273154234, "complete-iteration": 0.1988029291487148, "set_robot_commands": 0.0022768378257751465, "deviation-center-line": 0.4692982029858335, "driven_lanedir_consec": 1.6420969775642025, "sim_compute_sim_state": 0.011502129909319756, "sim_compute_performance-ego0": 0.002112887863420014}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.120682832596817, "get_ui_image": 0.02492499490463168, "step_physics": 0.09105022285105684, "survival_time": 59.99999999999873, "driven_lanedir": 6.1830383214926, "get_state_dump": 0.00449623692343376, "get_robot_state": 0.003570835358097988, "sim_render-ego0": 0.003652149195675052, "get_duckie_state": 1.129758645850951e-06, "in-drivable-lane": 22.699999999999896, "deviation-heading": 9.993633895610635, "agent_compute-ego0": 0.01199142700627285, "complete-iteration": 0.14959228009010334, "set_robot_commands": 0.0021380933496378345, "deviation-center-line": 3.559669970054228, "driven_lanedir_consec": 6.1830383214926, "sim_compute_sim_state": 0.00579483522959891, "sim_compute_performance-ego0": 0.0018923810677762632}}
set_robot_commands_max0.0022768378257751465
set_robot_commands_mean0.002132014455412567
set_robot_commands_median0.002105886390001588
set_robot_commands_min0.002039447215871946
sim_compute_performance-ego0_max0.002112887863420014
sim_compute_performance-ego0_mean0.0019507996931933547
sim_compute_performance-ego0_median0.0018989649207885708
sim_compute_performance-ego0_min0.0018923810677762632
sim_compute_sim_state_max0.011502129909319756
sim_compute_sim_state_mean0.009683743818472711
sim_compute_sim_state_median0.01071900506748609
sim_compute_sim_state_min0.00579483522959891
sim_render-ego0_max0.004123487533667149
sim_render-ego0_mean0.0037973412279253688
sim_render-ego0_median0.0037068640911796374
sim_render-ego0_min0.003652149195675052
simulation-passed1
step_physics_max0.12411641921752538
step_physics_mean0.10655065550415
step_physics_median0.10551798997400888
step_physics_min0.09105022285105684
survival_time_max59.99999999999873
survival_time_mean42.562499999999474
survival_time_min23.350000000000197
No reset possible
50835LFv-simerrorno0:06:29
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140633622893328
- M:video_aido:cmdline(in:/;out:/) 140633623193968
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50826LFv-simhost-errorno0:08:48
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50815LFv-simerrorno0:04:02
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140650645724176
- M:video_aido:cmdline(in:/;out:/) 140650645668576
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50814LFv-simerrorno0:06:50
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139934643480560
- M:video_aido:cmdline(in:/;out:/) 139934647886656
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40380LFv-simsuccessno0:08:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40378LFv-simsuccessno0:08:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40377LFv-simsuccessno0:07:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40376LFv-simsuccessno0:07:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39720LFv-simsuccessno0:07:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38799LFv-simsuccessno0:08:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38798LFv-simsuccessno0:08:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible