Duckietown Challenges Home Challenges Submissions

Submission 11648

Submission11648
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54145
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54145

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54145LFv-simsuccessyes0:27:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.441324189242913
survival_time_median47.39999999999944
deviation-center-line_median1.750933568982274
in-drivable-lane_median22.09999999999954


other stats
agent_compute-ego0_max0.012670824433912785
agent_compute-ego0_mean0.012274750593498873
agent_compute-ego0_median0.012285994627412733
agent_compute-ego0_min0.011856188685257242
complete-iteration_max0.2224809211630874
complete-iteration_mean0.19609289071749747
complete-iteration_median0.20446618526708976
complete-iteration_min0.15295827117272293
deviation-center-line_max2.5615704256576204
deviation-center-line_mean1.5492956468722747
deviation-center-line_min0.13374502386693107
deviation-heading_max10.39708941509469
deviation-heading_mean6.419089001327099
deviation-heading_median7.217271623354347
deviation-heading_min0.8447233435050125
driven_any_max11.354745352111063
driven_any_mean8.681084243401889
driven_any_median9.211422385372202
driven_any_min4.946746850752082
driven_lanedir_consec_max7.029069598066369
driven_lanedir_consec_mean4.661141507046227
driven_lanedir_consec_min0.7328480516327145
driven_lanedir_max7.029069598066369
driven_lanedir_mean4.661141507046227
driven_lanedir_median5.441324189242913
driven_lanedir_min0.7328480516327145
get_duckie_state_max1.547486411434287e-06
get_duckie_state_mean1.3774652634637243e-06
get_duckie_state_median1.3461448739760195e-06
get_duckie_state_min1.270084894468572e-06
get_robot_state_max0.004045412585122845
get_robot_state_mean0.0038509295513505255
get_robot_state_median0.0038932779497591274
get_robot_state_min0.003571749720761001
get_state_dump_max0.0050872801290872
get_state_dump_mean0.004894559868957942
get_state_dump_median0.004963277552972414
get_state_dump_min0.00456440424079974
get_ui_image_max0.035786355564139664
get_ui_image_mean0.030935473209759767
get_ui_image_median0.03181869777741822
get_ui_image_min0.02431814172006295
in-drivable-lane_max26.99999999999984
in-drivable-lane_mean21.79999999999957
in-drivable-lane_min15.999999999999368
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 9.097763951512134, "get_ui_image": 0.02952479317275009, "step_physics": 0.12007603229997964, "survival_time": 45.29999999999957, "driven_lanedir": 6.315271512802942, "get_state_dump": 0.005076520892461842, "get_robot_state": 0.004045412585122845, "sim_render-ego0": 0.0040908191012179575, "get_duckie_state": 1.547486411434287e-06, "in-drivable-lane": 15.999999999999368, "deviation-heading": 7.706349542356062, "agent_compute-ego0": 0.012289892878001422, "complete-iteration": 0.189376460152135, "set_robot_commands": 0.002405966524252423, "deviation-center-line": 2.0575672958597147, "driven_lanedir_consec": 6.315271512802942, "sim_compute_sim_state": 0.009513364737194193, "sim_compute_performance-ego0": 0.0022603934497265462}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.354745352111063, "get_ui_image": 0.035786355564139664, "step_physics": 0.140669824876555, "survival_time": 59.99999999999873, "driven_lanedir": 7.029069598066369, "get_state_dump": 0.004850034213482986, "get_robot_state": 0.003781676391677793, "sim_render-ego0": 0.003815849456660059, "get_duckie_state": 1.3298634982525955e-06, "in-drivable-lane": 23.44999999999887, "deviation-heading": 10.39708941509469, "agent_compute-ego0": 0.012282096376824042, "complete-iteration": 0.2195559103820445, "set_robot_commands": 0.002218622649142784, "deviation-center-line": 2.5615704256576204, "driven_lanedir_consec": 7.029069598066369, "sim_compute_sim_state": 0.013992869983009255, "sim_compute_performance-ego0": 0.002069970948015224}, "LF-norm-techtrack-000-ego0": {"driven_any": 9.325080819232271, "get_ui_image": 0.03411260238208636, "step_physics": 0.14402668242738417, "survival_time": 49.499999999999325, "driven_lanedir": 4.567376865682884, "get_state_dump": 0.0050872801290872, "get_robot_state": 0.004004879507840462, "sim_render-ego0": 0.0041213555244576676, "get_duckie_state": 1.3624262496994435e-06, "in-drivable-lane": 26.99999999999984, "deviation-heading": 6.728193704352632, "agent_compute-ego0": 0.012670824433912785, "complete-iteration": 0.2224809211630874, "set_robot_commands": 0.002455632212664838, "deviation-center-line": 1.4442998421048332, "driven_lanedir_consec": 4.567376865682884, "sim_compute_sim_state": 0.013611745401299686, "sim_compute_performance-ego0": 0.00229381961610796}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.946746850752082, "get_ui_image": 0.02431814172006295, "step_physics": 0.09492549343385558, "survival_time": 24.100000000000207, "driven_lanedir": 0.7328480516327145, "get_state_dump": 0.00456440424079974, "get_robot_state": 0.003571749720761001, "sim_render-ego0": 0.003604032484887796, "get_duckie_state": 1.270084894468572e-06, "in-drivable-lane": 20.75000000000021, "deviation-heading": 0.8447233435050125, "agent_compute-ego0": 0.011856188685257242, "complete-iteration": 0.15295827117272293, "set_robot_commands": 0.0021409257845355365, "deviation-center-line": 0.13374502386693107, "driven_lanedir_consec": 0.7328480516327145, "sim_compute_sim_state": 0.006029669048870079, "sim_compute_performance-ego0": 0.001869325558838143}}
set_robot_commands_max0.002455632212664838
set_robot_commands_mean0.002305286792648895
set_robot_commands_median0.0023122945866976036
set_robot_commands_min0.0021409257845355365
sim_compute_performance-ego0_max0.00229381961610796
sim_compute_performance-ego0_mean0.0021233773931719686
sim_compute_performance-ego0_median0.002165182198870885
sim_compute_performance-ego0_min0.001869325558838143
sim_compute_sim_state_max0.013992869983009255
sim_compute_sim_state_mean0.010786912292593305
sim_compute_sim_state_median0.011562555069246938
sim_compute_sim_state_min0.006029669048870079
sim_render-ego0_max0.0041213555244576676
sim_render-ego0_mean0.00390801414180587
sim_render-ego0_median0.003953334278939008
sim_render-ego0_min0.003604032484887796
simulation-passed1
step_physics_max0.14402668242738417
step_physics_mean0.1249245082594436
step_physics_median0.13037292858826732
step_physics_min0.09492549343385558
survival_time_max59.99999999999873
survival_time_mean44.724999999999454
survival_time_min24.100000000000207
No reset possible
49240LFv-simerrorno0:07:48
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139739876690720
- M:video_aido:cmdline(in:/;out:/) 139739876723104
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43255LFv-simsuccessno0:09:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43254LFv-simsuccessno0:06:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43253LFv-simsuccessno0:05:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible