Duckietown Challenges Home Challenges Submissions

Submission 10972

Submission10972
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57267
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57267

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57267LFv-simsuccessyes0:18:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.496292100212059
survival_time_median26.00000000000022
deviation-center-line_median0.7637009692307526
in-drivable-lane_median13.825000000000175


other stats
agent_compute-ego0_max0.013324005954761796
agent_compute-ego0_mean0.01303744664039436
agent_compute-ego0_median0.0130753339885027
agent_compute-ego0_min0.01267511262981024
complete-iteration_max0.2203128150215855
complete-iteration_mean0.19588847050088457
complete-iteration_median0.19824268711776685
complete-iteration_min0.1667556927464189
deviation-center-line_max2.825354997071291
deviation-center-line_mean1.2555162659787769
deviation-center-line_min0.669308128382311
deviation-heading_max12.815434218249733
deviation-heading_mean5.5800170933590465
deviation-heading_median4.096175935492331
deviation-heading_min1.3122822842017876
driven_any_max12.496717454685609
driven_any_mean6.256868512341309
driven_any_median5.140942704734899
driven_any_min2.248871185209828
driven_lanedir_consec_max6.735429793153686
driven_lanedir_consec_mean3.3560070393979684
driven_lanedir_consec_min1.6960141640140705
driven_lanedir_max6.735429793153686
driven_lanedir_mean3.387874136384603
driven_lanedir_median2.560026294185328
driven_lanedir_min1.6960141640140705
get_duckie_state_max1.9354640312914577e-06
get_duckie_state_mean1.7177579617913756e-06
get_duckie_state_median1.7289283620410631e-06
get_duckie_state_min1.4777110917919175e-06
get_robot_state_max0.004009549264554624
get_robot_state_mean0.003902052351403656
get_robot_state_median0.003947541701383785
get_robot_state_min0.0037035767382924303
get_state_dump_max0.005272764079975632
get_state_dump_mean0.005067668773412126
get_state_dump_median0.005041329150563861
get_state_dump_min0.004915252712545149
get_ui_image_max0.03606987109890691
get_ui_image_mean0.030296485127387328
get_ui_image_median0.02969379221990001
get_ui_image_min0.02572848497084237
in-drivable-lane_max26.29999999999966
in-drivable-lane_mean14.62500000000001
in-drivable-lane_min4.550000000000028
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.248871185209828, "get_ui_image": 0.027462818712558387, "step_physics": 0.12927824933573884, "survival_time": 10.550000000000017, "driven_lanedir": 1.6960141640140705, "get_state_dump": 0.005272764079975632, "get_robot_state": 0.00400516447031273, "sim_render-ego0": 0.004324107799889907, "get_duckie_state": 1.9354640312914577e-06, "in-drivable-lane": 4.550000000000028, "deviation-heading": 1.3122822842017876, "agent_compute-ego0": 0.012836846540559011, "complete-iteration": 0.19809616174338, "set_robot_commands": 0.00230756458246483, "deviation-center-line": 0.7124473090383996, "driven_lanedir_consec": 1.6960141640140705, "sim_compute_sim_state": 0.010249561858627031, "sim_compute_performance-ego0": 0.0022378383942370144}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.0169862222795185, "get_ui_image": 0.03606987109890691, "step_physics": 0.14088060237743236, "survival_time": 32.3500000000003, "driven_lanedir": 3.0975289122227543, "get_state_dump": 0.0050730252707446065, "get_robot_state": 0.004009549264554624, "sim_render-ego0": 0.00424106584654914, "get_duckie_state": 1.9036693337522907e-06, "in-drivable-lane": 16.8000000000002, "deviation-heading": 5.293091774705537, "agent_compute-ego0": 0.01331382143644639, "complete-iteration": 0.2203128150215855, "set_robot_commands": 0.0023685127128789452, "deviation-center-line": 0.8149546294231056, "driven_lanedir_consec": 3.0975289122227543, "sim_compute_sim_state": 0.011934548248479396, "sim_compute_performance-ego0": 0.002296766013274958}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.26489918719028, "get_ui_image": 0.031924765727241636, "step_physics": 0.12503308451115178, "survival_time": 19.650000000000144, "driven_lanedir": 2.022523676147902, "get_state_dump": 0.005009633030383115, "get_robot_state": 0.00388991893245484, "sim_render-ego0": 0.004154023180153164, "get_duckie_state": 1.4777110917919175e-06, "in-drivable-lane": 10.850000000000152, "deviation-heading": 2.8992600962791264, "agent_compute-ego0": 0.013324005954761796, "complete-iteration": 0.19838921249215372, "set_robot_commands": 0.0023263475011447965, "deviation-center-line": 0.669308128382311, "driven_lanedir_consec": 1.895055288201364, "sim_compute_sim_state": 0.010485520217624411, "sim_compute_performance-ego0": 0.002136255883928483}, "LF-norm-small_loop-000-ego0": {"driven_any": 12.496717454685609, "get_ui_image": 0.02572848497084237, "step_physics": 0.10535965612984816, "survival_time": 59.99999999999873, "driven_lanedir": 6.735429793153686, "get_state_dump": 0.004915252712545149, "get_robot_state": 0.0037035767382924303, "sim_render-ego0": 0.00389615741002371, "get_duckie_state": 1.5541873903298358e-06, "in-drivable-lane": 26.29999999999966, "deviation-heading": 12.815434218249733, "agent_compute-ego0": 0.01267511262981024, "complete-iteration": 0.1667556927464189, "set_robot_commands": 0.002204699679080096, "deviation-center-line": 2.825354997071291, "driven_lanedir_consec": 6.735429793153686, "sim_compute_sim_state": 0.006136305028453259, "sim_compute_performance-ego0": 0.002031911918265337}}
set_robot_commands_max0.0023685127128789452
set_robot_commands_mean0.002301781118892167
set_robot_commands_median0.0023169560418048133
set_robot_commands_min0.002204699679080096
sim_compute_performance-ego0_max0.002296766013274958
sim_compute_performance-ego0_mean0.002175693052426448
sim_compute_performance-ego0_median0.002187047139082749
sim_compute_performance-ego0_min0.002031911918265337
sim_compute_sim_state_max0.011934548248479396
sim_compute_sim_state_mean0.009701483838296028
sim_compute_sim_state_median0.010367541038125722
sim_compute_sim_state_min0.006136305028453259
sim_render-ego0_max0.004324107799889907
sim_render-ego0_mean0.004153838559153981
sim_render-ego0_median0.0041975445133511525
sim_render-ego0_min0.00389615741002371
simulation-passed1
step_physics_max0.14088060237743236
step_physics_mean0.1251378980885428
step_physics_median0.12715566692344532
step_physics_min0.10535965612984816
survival_time_max59.99999999999873
survival_time_mean30.637499999999797
survival_time_min10.550000000000017
No reset possible
51057LFv-simerrorno0:03:43
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139689942410720
- M:video_aido:cmdline(in:/;out:/) 139689942413168
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51043LFv-simhost-errorno0:02:51
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51038LFv-simerrorno0:02:18
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139701690966464
- M:video_aido:cmdline(in:/;out:/) 139701726114768
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40518LFv-simsuccessno0:09:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39810LFv-simsuccessno0:10:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38574LFv-simsuccessno0:08:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38573LFv-simsuccessno0:11:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible