Duckietown Challenges Home Challenges Submissions

Submission 10931

Submission10931
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57407
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57407

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57407LFv-simsuccessyes0:36:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.009278116783561
survival_time_median59.99999999999873
deviation-center-line_median1.2766452253152436
in-drivable-lane_median15.274999999999222


other stats
agent_compute-ego0_max0.04911403790997228
agent_compute-ego0_mean0.0380165204715967
agent_compute-ego0_median0.040701506735383225
agent_compute-ego0_min0.021549030505648065
complete-iteration_max0.2222801662305313
complete-iteration_mean0.20020298899262276
complete-iteration_median0.1959986681743625
complete-iteration_min0.18653445339123473
deviation-center-line_max2.703965477861179
deviation-center-line_mean1.4475825377679412
deviation-center-line_min0.5330742225800983
deviation-heading_max10.239805832874453
deviation-heading_mean5.942806256709256
deviation-heading_median5.753878315367835
deviation-heading_min2.0236625632269
driven_any_max8.126009877813281
driven_any_mean7.225065236874509
driven_any_median7.080495627571432
driven_any_min6.613259814541894
driven_lanedir_consec_max7.348429343220604
driven_lanedir_consec_mean4.8512152579430525
driven_lanedir_consec_min2.037875454984482
driven_lanedir_max7.348429343220604
driven_lanedir_mean4.8512152579430525
driven_lanedir_median5.009278116783561
driven_lanedir_min2.037875454984482
get_duckie_state_max1.365596507610032e-06
get_duckie_state_mean1.2631122516057176e-06
get_duckie_state_median1.253434561571412e-06
get_duckie_state_min1.1799833756700146e-06
get_robot_state_max0.003751207450148863
get_robot_state_mean0.0036929872609693538
get_robot_state_median0.003697307282542309
get_robot_state_min0.003626127028643936
get_state_dump_max0.004859600734154847
get_state_dump_mean0.00465777702474475
get_state_dump_median0.004627498560007367
get_state_dump_min0.00451651024480942
get_ui_image_max0.035582093573132724
get_ui_image_mean0.03104515486613202
get_ui_image_median0.03066477777558898
get_ui_image_min0.027268970340217386
in-drivable-lane_max44.99999999999865
in-drivable-lane_mean18.887499999999275
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.126009877813281, "get_ui_image": 0.02878536213248298, "step_physics": 0.09360512150614386, "survival_time": 59.99999999999873, "driven_lanedir": 2.037875454984482, "get_state_dump": 0.004623474030570126, "get_robot_state": 0.00370266495100366, "sim_render-ego0": 0.003993963818069699, "get_duckie_state": 1.365596507610032e-06, "in-drivable-lane": 44.99999999999865, "deviation-heading": 2.0236625632269, "agent_compute-ego0": 0.04911403790997228, "complete-iteration": 0.19741857220588577, "set_robot_commands": 0.0022930822999749356, "deviation-center-line": 0.5330742225800983, "driven_lanedir_consec": 2.037875454984482, "sim_compute_sim_state": 0.009187115122138411, "sim_compute_performance-ego0": 0.002023327062767213}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.613259814541894, "get_ui_image": 0.035582093573132724, "step_physics": 0.11672391859716816, "survival_time": 59.99999999999873, "driven_lanedir": 3.3926359681626903, "get_state_dump": 0.004631523089444607, "get_robot_state": 0.003626127028643936, "sim_render-ego0": 0.003932989209418888, "get_duckie_state": 1.2365606404859556e-06, "in-drivable-lane": 30.549999999998445, "deviation-heading": 5.443894859314437, "agent_compute-ego0": 0.04175185045532938, "complete-iteration": 0.2222801662305313, "set_robot_commands": 0.002258530465093481, "deviation-center-line": 1.3408542086195088, "driven_lanedir_consec": 3.3926359681626903, "sim_compute_sim_state": 0.011685994542905632, "sim_compute_performance-ego0": 0.0019981535547083365}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.761508136366983, "get_ui_image": 0.032544193418694974, "step_physics": 0.11128257037598724, "survival_time": 59.99999999999873, "driven_lanedir": 6.625920265404432, "get_state_dump": 0.00451651024480942, "get_robot_state": 0.003691949614080958, "sim_render-ego0": 0.0039034112109232705, "get_duckie_state": 1.1799833756700146e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.239805832874453, "agent_compute-ego0": 0.021549030505648065, "complete-iteration": 0.19457876414283923, "set_robot_commands": 0.0022651748196667776, "deviation-center-line": 2.703965477861179, "driven_lanedir_consec": 6.625920265404432, "sim_compute_sim_state": 0.012709704366552143, "sim_compute_performance-ego0": 0.0020296059480614708}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.399483118775881, "get_ui_image": 0.027268970340217386, "step_physics": 0.0960880953703792, "survival_time": 59.99999999999873, "driven_lanedir": 7.348429343220604, "get_state_dump": 0.004859600734154847, "get_robot_state": 0.003751207450148863, "sim_render-ego0": 0.003994097419821352, "get_duckie_state": 1.270308482656868e-06, "in-drivable-lane": 0.0, "deviation-heading": 6.063861771421233, "agent_compute-ego0": 0.039651163015437065, "complete-iteration": 0.18653445339123473, "set_robot_commands": 0.0023535070173150794, "deviation-center-line": 1.2124362420109789, "driven_lanedir_consec": 7.348429343220604, "sim_compute_sim_state": 0.006441130229178118, "sim_compute_performance-ego0": 0.002034975031234144}}
set_robot_commands_max0.0023535070173150794
set_robot_commands_mean0.0022925736505125683
set_robot_commands_median0.0022791285598208564
set_robot_commands_min0.002258530465093481
sim_compute_performance-ego0_max0.002034975031234144
sim_compute_performance-ego0_mean0.002021515399192791
sim_compute_performance-ego0_median0.0020264665054143417
sim_compute_performance-ego0_min0.0019981535547083365
sim_compute_sim_state_max0.012709704366552143
sim_compute_sim_state_mean0.010005986065193576
sim_compute_sim_state_median0.01043655483252202
sim_compute_sim_state_min0.006441130229178118
sim_render-ego0_max0.003994097419821352
sim_render-ego0_mean0.003956115414558302
sim_render-ego0_median0.003963476513744294
sim_render-ego0_min0.0039034112109232705
simulation-passed1
step_physics_max0.11672391859716816
step_physics_mean0.10442492646241962
step_physics_median0.10368533287318324
step_physics_min0.09360512150614386
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
51201LFv-simerrorno0:10:37
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140553551781024
- M:video_aido:cmdline(in:/;out:/) 140553551329456
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51198LFv-simerrorno0:09:07
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139863038235792
- M:video_aido:cmdline(in:/;out:/) 139863038236800
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51184LFv-simerrorno0:08:53
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139901223319344
- M:video_aido:cmdline(in:/;out:/) 139901223304832
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51179LFv-simerrorno0:09:00
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139798065705072
- M:video_aido:cmdline(in:/;out:/) 139798065706224
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40592LFv-simsuccessno0:09:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40591LFv-simsuccessno0:11:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40589LFv-simsuccessno0:09:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39875LFv-simsuccessno0:09:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38493LFv-simsuccessno0:08:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38492LFv-simsuccessno0:08:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible