Duckietown Challenges Home Challenges Submissions

Submission 10889

Submission10889
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57651
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57651

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57651LFv-simsuccessyes0:25:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.90837115192834
survival_time_median53.64999999999909
deviation-center-line_median0.8906545855714834
in-drivable-lane_median14.19999999999981


other stats
agent_compute-ego0_max0.014278497464301845
agent_compute-ego0_mean0.013570294442560513
agent_compute-ego0_median0.013425283172300255
agent_compute-ego0_min0.013152113961339692
complete-iteration_max0.240299287964316
complete-iteration_mean0.19622814480189155
complete-iteration_median0.19019858156250977
complete-iteration_min0.16421612811823075
deviation-center-line_max1.309013801529915
deviation-center-line_mean0.8188836946178241
deviation-center-line_min0.18521180579841437
deviation-heading_max5.418176445166056
deviation-heading_mean3.5855123318800866
deviation-heading_median3.4097049908079486
deviation-heading_min2.1044629007383926
driven_any_max6.804885165214906
driven_any_mean4.435136651302222
driven_any_median5.324575231852295
driven_any_min0.2865109762893948
driven_lanedir_consec_max6.248487178119438
driven_lanedir_consec_mean2.54468640357284
driven_lanedir_consec_min0.11351613231524248
driven_lanedir_max6.248487178119438
driven_lanedir_mean2.6390958498202535
driven_lanedir_median2.097190044423167
driven_lanedir_min0.11351613231524248
get_duckie_state_max1.5123003626941742e-06
get_duckie_state_mean1.477221788113094e-06
get_duckie_state_median1.4780747396081026e-06
get_duckie_state_min1.4404373105419963e-06
get_robot_state_max0.004162547958438474
get_robot_state_mean0.003958971351069445
get_robot_state_median0.003911276244799561
get_robot_state_min0.0038507849562401863
get_state_dump_max0.005305963181895464
get_state_dump_mean0.005070575199122279
get_state_dump_median0.005040260539210804
get_state_dump_min0.004895816536172046
get_ui_image_max0.036798834800720215
get_ui_image_mean0.03237968911005096
get_ui_image_median0.03262904168692153
get_ui_image_min0.02746183826564055
in-drivable-lane_max41.6999999999986
in-drivable-lane_mean17.524999999999558
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 6.804885165214906, "get_ui_image": 0.03000583855139028, "step_physics": 0.10709751376899256, "survival_time": 59.99999999999873, "driven_lanedir": 2.071330445547987, "get_state_dump": 0.005121068692425705, "get_robot_state": 0.003905575837223457, "sim_render-ego0": 0.004165688124822637, "get_duckie_state": 1.4404373105419963e-06, "in-drivable-lane": 41.6999999999986, "deviation-heading": 2.2168133299357695, "agent_compute-ego0": 0.013337590315260555, "complete-iteration": 0.177925982145743, "set_robot_commands": 0.002315210958603121, "deviation-center-line": 0.8093630939894975, "driven_lanedir_consec": 2.071330445547987, "sim_compute_sim_state": 0.009720197029653734, "sim_compute_performance-ego0": 0.0021568267768269076}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.2865109762893948, "get_ui_image": 0.036798834800720215, "step_physics": 0.1615204834470562, "survival_time": 5.04999999999999, "driven_lanedir": 0.11351613231524248, "get_state_dump": 0.004959452385995902, "get_robot_state": 0.0038507849562401863, "sim_render-ego0": 0.004188369302188649, "get_duckie_state": 1.453885845109528e-06, "in-drivable-lane": 2.3499999999999917, "deviation-heading": 2.1044629007383926, "agent_compute-ego0": 0.01351297602933996, "complete-iteration": 0.240299287964316, "set_robot_commands": 0.002331352701374129, "deviation-center-line": 0.18521180579841437, "driven_lanedir_consec": 0.11351613231524248, "sim_compute_sim_state": 0.010979287764605354, "sim_compute_performance-ego0": 0.0020630850511438705}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.364046553364432, "get_ui_image": 0.03525224482245279, "step_physics": 0.1227566035269684, "survival_time": 47.29999999999945, "driven_lanedir": 2.123049643298347, "get_state_dump": 0.005305963181895464, "get_robot_state": 0.004162547958438474, "sim_render-ego0": 0.004480108414179672, "get_duckie_state": 1.5022636341066772e-06, "in-drivable-lane": 26.049999999999628, "deviation-heading": 4.602596651680128, "agent_compute-ego0": 0.014278497464301845, "complete-iteration": 0.20247118097927652, "set_robot_commands": 0.00255605185044479, "deviation-center-line": 0.9719460771534694, "driven_lanedir_consec": 1.7454118583086928, "sim_compute_sim_state": 0.011216198126642858, "sim_compute_performance-ego0": 0.002355706981268201}, "LF-norm-small_loop-000-ego0": {"driven_any": 6.285103910340157, "get_ui_image": 0.02746183826564055, "step_physics": 0.09957843795604848, "survival_time": 59.99999999999873, "driven_lanedir": 6.248487178119438, "get_state_dump": 0.004895816536172046, "get_robot_state": 0.003916976652375665, "sim_render-ego0": 0.0040719401925727785, "get_duckie_state": 1.5123003626941742e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.418176445166056, "agent_compute-ego0": 0.013152113961339692, "complete-iteration": 0.16421612811823075, "set_robot_commands": 0.002366532096259302, "deviation-center-line": 1.309013801529915, "driven_lanedir_consec": 6.248487178119438, "sim_compute_sim_state": 0.006579517821884473, "sim_compute_performance-ego0": 0.002099589642438166}}
set_robot_commands_max0.00255605185044479
set_robot_commands_mean0.0023922869016703356
set_robot_commands_median0.0023489423988167155
set_robot_commands_min0.002315210958603121
sim_compute_performance-ego0_max0.002355706981268201
sim_compute_performance-ego0_mean0.002168802112919286
sim_compute_performance-ego0_median0.0021282082096325365
sim_compute_performance-ego0_min0.0020630850511438705
sim_compute_sim_state_max0.011216198126642858
sim_compute_sim_state_mean0.009623800185696604
sim_compute_sim_state_median0.010349742397129545
sim_compute_sim_state_min0.006579517821884473
sim_render-ego0_max0.004480108414179672
sim_render-ego0_mean0.004226526508440935
sim_render-ego0_median0.004177028713505643
sim_render-ego0_min0.0040719401925727785
simulation-passed1
step_physics_max0.1615204834470562
step_physics_mean0.1227382596747664
step_physics_median0.11492705864798047
step_physics_min0.09957843795604848
survival_time_max59.99999999999873
survival_time_mean43.087499999999224
survival_time_min5.04999999999999
No reset possible
57581LFv-simtimeoutyes----No reset possible
57573LFv-simtimeoutyes----No reset possible
51387LFv-simerrorno0:08:11
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139641497118752
- M:video_aido:cmdline(in:/;out:/) 139641497117696
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51386LFv-simerrorno0:07:28
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140132820125824
- M:video_aido:cmdline(in:/;out:/) 140132820212272
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51370LFv-simerrorno0:08:16
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140066576487136
- M:video_aido:cmdline(in:/;out:/) 140066575104560
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51366LFv-simhost-errorno0:10:33
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51361LFv-simerrorno0:10:40
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140331524080688
- M:video_aido:cmdline(in:/;out:/) 140331524079872
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40649LFv-simsuccessno0:12:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38349LFv-simsuccessno0:09:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible