Duckietown Challenges Home Challenges Submissions

Submission 11074

Submission11074
Competingyes
Challengeaido5-LF-sim-validation
UserAyman Shams 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 56859
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

56859

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
56859LFv-simsuccessyes0:06:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4683189245930201
survival_time_median7.324999999999981
deviation-center-line_median0.10780536854030706
in-drivable-lane_median4.7999999999999865


other stats
agent_compute-ego0_max0.012886566816320742
agent_compute-ego0_mean0.012190758033142547
agent_compute-ego0_median0.012063140854423428
agent_compute-ego0_min0.0117501836074026
complete-iteration_max0.19829785375666795
complete-iteration_mean0.17758454643400626
complete-iteration_median0.1734799094822096
complete-iteration_min0.1650805130149379
deviation-center-line_max0.43283025350249094
deviation-center-line_mean0.1831047623386008
deviation-center-line_min0.0839780587712981
deviation-heading_max1.2092541162534414
deviation-heading_mean0.7583560781324769
deviation-heading_median0.7318627381073197
deviation-heading_min0.360444720061827
driven_any_max3.2180208704397777
driven_any_mean2.1842287143276096
driven_any_median2.1025517519198216
driven_any_min1.3137904830310163
driven_lanedir_consec_max0.9633582081973676
driven_lanedir_consec_mean0.568105994728193
driven_lanedir_consec_min0.3724279215293642
driven_lanedir_max0.9633582081973676
driven_lanedir_mean0.568105994728193
driven_lanedir_median0.4683189245930201
driven_lanedir_min0.3724279215293642
get_duckie_state_max1.5173429324303145e-06
get_duckie_state_mean1.4247705696708094e-06
get_duckie_state_median1.4060884910452822e-06
get_duckie_state_min1.369562364162359e-06
get_robot_state_max0.003776882005774457
get_robot_state_mean0.003700286894383643
get_robot_state_median0.003744527914976298
get_robot_state_min0.0035352097418075218
get_state_dump_max0.004762315388881799
get_state_dump_mean0.0045762133669285
get_state_dump_median0.004565801277508738
get_state_dump_min0.004410935523814725
get_ui_image_max0.035018124974759895
get_ui_image_mean0.03072025834757324
get_ui_image_median0.030553057125929026
get_ui_image_min0.02675679416367502
in-drivable-lane_max8.70000000000001
in-drivable-lane_mean5.387499999999992
in-drivable-lane_min3.2499999999999885
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.393479745505811, "get_ui_image": 0.028171286170865284, "step_physics": 0.10423859107641528, "survival_time": 8.04999999999998, "driven_lanedir": 0.9633582081973676, "get_state_dump": 0.004543116063247492, "get_robot_state": 0.003753417803917402, "sim_render-ego0": 0.003825034624264564, "get_duckie_state": 1.5173429324303145e-06, "in-drivable-lane": 4.549999999999988, "deviation-heading": 1.2092541162534414, "agent_compute-ego0": 0.011997113993138443, "complete-iteration": 0.17010005315144858, "set_robot_commands": 0.002133999341799889, "deviation-center-line": 0.43283025350249094, "driven_lanedir_consec": 0.9633582081973676, "sim_compute_sim_state": 0.009322825773262685, "sim_compute_performance-ego0": 0.00202566017339259}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3137904830310163, "get_ui_image": 0.035018124974759895, "step_physics": 0.12558191700985558, "survival_time": 6.5999999999999845, "driven_lanedir": 0.5357785609306029, "get_state_dump": 0.004410935523814725, "get_robot_state": 0.0035352097418075218, "sim_render-ego0": 0.0038019033302938137, "get_duckie_state": 1.369562364162359e-06, "in-drivable-lane": 3.2499999999999885, "deviation-heading": 1.0801222005967757, "agent_compute-ego0": 0.0117501836074026, "complete-iteration": 0.19829785375666795, "set_robot_commands": 0.00209070506848787, "deviation-center-line": 0.12567976998604052, "driven_lanedir_consec": 0.5357785609306029, "sim_compute_sim_state": 0.010096731042503417, "sim_compute_performance-ego0": 0.001927517410507776}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.2180208704397777, "get_ui_image": 0.03293482808099277, "step_physics": 0.10610875530519348, "survival_time": 10.300000000000011, "driven_lanedir": 0.4008592882554371, "get_state_dump": 0.004588486491769984, "get_robot_state": 0.003776882005774457, "sim_render-ego0": 0.003965382414739489, "get_duckie_state": 1.3925027156221694e-06, "in-drivable-lane": 8.70000000000001, "deviation-heading": 0.3836032756178637, "agent_compute-ego0": 0.012886566816320742, "complete-iteration": 0.17685976581297058, "set_robot_commands": 0.002236958287188396, "deviation-center-line": 0.0839780587712981, "driven_lanedir_consec": 0.4008592882554371, "sim_compute_sim_state": 0.00826791288772067, "sim_compute_performance-ego0": 0.002007934782240126}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.8116237583338315, "get_ui_image": 0.02675679416367502, "step_physics": 0.10398602124416469, "survival_time": 6.549999999999985, "driven_lanedir": 0.3724279215293642, "get_state_dump": 0.004762315388881799, "get_robot_state": 0.0037356380260351934, "sim_render-ego0": 0.003949210499272202, "get_duckie_state": 1.4196742664683948e-06, "in-drivable-lane": 5.049999999999984, "deviation-heading": 0.360444720061827, "agent_compute-ego0": 0.012129167715708414, "complete-iteration": 0.1650805130149379, "set_robot_commands": 0.0022803888176426744, "deviation-center-line": 0.0899309670945736, "driven_lanedir_consec": 0.3724279215293642, "sim_compute_sim_state": 0.005363887006586248, "sim_compute_performance-ego0": 0.002025577155026522}}
set_robot_commands_max0.0022803888176426744
set_robot_commands_mean0.0021855128787797076
set_robot_commands_median0.0021854788144941425
set_robot_commands_min0.00209070506848787
sim_compute_performance-ego0_max0.00202566017339259
sim_compute_performance-ego0_mean0.0019966723802917535
sim_compute_performance-ego0_median0.002016755968633324
sim_compute_performance-ego0_min0.001927517410507776
sim_compute_sim_state_max0.010096731042503417
sim_compute_sim_state_mean0.008262839177518255
sim_compute_sim_state_median0.008795369330491676
sim_compute_sim_state_min0.005363887006586248
sim_render-ego0_max0.003965382414739489
sim_render-ego0_mean0.003885382717142517
sim_render-ego0_median0.003887122561768383
sim_render-ego0_min0.0038019033302938137
simulation-passed1
step_physics_max0.12558191700985558
step_physics_mean0.10997882115890724
step_physics_median0.10517367319080435
step_physics_min0.10398602124416469
survival_time_max10.300000000000011
survival_time_mean7.874999999999989
survival_time_min6.549999999999985
No reset possible
56856LFv-simsuccessyes0:05:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
56835LFv-simsuccessyes0:05:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50762LFv-simerrorno0:02:29
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140396820032336
- M:video_aido:cmdline(in:/;out:/) 140396820030512
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50758LFv-simerrorno0:03:49
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140624423303248
- M:video_aido:cmdline(in:/;out:/) 140624459553856
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40308LFv-simsuccessno0:09:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40305LFv-simsuccessno0:08:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40302LFv-simsuccessno0:12:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39679LFv-simsuccessno0:09:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38854LFv-simsuccessno0:09:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38853LFv-simsuccessno0:07:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible