Duckietown Challenges Home Challenges Submissions

Submission 11352

Submission11352
Competingyes
Challengeaido5-LF-sim-validation
UserCharlie Gauthier 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54842
Next
User labelBaseline solution using reinforcement learning
Admin priority50
Blessingn/a
User priority50

54842

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54842LFv-simsuccessyes0:11:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4953415167970959
survival_time_median3.8749999999999942
deviation-center-line_median0.10019431734994318
in-drivable-lane_median2.5999999999999948


other stats
agent_compute-ego0_max0.012525453800108374
agent_compute-ego0_mean0.012143681882755438
agent_compute-ego0_median0.012395107173734396
agent_compute-ego0_min0.01125905938344459
complete-iteration_max0.23632793310211925
complete-iteration_mean0.2021110316950288
complete-iteration_median0.20139139725968072
complete-iteration_min0.16933339915863457
deviation-center-line_max1.3347053459531408
deviation-center-line_mean0.3964925199382131
deviation-center-line_min0.050876099099825205
deviation-heading_max7.092505358170257
deviation-heading_mean2.1832773338209277
deviation-heading_median0.6958540662065884
deviation-heading_min0.24889584470027704
driven_any_max32.21451956876099
driven_any_mean9.164620243328644
driven_any_median1.53081754802698
driven_any_min1.3823263084996291
driven_lanedir_consec_max9.151844330275928
driven_lanedir_consec_mean2.6196278682405016
driven_lanedir_consec_min0.3359841090918858
driven_lanedir_max10.09337842757754
driven_lanedir_mean2.85860808999487
driven_lanedir_median0.5025349116550267
driven_lanedir_min0.3359841090918858
get_duckie_state_max1.2429555257161455e-06
get_duckie_state_mean1.166695747432473e-06
get_duckie_state_median1.1559676614207563e-06
get_duckie_state_min1.1118921411722328e-06
get_robot_state_max0.0035006301190632764
get_robot_state_mean0.003482871488900311
get_robot_state_median0.0034834329714712947
get_robot_state_min0.0034639898935953776
get_state_dump_max0.004396213877707298
get_state_dump_mean0.0043807603151646
get_state_dump_median0.004390832336698533
get_state_dump_min0.004345162709554037
get_ui_image_max0.03620511148034072
get_ui_image_mean0.031075520405681095
get_ui_image_median0.03070216410893915
get_ui_image_min0.026692641924505364
in-drivable-lane_max39.1499999999992
in-drivable-lane_mean11.612499999999796
in-drivable-lane_min2.0999999999999934
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 32.21451956876099, "get_ui_image": 0.028109942943626995, "step_physics": 0.11880401786816903, "survival_time": 59.99999999999873, "driven_lanedir": 10.09337842757754, "get_state_dump": 0.004396213877707298, "get_robot_state": 0.0035006301190632764, "sim_render-ego0": 0.0036171900044869223, "get_duckie_state": 1.1118921411722328e-06, "in-drivable-lane": 39.1499999999992, "deviation-heading": 7.092505358170257, "agent_compute-ego0": 0.012287798372533895, "complete-iteration": 0.18289603698660592, "set_robot_commands": 0.0023690077982576165, "deviation-center-line": 1.3347053459531408, "driven_lanedir_consec": 9.151844330275928, "sim_compute_sim_state": 0.007835592854330681, "sim_compute_performance-ego0": 0.001898562282844944}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.6253119880518745, "get_ui_image": 0.03620511148034072, "step_physics": 0.16277252174005274, "survival_time": 4.049999999999994, "driven_lanedir": 0.6188196930254084, "get_state_dump": 0.004385500419430617, "get_robot_state": 0.0034823359512701265, "sim_render-ego0": 0.003569748343490973, "get_duckie_state": 1.1688325463271722e-06, "in-drivable-lane": 2.0999999999999934, "deviation-heading": 1.1348402688492234, "agent_compute-ego0": 0.012525453800108374, "complete-iteration": 0.23632793310211925, "set_robot_commands": 0.0025493517154600563, "deviation-center-line": 0.14647640362993186, "driven_lanedir_consec": 0.6044329033095468, "sim_compute_sim_state": 0.008931284997521377, "sim_compute_performance-ego0": 0.0018317583130627144}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.436323108002086, "get_ui_image": 0.033294385274251305, "step_physics": 0.1512198575337728, "survival_time": 3.699999999999995, "driven_lanedir": 0.38625013028464505, "get_state_dump": 0.004345162709554037, "get_robot_state": 0.0034639898935953776, "sim_render-ego0": 0.0035135650634765623, "get_duckie_state": 1.2429555257161455e-06, "in-drivable-lane": 2.5999999999999943, "deviation-heading": 0.25686786356395347, "agent_compute-ego0": 0.012502415974934896, "complete-iteration": 0.21988675753275552, "set_robot_commands": 0.0023477776845296223, "deviation-center-line": 0.050876099099825205, "driven_lanedir_consec": 0.38625013028464505, "sim_compute_sim_state": 0.007290611267089844, "sim_compute_performance-ego0": 0.0018352381388346355}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.3823263084996291, "get_ui_image": 0.026692641924505364, "step_physics": 0.11102690762036468, "survival_time": 3.599999999999995, "driven_lanedir": 0.3359841090918858, "get_state_dump": 0.004396164253966449, "get_robot_state": 0.003484529991672463, "sim_render-ego0": 0.00353583571029036, "get_duckie_state": 1.1431027765143409e-06, "in-drivable-lane": 2.599999999999995, "deviation-heading": 0.24889584470027704, "agent_compute-ego0": 0.01125905938344459, "complete-iteration": 0.16933339915863457, "set_robot_commands": 0.0021404076928961766, "deviation-center-line": 0.053912231069954485, "driven_lanedir_consec": 0.3359841090918858, "sim_compute_sim_state": 0.004887976058541912, "sim_compute_performance-ego0": 0.00183788391008769}}
set_robot_commands_max0.0025493517154600563
set_robot_commands_mean0.0023516362227858677
set_robot_commands_median0.002358392741393619
set_robot_commands_min0.0021404076928961766
sim_compute_performance-ego0_max0.001898562282844944
sim_compute_performance-ego0_mean0.0018508606612074956
sim_compute_performance-ego0_median0.0018365610244611627
sim_compute_performance-ego0_min0.0018317583130627144
sim_compute_sim_state_max0.008931284997521377
sim_compute_sim_state_mean0.007236366294370954
sim_compute_sim_state_median0.007563102060710263
sim_compute_sim_state_min0.004887976058541912
sim_render-ego0_max0.0036171900044869223
sim_render-ego0_mean0.0035590847804362045
sim_render-ego0_median0.0035527920268906667
sim_render-ego0_min0.0035135650634765623
simulation-passed1
step_physics_max0.16277252174005274
step_physics_mean0.13595582619058982
step_physics_median0.13501193770097092
step_physics_min0.11102690762036468
survival_time_max59.99999999999873
survival_time_mean17.83749999999968
survival_time_min3.599999999999995
No reset possible
49973LFv-simerrorno0:03:26
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140297393473856
- M:video_aido:cmdline(in:/;out:/) 140297393474672
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49963LFv-simerrorno0:01:31
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140154335943648
- M:video_aido:cmdline(in:/;out:/) 140154335954736
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42023LFv-simerrorno1:01:04
Waited 3602 seconds [...]
Waited 3602 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42018LFv-simsuccessno0:15:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42017LFv-simtimeoutno----No reset possible
42016LFv-simtimeoutno----No reset possible