Duckietown Challenges Home Challenges Submissions

Submission 10053

Submission10053
Competingyes
Challengeaido5-LF-sim-validation
UserJean-Sébastien Grondin 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57996
Next
User labelsim-exercise-1
Admin priority50
Blessingn/a
User priority50

57996

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57996LFv-simsuccessyes0:20:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.949768238756933
survival_time_median25.92500000000023
deviation-center-line_median0.8738968686281618
in-drivable-lane_median14.975000000000197


other stats
agent_compute-ego0_max0.04177256433804143
agent_compute-ego0_mean0.0270793002853447
agent_compute-ego0_median0.026687260973718686
agent_compute-ego0_min0.01317011485589999
complete-iteration_max0.22527808948520023
complete-iteration_mean0.1993203992663084
complete-iteration_median0.19544526303758464
complete-iteration_min0.18111298150486416
deviation-center-line_max2.7841273050942763
deviation-center-line_mean1.2574497368701691
deviation-center-line_min0.4978779051300761
deviation-heading_max11.48878380426352
deviation-heading_mean5.67234779317644
deviation-heading_median4.138340599025164
deviation-heading_min2.9239261703919124
driven_any_max9.51372574947282
driven_any_mean5.211000214966301
driven_any_median4.119175131188098
driven_any_min3.0919248480161925
driven_lanedir_consec_max5.447814540184594
driven_lanedir_consec_mean2.563346657171923
driven_lanedir_consec_min0.90603561098923
driven_lanedir_max5.447814540184594
driven_lanedir_mean2.563346657171923
driven_lanedir_median1.949768238756933
driven_lanedir_min0.90603561098923
get_duckie_state_max1.4665574800890928e-06
get_duckie_state_mean1.355589460187993e-06
get_duckie_state_median1.3522747414983217e-06
get_duckie_state_min1.2512508776662352e-06
get_robot_state_max0.003934252596828386
get_robot_state_mean0.0037951080263124498
get_robot_state_median0.003782389040093025
get_robot_state_min0.003681401428235361
get_state_dump_max0.004884500147026276
get_state_dump_mean0.004761517435941536
get_state_dump_median0.00476597514131357
get_state_dump_min0.00462961931411273
get_ui_image_max0.0345997069896409
get_ui_image_mean0.030260698563565665
get_ui_image_median0.030018047337856728
get_ui_image_min0.02640699258890832
in-drivable-lane_max22.74999999999896
in-drivable-lane_mean16.312499999999883
in-drivable-lane_min12.550000000000178
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.240093018196147, "get_ui_image": 0.02756636483328683, "step_physics": 0.10949657313407414, "survival_time": 25.150000000000222, "driven_lanedir": 1.9831063085667349, "get_state_dump": 0.004724459515677558, "get_robot_state": 0.003725680566969372, "sim_render-ego0": 0.003897873655198112, "get_duckie_state": 1.3680685134161086e-06, "in-drivable-lane": 13.600000000000191, "deviation-heading": 2.9239261703919124, "agent_compute-ego0": 0.01718222669192723, "complete-iteration": 0.18111298150486416, "set_robot_commands": 0.002198567939183069, "deviation-center-line": 0.7652224432773199, "driven_lanedir_consec": 1.9831063085667349, "sim_compute_sim_state": 0.010217195938503933, "sim_compute_performance-ego0": 0.0020202535485464427}, "LF-norm-zigzag-000-ego0": {"driven_any": 9.51372574947282, "get_ui_image": 0.0345997069896409, "step_physics": 0.1255049620143182, "survival_time": 59.99999999999873, "driven_lanedir": 5.447814540184594, "get_state_dump": 0.00462961931411273, "get_robot_state": 0.003681401428235361, "sim_render-ego0": 0.003823518951568477, "get_duckie_state": 1.2512508776662352e-06, "in-drivable-lane": 22.74999999999896, "deviation-heading": 11.48878380426352, "agent_compute-ego0": 0.03619229525551014, "complete-iteration": 0.22527808948520023, "set_robot_commands": 0.002177703986060709, "deviation-center-line": 2.7841273050942763, "driven_lanedir_consec": 5.447814540184594, "sim_compute_sim_state": 0.012646249092985054, "sim_compute_performance-ego0": 0.0019377339988028775}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.9982572441800497, "get_ui_image": 0.032469729842426624, "step_physics": 0.12462233204707922, "survival_time": 26.700000000000244, "driven_lanedir": 1.916430168947131, "get_state_dump": 0.004884500147026276, "get_robot_state": 0.0038390975132166784, "sim_render-ego0": 0.004043258239175672, "get_duckie_state": 1.3364809695805345e-06, "in-drivable-lane": 12.550000000000178, "deviation-heading": 5.3476592240488285, "agent_compute-ego0": 0.01317011485589999, "complete-iteration": 0.1984011837255175, "set_robot_commands": 0.002341100211455443, "deviation-center-line": 0.9825712939790036, "driven_lanedir_consec": 1.916430168947131, "sim_compute_sim_state": 0.01083829068691931, "sim_compute_performance-ego0": 0.0020988624786662164}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.0919248480161925, "get_ui_image": 0.02640699258890832, "step_physics": 0.1007957376105224, "survival_time": 23.100000000000193, "driven_lanedir": 0.90603561098923, "get_state_dump": 0.004807490766949581, "get_robot_state": 0.003934252596828386, "sim_render-ego0": 0.0039701168274519, "get_duckie_state": 1.4665574800890928e-06, "in-drivable-lane": 16.3500000000002, "deviation-heading": 2.929021974001499, "agent_compute-ego0": 0.04177256433804143, "complete-iteration": 0.19248934234965184, "set_robot_commands": 0.0023093192675201217, "deviation-center-line": 0.4978779051300761, "driven_lanedir_consec": 0.90603561098923, "sim_compute_sim_state": 0.006405545104914556, "sim_compute_performance-ego0": 0.0019862811457261155}}
set_robot_commands_max0.002341100211455443
set_robot_commands_mean0.0022566728510548355
set_robot_commands_median0.0022539436033515953
set_robot_commands_min0.002177703986060709
sim_compute_performance-ego0_max0.0020988624786662164
sim_compute_performance-ego0_mean0.002010782792935413
sim_compute_performance-ego0_median0.0020032673471362793
sim_compute_performance-ego0_min0.0019377339988028775
sim_compute_sim_state_max0.012646249092985054
sim_compute_sim_state_mean0.010026820205830712
sim_compute_sim_state_median0.01052774331271162
sim_compute_sim_state_min0.006405545104914556
sim_render-ego0_max0.004043258239175672
sim_render-ego0_mean0.00393369191834854
sim_render-ego0_median0.003933995241325006
sim_render-ego0_min0.003823518951568477
simulation-passed1
step_physics_max0.1255049620143182
step_physics_mean0.1151049012014985
step_physics_median0.11705945259057668
step_physics_min0.1007957376105224
survival_time_max59.99999999999873
survival_time_mean33.73749999999985
survival_time_min23.100000000000193
No reset possible
51996LFv-simerrorno0:04:42
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140142258110816
- M:video_aido:cmdline(in:/;out:/) 140142258124880
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51994LFv-simerrorno0:05:28
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140120683842240
- M:video_aido:cmdline(in:/;out:/) 140116725889296
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51945LFv-simhost-errorno0:03:48
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51937LFv-simerrorno0:03:33
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140187908817248
- M:video_aido:cmdline(in:/;out:/) 140187873484272
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51927LFv-simerrorno0:04:49
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140460509624736
- M:video_aido:cmdline(in:/;out:/) 140460509626320
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41524LFv-simsuccessno0:07:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37183LFv-simsuccessno0:08:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36579LFv-simsuccessno0:09:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible