Duckietown Challenges Home Challenges Submissions

Submission 10996

Submission10996
Competingyes
Challengeaido5-LF-sim-validation
UserFernanda Custodio Pereira do Carmo 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57180
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57180

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57180LFv-simsuccessyes0:24:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.8485165924856366
survival_time_median41.0499999999997
deviation-center-line_median0.9059883620947816
in-drivable-lane_median10.674999999999692


other stats
agent_compute-ego0_max0.013479751870949014
agent_compute-ego0_mean0.01298622475271005
agent_compute-ego0_median0.012921987728434631
agent_compute-ego0_min0.012621171683021924
complete-iteration_max0.2017460049323316
complete-iteration_mean0.17480898961624575
complete-iteration_median0.1698418744232065
complete-iteration_min0.15780620468623846
deviation-center-line_max2.93372342163456
deviation-center-line_mean1.3530409983773033
deviation-center-line_min0.6664638476850905
deviation-heading_max7.1996933852617815
deviation-heading_mean3.5634264165499485
deviation-heading_median2.7134701704277457
deviation-heading_min1.6270719400825209
driven_any_max10.672664580804806
driven_any_mean6.621417786986354
driven_any_median6.418366324183636
driven_any_min2.9762739187733382
driven_lanedir_consec_max8.208488096617563
driven_lanedir_consec_mean3.879740001537144
driven_lanedir_consec_min1.6134387245597384
driven_lanedir_max8.208488096617563
driven_lanedir_mean3.879740001537144
driven_lanedir_median2.8485165924856366
driven_lanedir_min1.6134387245597384
get_duckie_state_max1.3821985674398785e-06
get_duckie_state_mean1.3445331130242475e-06
get_duckie_state_median1.3432588042128583e-06
get_duckie_state_min1.3094162762313955e-06
get_robot_state_max0.0039034599857887607
get_robot_state_mean0.0038056106745098864
get_robot_state_median0.003791136429533516
get_robot_state_min0.0037367098531837528
get_state_dump_max0.004962672836706393
get_state_dump_mean0.004696569972400877
get_state_dump_median0.004653951160244844
get_state_dump_min0.004515704732407428
get_ui_image_max0.036024687425145564
get_ui_image_mean0.03036874500140936
get_ui_image_median0.029472589618836552
get_ui_image_min0.02650511334281877
in-drivable-lane_max42.39999999999911
in-drivable-lane_mean17.487499999999645
in-drivable-lane_min6.200000000000088
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.771744758870993, "get_ui_image": 0.02674467649508913, "step_physics": 0.09165932342895718, "survival_time": 29.10000000000028, "driven_lanedir": 3.9877001245328376, "get_state_dump": 0.004515704732407428, "get_robot_state": 0.0037367098531837528, "sim_render-ego0": 0.003841964090258957, "get_duckie_state": 1.320912539447765e-06, "in-drivable-lane": 6.200000000000088, "deviation-heading": 2.468367098502538, "agent_compute-ego0": 0.012621171683021924, "complete-iteration": 0.15780620468623846, "set_robot_commands": 0.0022190005660670496, "deviation-center-line": 0.9908902376690276, "driven_lanedir_consec": 3.9877001245328376, "sim_compute_sim_state": 0.01035312845612881, "sim_compute_performance-ego0": 0.002028855001537918}, "LF-norm-zigzag-000-ego0": {"driven_any": 2.9762739187733382, "get_ui_image": 0.036024687425145564, "step_physics": 0.12387194080815482, "survival_time": 18.500000000000128, "driven_lanedir": 1.7093330604384358, "get_state_dump": 0.0045912143676428785, "get_robot_state": 0.0037666890177765304, "sim_render-ego0": 0.004015256773750737, "get_duckie_state": 1.3656050689779523e-06, "in-drivable-lane": 8.40000000000012, "deviation-heading": 1.6270719400825209, "agent_compute-ego0": 0.013144728951055729, "complete-iteration": 0.2017460049323316, "set_robot_commands": 0.002300091509549123, "deviation-center-line": 0.8210864865205353, "driven_lanedir_consec": 1.7093330604384358, "sim_compute_sim_state": 0.01185368591884397, "sim_compute_performance-ego0": 0.0020901552750415236}, "LF-norm-techtrack-000-ego0": {"driven_any": 8.06498788949628, "get_ui_image": 0.03220050274258397, "step_physics": 0.10426600287254974, "survival_time": 52.999999999999126, "driven_lanedir": 1.6134387245597384, "get_state_dump": 0.004962672836706393, "get_robot_state": 0.0039034599857887607, "sim_render-ego0": 0.004192932049807909, "get_duckie_state": 1.3821985674398785e-06, "in-drivable-lane": 42.39999999999911, "deviation-heading": 2.958573242352954, "agent_compute-ego0": 0.013479751870949014, "complete-iteration": 0.17892562275669915, "set_robot_commands": 0.0023207765609784357, "deviation-center-line": 0.6664638476850905, "driven_lanedir_consec": 1.6134387245597384, "sim_compute_sim_state": 0.011349471979383925, "sim_compute_performance-ego0": 0.0021508137759569117}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.672664580804806, "get_ui_image": 0.02650511334281877, "step_physics": 0.09814525027755495, "survival_time": 59.99999999999873, "driven_lanedir": 8.208488096617563, "get_state_dump": 0.00471668795284681, "get_robot_state": 0.0038155838412905017, "sim_render-ego0": 0.003943339077062551, "get_duckie_state": 1.3094162762313955e-06, "in-drivable-lane": 12.949999999999264, "deviation-heading": 7.1996933852617815, "agent_compute-ego0": 0.012699246505813534, "complete-iteration": 0.16075812608971385, "set_robot_commands": 0.002329474781871735, "deviation-center-line": 2.93372342163456, "driven_lanedir_consec": 8.208488096617563, "sim_compute_sim_state": 0.006456089853545609, "sim_compute_performance-ego0": 0.002055690052308806}}
set_robot_commands_max0.002329474781871735
set_robot_commands_mean0.0022923358546165858
set_robot_commands_median0.0023104340352637794
set_robot_commands_min0.0022190005660670496
sim_compute_performance-ego0_max0.0021508137759569117
sim_compute_performance-ego0_mean0.00208137852621129
sim_compute_performance-ego0_median0.002072922663675165
sim_compute_performance-ego0_min0.002028855001537918
sim_compute_sim_state_max0.01185368591884397
sim_compute_sim_state_mean0.010003094051975575
sim_compute_sim_state_median0.010851300217756366
sim_compute_sim_state_min0.006456089853545609
sim_render-ego0_max0.004192932049807909
sim_render-ego0_mean0.003998372997720039
sim_render-ego0_median0.003979297925406644
sim_render-ego0_min0.003841964090258957
simulation-passed1
step_physics_max0.12387194080815482
step_physics_mean0.10448562934680417
step_physics_median0.10120562657505236
step_physics_min0.09165932342895718
survival_time_max59.99999999999873
survival_time_mean40.149999999999565
survival_time_min18.500000000000128
No reset possible
50952LFv-simerrorno0:05:28
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139623822707728
- M:video_aido:cmdline(in:/;out:/) 139623832985360
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50947LFv-simerrorno0:03:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139670008479856
- M:video_aido:cmdline(in:/;out:/) 139670008487312
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50946LFv-simerrorno0:03:33
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139822006103536
- M:video_aido:cmdline(in:/;out:/) 139822005520896
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40483LFv-simsuccessno0:11:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39790LFv-simsuccessno0:13:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39780LFv-simtimeoutno0:02:07
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39768LFv-simsuccessno0:14:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38624LFv-simsuccessno0:08:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38623LFv-simsuccessno0:10:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible