Duckietown Challenges Home Challenges Submissions

Submission 11608

Submission11608
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54280
Next
User labelexercise_state_estimation
Admin priority50
Blessingn/a
User priority50

54280

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54280LFv-simsuccessyes0:33:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.704465335547944
survival_time_median59.99999999999873
deviation-center-line_median2.966269474125017
in-drivable-lane_median13.649999999999636


other stats
agent_compute-ego0_max0.012235636318057503
agent_compute-ego0_mean0.011561207606474009
agent_compute-ego0_median0.011457154196962327
agent_compute-ego0_min0.01109488571391386
complete-iteration_max0.2459871165892657
complete-iteration_mean0.21426341899019233
complete-iteration_median0.21320833810461648
complete-iteration_min0.18464988316227057
deviation-center-line_max4.593470679684905
deviation-center-line_mean2.7102726757143216
deviation-center-line_min0.3150810749223459
deviation-heading_max17.205885331872494
deviation-heading_mean11.487475130073864
deviation-heading_median13.019226558874305
deviation-heading_min2.70556207067436
driven_any_max7.920818301409121
driven_any_mean6.1846851252177215
driven_any_median7.917086253692983
driven_any_min0.9837496920757972
driven_lanedir_consec_max5.871997736717288
driven_lanedir_consec_mean3.4542187461919234
driven_lanedir_consec_min0.5359465769545191
driven_lanedir_max5.871997736717288
driven_lanedir_mean3.4542187461919234
driven_lanedir_median3.704465335547944
driven_lanedir_min0.5359465769545191
get_duckie_state_max1.352097370741667e-06
get_duckie_state_mean1.253261151378942e-06
get_duckie_state_median1.2722936498433922e-06
get_duckie_state_min1.1163599350873162e-06
get_robot_state_max0.0037719881802573984
get_robot_state_mean0.003606412101517101
get_robot_state_median0.0036069724482362415
get_robot_state_min0.0034397153293385225
get_state_dump_max0.004725623388870074
get_state_dump_mean0.004581076005719963
get_state_dump_median0.004576621404992452
get_state_dump_min0.0044454378240248735
get_ui_image_max0.033469134218552535
get_ui_image_mean0.029778088357963903
get_ui_image_median0.030379334059881233
get_ui_image_min0.02488455109354062
in-drivable-lane_max37.64999999999908
in-drivable-lane_mean17.012499999999587
in-drivable-lane_min3.099999999999996
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.920818301409121, "get_ui_image": 0.028091141822236865, "step_physics": 0.12557300739145397, "survival_time": 59.99999999999873, "driven_lanedir": 5.871997736717288, "get_state_dump": 0.0046341339813283245, "get_robot_state": 0.00365869171117168, "sim_render-ego0": 0.003709331539449446, "get_duckie_state": 1.3008800573293415e-06, "in-drivable-lane": 12.349999999999715, "deviation-heading": 13.187428086900828, "agent_compute-ego0": 0.011618878025496432, "complete-iteration": 0.19101453978850583, "set_robot_commands": 0.002204124377629441, "deviation-center-line": 3.7035946718366017, "driven_lanedir_consec": 5.871997736717288, "sim_compute_sim_state": 0.009460949877914442, "sim_compute_performance-ego0": 0.0019787630371805236}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.9837496920757972, "get_ui_image": 0.033469134218552535, "step_physics": 0.17651374340057374, "survival_time": 8.449999999999985, "driven_lanedir": 0.5359465769545191, "get_state_dump": 0.0044454378240248735, "get_robot_state": 0.0034397153293385225, "sim_render-ego0": 0.0035186627331902, "get_duckie_state": 1.1163599350873162e-06, "in-drivable-lane": 3.099999999999996, "deviation-heading": 2.70556207067436, "agent_compute-ego0": 0.01109488571391386, "complete-iteration": 0.2459871165892657, "set_robot_commands": 0.0020373849307789523, "deviation-center-line": 0.3150810749223459, "driven_lanedir_consec": 0.5359465769545191, "sim_compute_sim_state": 0.00956271255717558, "sim_compute_performance-ego0": 0.00182952319874483}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.913973288576705, "get_ui_image": 0.032667526297525604, "step_physics": 0.160430705716072, "survival_time": 59.99999999999873, "driven_lanedir": 5.160882417301926, "get_state_dump": 0.004725623388870074, "get_robot_state": 0.0037719881802573984, "sim_render-ego0": 0.0038094322052129, "get_duckie_state": 1.352097370741667e-06, "in-drivable-lane": 14.949999999999552, "deviation-heading": 17.205885331872494, "agent_compute-ego0": 0.012235636318057503, "complete-iteration": 0.23540213642072716, "set_robot_commands": 0.0022881434025315817, "deviation-center-line": 4.593470679684905, "driven_lanedir_consec": 5.160882417301926, "sim_compute_sim_state": 0.01331223139258646, "sim_compute_performance-ego0": 0.0020709246223316303}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.920199218809262, "get_ui_image": 0.02488455109354062, "step_physics": 0.12684483353442494, "survival_time": 59.99999999999873, "driven_lanedir": 2.2480482537939617, "get_state_dump": 0.00451910882865658, "get_robot_state": 0.0035552531853008033, "sim_render-ego0": 0.003603727593211508, "get_duckie_state": 1.243707242357443e-06, "in-drivable-lane": 37.64999999999908, "deviation-heading": 12.85102503084778, "agent_compute-ego0": 0.011295430368428226, "complete-iteration": 0.18464988316227057, "set_robot_commands": 0.002106059897849204, "deviation-center-line": 2.2289442764134324, "driven_lanedir_consec": 2.2480482537939617, "sim_compute_sim_state": 0.005877003483133054, "sim_compute_performance-ego0": 0.001884847159786685}}
set_robot_commands_max0.0022881434025315817
set_robot_commands_mean0.0021589281521972948
set_robot_commands_median0.0021550921377393225
set_robot_commands_min0.0020373849307789523
sim_compute_performance-ego0_max0.0020709246223316303
sim_compute_performance-ego0_mean0.0019410145045109172
sim_compute_performance-ego0_median0.0019318050984836045
sim_compute_performance-ego0_min0.00182952319874483
sim_compute_sim_state_max0.01331223139258646
sim_compute_sim_state_mean0.009553224327702383
sim_compute_sim_state_median0.009511831217545012
sim_compute_sim_state_min0.005877003483133054
sim_render-ego0_max0.0038094322052129
sim_render-ego0_mean0.0036602885177660135
sim_render-ego0_median0.003656529566330477
sim_render-ego0_min0.0035186627331902
simulation-passed1
step_physics_max0.17651374340057374
step_physics_mean0.14734057251063115
step_physics_median0.14363776962524846
step_physics_min0.12557300739145397
survival_time_max59.99999999999873
survival_time_mean47.11249999999904
survival_time_min8.449999999999985
No reset possible
54278LFv-simsuccessyes0:33:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54274LFv-simsuccessyes0:29:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54271LFv-simsuccessyes0:25:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49372LFv-simerrorno0:17:27
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139762823331744
- M:video_aido:cmdline(in:/;out:/) 139762822814688
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49364LFv-simerrorno0:08:32
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140695143292880
- M:video_aido:cmdline(in:/;out:/) 140695143163024
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49354LFv-simerrorno0:10:05
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140428494570832
- M:video_aido:cmdline(in:/;out:/) 140428494882752
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43117LFv-simsuccessno0:09:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43116LFv-simsuccessno0:09:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43115LFv-simsuccessno0:11:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible