Duckietown Challenges Home Challenges Submissions

Submission 11027

Submission11027
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57080
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57080

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57080LFv-simsuccessyes0:26:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.7501445513028457
survival_time_median36.500000000000064
deviation-center-line_median1.4653203798914325
in-drivable-lane_median9.99999999999993


other stats
agent_compute-ego0_max0.021849232251995507
agent_compute-ego0_mean0.015018626539459203
agent_compute-ego0_median0.013014910034465432
agent_compute-ego0_min0.012195453836910447
complete-iteration_max0.2138088188520292
complete-iteration_mean0.1807323739734746
complete-iteration_median0.1776907944789937
complete-iteration_min0.1537390880838818
deviation-center-line_max2.223735617240879
deviation-center-line_mean1.5036831923970242
deviation-center-line_min0.8603563925643528
deviation-heading_max12.35350860085299
deviation-heading_mean7.814138773146677
deviation-heading_median7.535962120470743
deviation-heading_min3.831122250792234
driven_any_max9.420619371240576
driven_any_mean6.046119870970125
driven_any_median5.527582353695336
driven_any_min3.708695405249258
driven_lanedir_consec_max4.071210758342638
driven_lanedir_consec_mean3.445565228781769
driven_lanedir_consec_min2.2107610541787457
driven_lanedir_max7.069388151507211
driven_lanedir_mean4.3176960173600945
driven_lanedir_median3.9953174318772096
driven_lanedir_min2.2107610541787457
get_duckie_state_max1.411200312439155e-06
get_duckie_state_mean1.3190189964301785e-06
get_duckie_state_median1.3294433037340383e-06
get_duckie_state_min1.2059890658134822e-06
get_robot_state_max0.00367744027114496
get_robot_state_mean0.003607923061698255
get_robot_state_median0.003618324338570001
get_robot_state_min0.0035176032985080588
get_state_dump_max0.004664770539304549
get_state_dump_mean0.004540124640335054
get_state_dump_median0.004579231047899229
get_state_dump_min0.004337265926237209
get_ui_image_max0.03496333331596561
get_ui_image_mean0.029603629889096157
get_ui_image_median0.02967294018312004
get_ui_image_min0.02410530587417895
in-drivable-lane_max13.199999999999772
in-drivable-lane_mean10.124999999999932
in-drivable-lane_min7.3000000000001
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.0357622338358246, "get_ui_image": 0.027372301181900167, "step_physics": 0.0990010202859421, "survival_time": 32.05000000000032, "driven_lanedir": 3.9275028038180304, "get_state_dump": 0.004664770539304549, "get_robot_state": 0.0035886615978965876, "sim_render-ego0": 0.0038335884100180176, "get_duckie_state": 1.411200312439155e-06, "in-drivable-lane": 7.3000000000001, "deviation-heading": 3.831122250792234, "agent_compute-ego0": 0.012195453836910447, "complete-iteration": 0.16534732621035472, "set_robot_commands": 0.002108095962310506, "deviation-center-line": 1.0003324049076805, "driven_lanedir_consec": 3.9275028038180304, "sim_compute_sim_state": 0.010526144987326172, "sim_compute_performance-ego0": 0.001970063860171309}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.019402473554847, "get_ui_image": 0.03496333331596561, "step_physics": 0.13850990097697188, "survival_time": 40.94999999999981, "driven_lanedir": 4.063132059936389, "get_state_dump": 0.004552159076783715, "get_robot_state": 0.00367744027114496, "sim_render-ego0": 0.003935637997417915, "get_duckie_state": 1.32496764020222e-06, "in-drivable-lane": 10.249999999999734, "deviation-heading": 12.35350860085299, "agent_compute-ego0": 0.012568488935144936, "complete-iteration": 0.2138088188520292, "set_robot_commands": 0.002238111670424298, "deviation-center-line": 1.930308354875184, "driven_lanedir_consec": 3.57278629878766, "sim_compute_sim_state": 0.0112295546182772, "sim_compute_performance-ego0": 0.0020460937081313714}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.708695405249258, "get_ui_image": 0.03197357918433992, "step_physics": 0.11739364772917252, "survival_time": 25.250000000000224, "driven_lanedir": 2.2107610541787457, "get_state_dump": 0.004606303019014743, "get_robot_state": 0.003647987079243415, "sim_render-ego0": 0.0039728413457455845, "get_duckie_state": 1.3339189672658569e-06, "in-drivable-lane": 9.750000000000128, "deviation-heading": 3.880723117614693, "agent_compute-ego0": 0.013461331133785929, "complete-iteration": 0.19003426274763263, "set_robot_commands": 0.0022816884187841606, "deviation-center-line": 0.8603563925643528, "driven_lanedir_consec": 2.2107610541787457, "sim_compute_sim_state": 0.010619324186573858, "sim_compute_performance-ego0": 0.001985603641615555}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.420619371240576, "get_ui_image": 0.02410530587417895, "step_physics": 0.08613819980700745, "survival_time": 59.99999999999873, "driven_lanedir": 7.069388151507211, "get_state_dump": 0.004337265926237209, "get_robot_state": 0.0035176032985080588, "sim_render-ego0": 0.003717559263370714, "get_duckie_state": 1.2059890658134822e-06, "in-drivable-lane": 13.199999999999772, "deviation-heading": 11.191201123326794, "agent_compute-ego0": 0.021849232251995507, "complete-iteration": 0.1537390880838818, "set_robot_commands": 0.0021745783403255264, "deviation-center-line": 2.223735617240879, "driven_lanedir_consec": 4.071210758342638, "sim_compute_sim_state": 0.005938920450647308, "sim_compute_performance-ego0": 0.0018817479167750832}}
set_robot_commands_max0.0022816884187841606
set_robot_commands_mean0.002200618597961123
set_robot_commands_median0.0022063450053749125
set_robot_commands_min0.002108095962310506
sim_compute_performance-ego0_max0.0020460937081313714
sim_compute_performance-ego0_mean0.0019708772816733294
sim_compute_performance-ego0_median0.0019778337508934317
sim_compute_performance-ego0_min0.0018817479167750832
sim_compute_sim_state_max0.0112295546182772
sim_compute_sim_state_mean0.009578486060706135
sim_compute_sim_state_median0.010572734586950017
sim_compute_sim_state_min0.005938920450647308
sim_render-ego0_max0.0039728413457455845
sim_render-ego0_mean0.003864906754138058
sim_render-ego0_median0.0038846132037179664
sim_render-ego0_min0.003717559263370714
simulation-passed1
step_physics_max0.13850990097697188
step_physics_mean0.1102606921997735
step_physics_median0.10819733400755732
step_physics_min0.08613819980700745
survival_time_max59.99999999999873
survival_time_mean39.562499999999766
survival_time_min25.250000000000224
No reset possible
57075LFv-simsuccessyes0:16:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
57066LFv-simsuccessyes0:12:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50866LFv-simerrorno0:04:18
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140025158526480
- M:video_aido:cmdline(in:/;out:/) 140025158773008
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50847LFv-simerrorno0:02:15
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140224050569808
- M:video_aido:cmdline(in:/;out:/) 140224050507680
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40414LFv-simsuccessno0:09:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39737LFv-simsuccessno0:13:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38767LFv-simsuccessno0:09:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38765LFv-simsuccessno0:09:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38763LFv-simsuccessno0:09:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38762LFv-simhost-errorno0:00:12
Error while running [...]
Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, reg01-c51804ca78e4-1-job38762-626246, pull]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'reg01-c51804ca78e4-1-job38762-626246', 'pull']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38758LFv-simhost-errorno0:00:15
Error while running [...]
Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, reg02-2e50246280f2-1-job38758-576022, pull]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'reg02-2e50246280f2-1-job38758-576022', 'pull']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38756LFv-simhost-errorno0:00:13
Error while running [...]
Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, reg05-646c53230949-1-job38756-363582, pull]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'reg05-646c53230949-1-job38756-363582', 'pull']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38753LFv-simhost-errorno0:00:06
Error while running [...]
Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, mont01-7f6431e4d888-1-job38753-17046, pull]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'mont01-7f6431e4d888-1-job38753-17046', 'pull']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38749LFv-simhost-errorno0:00:11
Error while running [...]
Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, reg03-34ec761a5593-1-job38749-531086, pull]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'reg03-34ec761a5593-1-job38749-531086', 'pull']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38748LFv-simhost-errorno0:00:12
Error while running [...]
Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, reg03-34ec761a5593-1-job38748-606087, pull]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'reg03-34ec761a5593-1-job38748-606087', 'pull']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38746LFv-simhost-errorno0:00:11
Error while running [...]
Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, reg04-60113faf58e1-1-job38746-427352, pull]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'reg04-60113faf58e1-1-job38746-427352', 'pull']' returned non-zero exit status 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible