Duckietown Challenges Home Challenges Submissions

Submission 10880

Submission10880
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57675
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57675

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57675LFv-simsuccessyes0:15:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.7543928754914856
survival_time_median14.37500000000007
deviation-center-line_median0.3239962142760431
in-drivable-lane_median9.0999999999998


other stats
agent_compute-ego0_max0.014107608349523812
agent_compute-ego0_mean0.013761106659617533
agent_compute-ego0_median0.0138380953609117
agent_compute-ego0_min0.013260627567122917
complete-iteration_max0.2269798745223082
complete-iteration_mean0.1971155445727363
complete-iteration_median0.1924103599519185
complete-iteration_min0.17666158386480027
deviation-center-line_max2.5397351984808574
deviation-center-line_mean0.8383020141762215
deviation-center-line_min0.16548042967194254
deviation-heading_max10.872501799184295
deviation-heading_mean3.976522894213273
deviation-heading_median2.150159931079364
deviation-heading_min0.7332699155100665
driven_any_max10.826559313333398
driven_any_mean4.230085061450847
driven_any_median2.334682192320112
driven_any_min1.4244165478297672
driven_lanedir_consec_max8.365525774734932
driven_lanedir_consec_mean2.6208423910115783
driven_lanedir_consec_min0.6090580383284112
driven_lanedir_max8.365525774734932
driven_lanedir_mean2.6220400328415803
driven_lanedir_median0.756788159151489
driven_lanedir_min0.6090580383284112
get_duckie_state_max1.8096361003938268e-06
get_duckie_state_mean1.6002346984179936e-06
get_duckie_state_median1.5964872214225083e-06
get_duckie_state_min1.398328250433131e-06
get_robot_state_max0.0040719535181431175
get_robot_state_mean0.0039359948567203856
get_robot_state_median0.003919929585936005
get_robot_state_min0.0038321667368664135
get_state_dump_max0.004868645485633058
get_state_dump_mean0.004813306080406907
get_state_dump_median0.004809555181228446
get_state_dump_min0.004765468473537677
get_ui_image_max0.03594860483388432
get_ui_image_mean0.03131512132379265
get_ui_image_median0.03104844158931709
get_ui_image_min0.027214997282652097
in-drivable-lane_max11.550000000000091
in-drivable-lane_mean8.699999999999923
in-drivable-lane_min5.050000000000001
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.826559313333398, "get_ui_image": 0.029382961576526905, "step_physics": 0.10994902379705349, "survival_time": 59.99999999999873, "driven_lanedir": 8.365525774734932, "get_state_dump": 0.004765468473537677, "get_robot_state": 0.0038321667368664135, "sim_render-ego0": 0.00407817441160534, "get_duckie_state": 1.4727955356823417e-06, "in-drivable-lane": 11.549999999999578, "deviation-heading": 10.872501799184295, "agent_compute-ego0": 0.013260627567122917, "complete-iteration": 0.1793624006838326, "set_robot_commands": 0.002354340390499982, "deviation-center-line": 2.5397351984808574, "driven_lanedir_consec": 8.365525774734932, "sim_compute_sim_state": 0.00953305154716244, "sim_compute_performance-ego0": 0.002115932531301227}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.4244165478297672, "get_ui_image": 0.03594860483388432, "step_physics": 0.14856240397594014, "survival_time": 9.099999999999994, "driven_lanedir": 0.6090580383284112, "get_state_dump": 0.004868645485633058, "get_robot_state": 0.0040719535181431175, "sim_render-ego0": 0.004279282575096589, "get_duckie_state": 1.8096361003938268e-06, "in-drivable-lane": 5.050000000000001, "deviation-heading": 1.4678222541424188, "agent_compute-ego0": 0.0138573841970475, "complete-iteration": 0.2269798745223082, "set_robot_commands": 0.002368864465932377, "deviation-center-line": 0.24162296806898337, "driven_lanedir_consec": 0.6090580383284112, "sim_compute_sim_state": 0.010689119172226536, "sim_compute_performance-ego0": 0.002229230651438562}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.961509142140226, "get_ui_image": 0.032713921602107274, "step_physics": 0.1314301812615933, "survival_time": 18.10000000000012, "driven_lanedir": 0.8834578812551488, "get_state_dump": 0.004810155259019415, "get_robot_state": 0.003865249886000452, "sim_render-ego0": 0.004104013942161539, "get_duckie_state": 1.398328250433131e-06, "in-drivable-lane": 11.550000000000091, "deviation-heading": 2.83249760801631, "agent_compute-ego0": 0.0138188065247759, "complete-iteration": 0.20545831922000432, "set_robot_commands": 0.0023314381433912545, "deviation-center-line": 0.4063694604831029, "driven_lanedir_consec": 0.878667313935142, "sim_compute_sim_state": 0.010247429540334654, "sim_compute_performance-ego0": 0.002047557147738034}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.707855242499997, "get_ui_image": 0.027214997282652097, "step_physics": 0.11181696775917696, "survival_time": 10.650000000000016, "driven_lanedir": 0.6301184370478292, "get_state_dump": 0.0048089551034374775, "get_robot_state": 0.0039746092858715595, "sim_render-ego0": 0.004187335477811154, "get_duckie_state": 1.7201789071626751e-06, "in-drivable-lane": 6.650000000000023, "deviation-heading": 0.7332699155100665, "agent_compute-ego0": 0.014107608349523812, "complete-iteration": 0.17666158386480027, "set_robot_commands": 0.0023839228621153072, "deviation-center-line": 0.16548042967194254, "driven_lanedir_consec": 0.6301184370478292, "sim_compute_sim_state": 0.005866701357832579, "sim_compute_performance-ego0": 0.0021986749684699226}}
set_robot_commands_max0.0023839228621153072
set_robot_commands_mean0.00235964146548473
set_robot_commands_median0.0023616024282161795
set_robot_commands_min0.0023314381433912545
sim_compute_performance-ego0_max0.002229230651438562
sim_compute_performance-ego0_mean0.0021478488247369367
sim_compute_performance-ego0_median0.002157303749885575
sim_compute_performance-ego0_min0.002047557147738034
sim_compute_sim_state_max0.010689119172226536
sim_compute_sim_state_mean0.009084075404389052
sim_compute_sim_state_median0.009890240543748548
sim_compute_sim_state_min0.005866701357832579
sim_render-ego0_max0.004279282575096589
sim_render-ego0_mean0.004162201601668655
sim_render-ego0_median0.004145674709986346
sim_render-ego0_min0.00407817441160534
simulation-passed1
step_physics_max0.14856240397594014
step_physics_mean0.12543964419844095
step_physics_median0.12162357451038514
step_physics_min0.10994902379705349
survival_time_max59.99999999999873
survival_time_mean24.46249999999972
survival_time_min9.099999999999994
No reset possible
57674LFv-simsuccessyes0:14:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51392LFv-simerrorno0:09:49
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139856878033744
- M:video_aido:cmdline(in:/;out:/) 139856553986704
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51367LFv-simerrorno0:05:55
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140095607156496
- M:video_aido:cmdline(in:/;out:/) 140095606306080
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40667LFv-simsuccessno0:09:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38317LFv-simsuccessno0:11:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible