Duckietown Challenges Home Challenges Submissions

Submission 11550

Submission11550
Competingyes
Challengeaido5-LF-sim-validation
UserDaniil Lisus
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54402
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54402

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54402LFv-simsuccessyes0:25:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.693182074153977
survival_time_median43.149999999999615
deviation-center-line_median2.6532890479222235
in-drivable-lane_median7.1249999999998685


other stats
agent_compute-ego0_max0.012765472332636515
agent_compute-ego0_mean0.012514603242013034
agent_compute-ego0_median0.0125269548615014
agent_compute-ego0_min0.01223903091241282
complete-iteration_max0.2359629934983896
complete-iteration_mean0.21133926709592624
complete-iteration_median0.2120964493429136
complete-iteration_min0.1852011761994882
deviation-center-line_max4.6134471685727405
deviation-center-line_mean2.5064424562733736
deviation-center-line_min0.10574456067630618
deviation-heading_max13.058269996394296
deviation-heading_mean7.623928337318577
deviation-heading_median8.358806316786875
deviation-heading_min0.7198307193062613
driven_any_max19.60536790717253
driven_any_mean10.754027072673376
driven_any_median10.617378907847446
driven_any_min2.175982567826093
driven_lanedir_consec_max17.111403240184657
driven_lanedir_consec_mean8.694662669424481
driven_lanedir_consec_min0.2808832892053097
driven_lanedir_max17.111403240184657
driven_lanedir_mean8.694662669424481
driven_lanedir_median8.693182074153977
driven_lanedir_min0.2808832892053097
get_duckie_state_max1.567264772811026e-06
get_duckie_state_mean1.4351973109330572e-06
get_duckie_state_median1.4042890527823784e-06
get_duckie_state_min1.3649463653564452e-06
get_robot_state_max0.004035286633473523
get_robot_state_mean0.0038811696979210446
get_robot_state_median0.00384916603987944
get_robot_state_min0.0037910600784517743
get_state_dump_max0.004975771454145323
get_state_dump_mean0.0048463132379871765
get_state_dump_median0.004858248766150833
get_state_dump_min0.004692983965501718
get_ui_image_max0.0356851708381734
get_ui_image_mean0.03203740691864351
get_ui_image_median0.031846171014347974
get_ui_image_min0.02877211480770471
in-drivable-lane_max11.800000000000054
in-drivable-lane_mean7.7999999999999075
in-drivable-lane_min5.149999999999839
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 19.60536790717253, "get_ui_image": 0.028876083280323542, "step_physics": 0.11745502073302257, "survival_time": 59.99999999999873, "driven_lanedir": 17.111403240184657, "get_state_dump": 0.004790220530602855, "get_robot_state": 0.0037910600784517743, "sim_render-ego0": 0.003862193780179623, "get_duckie_state": 1.401528033686121e-06, "in-drivable-lane": 5.149999999999839, "deviation-heading": 13.058269996394296, "agent_compute-ego0": 0.012340835488706108, "complete-iteration": 0.1852011761994882, "set_robot_commands": 0.0022417706911212498, "deviation-center-line": 4.6134471685727405, "driven_lanedir_consec": 17.111403240184657, "sim_compute_sim_state": 0.009665540612607475, "sim_compute_performance-ego0": 0.0020944661244464654}, "LF-norm-zigzag-000-ego0": {"driven_any": 14.754198953754209, "get_ui_image": 0.0356851708381734, "step_physics": 0.15718951779054413, "survival_time": 56.349999999998936, "driven_lanedir": 12.32064480376362, "get_state_dump": 0.004692983965501718, "get_robot_state": 0.003802822836747406, "sim_render-ego0": 0.003895420343317884, "get_duckie_state": 1.407050071878636e-06, "in-drivable-lane": 6.799999999999631, "deviation-heading": 11.678289770427329, "agent_compute-ego0": 0.01223903091241282, "complete-iteration": 0.2359629934983896, "set_robot_commands": 0.0022996968411384744, "deviation-center-line": 3.549311076001201, "driven_lanedir_consec": 12.32064480376362, "sim_compute_sim_state": 0.013954942953502032, "sim_compute_performance-ego0": 0.002116510842708831}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.480558861940685, "get_ui_image": 0.034816258748372395, "step_physics": 0.1510678458213806, "survival_time": 29.95000000000029, "driven_lanedir": 5.065719344544332, "get_state_dump": 0.004926277001698812, "get_robot_state": 0.003895509243011475, "sim_render-ego0": 0.004086402257283529, "get_duckie_state": 1.3649463653564452e-06, "in-drivable-lane": 7.450000000000106, "deviation-heading": 5.039322863146424, "agent_compute-ego0": 0.012765472332636515, "complete-iteration": 0.23052736004193625, "set_robot_commands": 0.002400614817937215, "deviation-center-line": 1.7572670198432456, "driven_lanedir_consec": 5.065719344544332, "sim_compute_sim_state": 0.014301234086354574, "sim_compute_performance-ego0": 0.0021777458985646567}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.175982567826093, "get_ui_image": 0.02877211480770471, "step_physics": 0.12917337147694713, "survival_time": 13.200000000000053, "driven_lanedir": 0.2808832892053097, "get_state_dump": 0.004975771454145323, "get_robot_state": 0.004035286633473523, "sim_render-ego0": 0.004085359033548607, "get_duckie_state": 1.567264772811026e-06, "in-drivable-lane": 11.800000000000054, "deviation-heading": 0.7198307193062613, "agent_compute-ego0": 0.012713074234296693, "complete-iteration": 0.19366553864389097, "set_robot_commands": 0.002355464899315024, "deviation-center-line": 0.10574456067630618, "driven_lanedir_consec": 0.2808832892053097, "sim_compute_sim_state": 0.005371817102972067, "sim_compute_performance-ego0": 0.002095225172222785}}
set_robot_commands_max0.002400614817937215
set_robot_commands_mean0.0023243868123779907
set_robot_commands_median0.0023275808702267493
set_robot_commands_min0.0022417706911212498
sim_compute_performance-ego0_max0.0021777458985646567
sim_compute_performance-ego0_mean0.0021209870094856843
sim_compute_performance-ego0_median0.002105868007465808
sim_compute_performance-ego0_min0.0020944661244464654
sim_compute_sim_state_max0.014301234086354574
sim_compute_sim_state_mean0.010823383688859038
sim_compute_sim_state_median0.011810241783054754
sim_compute_sim_state_min0.005371817102972067
sim_render-ego0_max0.004086402257283529
sim_render-ego0_mean0.003982343853582411
sim_render-ego0_median0.003990389688433246
sim_render-ego0_min0.003862193780179623
simulation-passed1
step_physics_max0.15718951779054413
step_physics_mean0.13872143895547362
step_physics_median0.14012060864916387
step_physics_min0.11745502073302257
survival_time_max59.99999999999873
survival_time_mean39.8749999999995
survival_time_min13.200000000000053
No reset possible
49568LFv-simerrorno0:04:07
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140569353346736
- M:video_aido:cmdline(in:/;out:/) 140569354260976
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49562LFv-simerrorno0:11:04
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139820773131888
- M:video_aido:cmdline(in:/;out:/) 139820773496192
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42863LFv-simsuccessno0:06:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42862LFv-simsuccessno0:08:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42861LFv-simsuccessno0:10:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible