Duckietown Challenges Home Challenges Submissions

Submission 10915

Submission10915
Competingyes
Challengeaido5-LF-sim-validation
UserDaniil Lisus
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57464
Next
User labelsim-exercise-1
Admin priority50
Blessingn/a
User priority50

57464

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57464LFv-simsuccessyes0:29:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.851770025942482
survival_time_median59.99999999999873
deviation-center-line_median3.625385221542503
in-drivable-lane_median3.4249999999999483


other stats
agent_compute-ego0_max0.013146631688956757
agent_compute-ego0_mean0.01295874000813052
agent_compute-ego0_median0.012971353074295335
agent_compute-ego0_min0.012745622194974647
complete-iteration_max0.19985135290446032
complete-iteration_mean0.1830935320428524
complete-iteration_median0.182562778045525
complete-iteration_min0.16739721917589936
deviation-center-line_max5.086352661550912
deviation-center-line_mean3.133935121106667
deviation-center-line_min0.19861737979074995
deviation-heading_max28.24899418168409
deviation-heading_mean15.381033204465478
deviation-heading_median16.208401326697153
deviation-heading_min0.8583359827835195
driven_any_max10.861374921397369
driven_any_mean7.634157579457584
driven_any_median8.71639400772347
driven_any_min2.242467380986036
driven_lanedir_consec_max7.3765995092623085
driven_lanedir_consec_mean4.300754330211445
driven_lanedir_consec_min0.12287775969850843
driven_lanedir_max9.983719996344838
driven_lanedir_mean6.155136422044015
driven_lanedir_median7.256973966066358
driven_lanedir_min0.12287775969850843
get_duckie_state_max2.08581516288103e-06
get_duckie_state_mean1.9037570946480851e-06
get_duckie_state_median1.8891478785984604e-06
get_duckie_state_min1.750917458514389e-06
get_robot_state_max0.0039470219592269914
get_robot_state_mean0.0038094358970558223
get_robot_state_median0.003783851180897884
get_robot_state_min0.0037230192672005305
get_state_dump_max0.004919031279767979
get_state_dump_mean0.004702261243919802
get_state_dump_median0.004659337658546959
get_state_dump_min0.004571338378817315
get_ui_image_max0.035327464714336156
get_ui_image_mean0.031184714430988147
get_ui_image_median0.03070600836583848
get_ui_image_min0.027999376277939463
in-drivable-lane_max14.8500000000001
in-drivable-lane_mean6.074999999999968
in-drivable-lane_min2.5999999999998735
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.242467380986036, "get_ui_image": 0.028912107628512096, "step_physics": 0.113934409905629, "survival_time": 16.5500000000001, "driven_lanedir": 0.12287775969850843, "get_state_dump": 0.0047168544976108045, "get_robot_state": 0.0037638889737876065, "sim_render-ego0": 0.004008150962461908, "get_duckie_state": 1.966235149337585e-06, "in-drivable-lane": 14.8500000000001, "deviation-heading": 0.8583359827835195, "agent_compute-ego0": 0.013146631688956757, "complete-iteration": 0.18121329991214247, "set_robot_commands": 0.0022510879011039273, "deviation-center-line": 0.19861737979074995, "driven_lanedir_consec": 0.12287775969850843, "sim_compute_sim_state": 0.008379771048764148, "sim_compute_performance-ego0": 0.002012052449835352}, "LF-norm-zigzag-000-ego0": {"driven_any": 8.391551297504453, "get_ui_image": 0.035327464714336156, "step_physics": 0.12258324555612224, "survival_time": 59.99999999999873, "driven_lanedir": 6.442333918108053, "get_state_dump": 0.004571338378817315, "get_robot_state": 0.0037230192672005305, "sim_render-ego0": 0.003960475040216628, "get_duckie_state": 1.812060607859336e-06, "in-drivable-lane": 3.799999999999964, "deviation-heading": 28.24899418168409, "agent_compute-ego0": 0.012838785694004791, "complete-iteration": 0.19985135290446032, "set_robot_commands": 0.0022562078194852475, "deviation-center-line": 5.086352661550912, "driven_lanedir_consec": 2.899796605767963, "sim_compute_sim_state": 0.01242979519770207, "sim_compute_performance-ego0": 0.002073282405399065}, "LF-norm-techtrack-000-ego0": {"driven_any": 10.861374921397369, "get_ui_image": 0.032499909103164866, "step_physics": 0.10878699447193511, "survival_time": 59.99999999999873, "driven_lanedir": 9.983719996344838, "get_state_dump": 0.004601820819483113, "get_robot_state": 0.003803813388008162, "sim_render-ego0": 0.003987376239277938, "get_duckie_state": 1.750917458514389e-06, "in-drivable-lane": 2.5999999999998735, "deviation-heading": 15.375161317159684, "agent_compute-ego0": 0.012745622194974647, "complete-iteration": 0.18391225617890752, "set_robot_commands": 0.0022886730451369464, "deviation-center-line": 3.152257027779147, "driven_lanedir_consec": 7.3765995092623085, "sim_compute_sim_state": 0.013016785908301208, "sim_compute_performance-ego0": 0.002095312202701362}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.041236717942484, "get_ui_image": 0.027999376277939463, "step_physics": 0.10202351279500918, "survival_time": 59.99999999999873, "driven_lanedir": 8.071614014024663, "get_state_dump": 0.004919031279767979, "get_robot_state": 0.0039470219592269914, "sim_render-ego0": 0.004164857927905233, "get_duckie_state": 2.08581516288103e-06, "in-drivable-lane": 3.0499999999999323, "deviation-heading": 17.041641336234623, "agent_compute-ego0": 0.01310392045458588, "complete-iteration": 0.16739721917589936, "set_robot_commands": 0.00240948476164069, "deviation-center-line": 4.0985134153058596, "driven_lanedir_consec": 6.803743446117, "sim_compute_sim_state": 0.006580719840615913, "sim_compute_performance-ego0": 0.0021562627908292163}}
set_robot_commands_max0.00240948476164069
set_robot_commands_mean0.002301363381841703
set_robot_commands_median0.002272440432311097
set_robot_commands_min0.0022510879011039273
sim_compute_performance-ego0_max0.0021562627908292163
sim_compute_performance-ego0_mean0.002084227462191249
sim_compute_performance-ego0_median0.0020842973040502135
sim_compute_performance-ego0_min0.002012052449835352
sim_compute_sim_state_max0.013016785908301208
sim_compute_sim_state_mean0.010101767998845835
sim_compute_sim_state_median0.010404783123233109
sim_compute_sim_state_min0.006580719840615913
sim_render-ego0_max0.004164857927905233
sim_render-ego0_mean0.004030215042465427
sim_render-ego0_median0.003997763600869923
sim_render-ego0_min0.003960475040216628
simulation-passed1
step_physics_max0.12258324555612224
step_physics_mean0.1118320406821739
step_physics_median0.11136070218878204
step_physics_min0.10202351279500918
survival_time_max59.99999999999873
survival_time_mean49.13749999999907
survival_time_min16.5500000000001
No reset possible
51287LFv-simerrorno0:08:41
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140685515012464
- M:video_aido:cmdline(in:/;out:/) 140685515011648
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51285LFv-simerrorno0:09:23
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139672499606672
- M:video_aido:cmdline(in:/;out:/) 139672499606960
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51272LFv-simerrorno0:09:42
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140543554056976
- M:video_aido:cmdline(in:/;out:/) 140543553923488
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51263LFv-simerrorno0:15:42
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140668440297184
- M:video_aido:cmdline(in:/;out:/) 140668440757296
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51256LFv-simhost-errorno0:15:04
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40614LFv-simsuccessno0:11:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40613LFv-simsuccessno0:09:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39901LFv-simsuccessno0:09:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38457LFv-simsuccessno0:11:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38456LFv-simsuccessno0:09:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible