Duckietown Challenges Home Challenges Submissions

Submission 11667

Submission11667
Competingyes
Challengeaido5-LF-sim-validation
UserDaniil Lisus
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54098
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54098

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54098LFv-simsuccessyes0:14:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.737510048544816
survival_time_median21.725000000000172
deviation-center-line_median1.1602908900754918
in-drivable-lane_median8.975000000000104


other stats
agent_compute-ego0_max0.012698405318790011
agent_compute-ego0_mean0.012322342025978225
agent_compute-ego0_median0.01247194834184552
agent_compute-ego0_min0.011647066101431848
complete-iteration_max0.24404177069664
complete-iteration_mean0.20540182842690455
complete-iteration_median0.2000365304374936
complete-iteration_min0.17749248213599098
deviation-center-line_max2.5552301113472
deviation-center-line_mean1.239093673691028
deviation-center-line_min0.08056280326592859
deviation-heading_max9.097975707854845
deviation-heading_mean4.633995397115041
deviation-heading_median4.483298949405437
deviation-heading_min0.4714079817944466
driven_any_max14.385431858570566
driven_any_mean6.749342469167778
driven_any_median5.647751425808755
driven_any_min1.3164351664830334
driven_lanedir_consec_max9.656713432077334
driven_lanedir_consec_mean3.885922582205557
driven_lanedir_consec_min0.4119567996552602
driven_lanedir_max9.656713432077334
driven_lanedir_mean4.15885796895112
driven_lanedir_median3.2833808220359426
driven_lanedir_min0.4119567996552602
get_duckie_state_max2.131407911127264e-06
get_duckie_state_mean2.0441604287533637e-06
get_duckie_state_median2.0531101017683817e-06
get_duckie_state_min1.9390136003494263e-06
get_robot_state_max0.003793797322681972
get_robot_state_mean0.0036694967343364735
get_robot_state_median0.0037457829386047543
get_robot_state_min0.0033926237374544144
get_state_dump_max0.004713454300707037
get_state_dump_mean0.004578811774570869
get_state_dump_median0.004657173439925437
get_state_dump_min0.004287445917725563
get_ui_image_max0.03339545056223869
get_ui_image_mean0.03029604148891958
get_ui_image_median0.03080394630781328
get_ui_image_min0.02618082277781307
in-drivable-lane_max13.599999999999646
in-drivable-lane_mean9.07499999999996
in-drivable-lane_min4.749999999999985
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 14.385431858570566, "get_ui_image": 0.028705893592400986, "step_physics": 0.11775935238057916, "survival_time": 43.94999999999964, "driven_lanedir": 9.656713432077334, "get_state_dump": 0.004713454300707037, "get_robot_state": 0.003729248588735407, "sim_render-ego0": 0.0037797749042510985, "get_duckie_state": 2.131407911127264e-06, "in-drivable-lane": 13.599999999999646, "deviation-heading": 9.097975707854845, "agent_compute-ego0": 0.012617286497896368, "complete-iteration": 0.18520577035167, "set_robot_commands": 0.00220848565751856, "deviation-center-line": 2.5552301113472, "driven_lanedir_consec": 9.656713432077334, "sim_compute_sim_state": 0.00959720719944347, "sim_compute_performance-ego0": 0.0020050520246679133}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3164351664830334, "get_ui_image": 0.03339545056223869, "step_physics": 0.1746907066553831, "survival_time": 6.349999999999985, "driven_lanedir": 0.4119567996552602, "get_state_dump": 0.004287445917725563, "get_robot_state": 0.0033926237374544144, "sim_render-ego0": 0.0034792181104421616, "get_duckie_state": 1.9390136003494263e-06, "in-drivable-lane": 4.749999999999985, "deviation-heading": 0.4714079817944466, "agent_compute-ego0": 0.011647066101431848, "complete-iteration": 0.24404177069664, "set_robot_commands": 0.0020414888858795166, "deviation-center-line": 0.08056280326592859, "driven_lanedir_consec": 0.4119567996552602, "sim_compute_sim_state": 0.00918952375650406, "sim_compute_performance-ego0": 0.0018365755677223208}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.421514273352935, "get_ui_image": 0.032901999023225575, "step_physics": 0.14123015555124435, "survival_time": 25.150000000000222, "driven_lanedir": 4.610677065516508, "get_state_dump": 0.004686116699188474, "get_robot_state": 0.003793797322681972, "sim_render-ego0": 0.0038357069568028526, "get_duckie_state": 2.0591985611688525e-06, "in-drivable-lane": 9.100000000000112, "deviation-heading": 5.154382574791781, "agent_compute-ego0": 0.012698405318790011, "complete-iteration": 0.21486729052331713, "set_robot_commands": 0.00226519505182902, "deviation-center-line": 1.6780905020214054, "driven_lanedir_consec": 3.5189355185342546, "sim_compute_sim_state": 0.011333397456577844, "sim_compute_performance-ego0": 0.002031578431053767}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.873988578264575, "get_ui_image": 0.02618082277781307, "step_physics": 0.11646373590266672, "survival_time": 18.300000000000125, "driven_lanedir": 1.9560845785553775, "get_state_dump": 0.0046282301806624, "get_robot_state": 0.003762317288474101, "sim_render-ego0": 0.0037148622466043166, "get_duckie_state": 2.0470216423679113e-06, "in-drivable-lane": 8.850000000000097, "deviation-heading": 3.812215324019093, "agent_compute-ego0": 0.012326610185794677, "complete-iteration": 0.17749248213599098, "set_robot_commands": 0.00212357349551666, "deviation-center-line": 0.6424912781295781, "driven_lanedir_consec": 1.9560845785553775, "sim_compute_sim_state": 0.006280702206354375, "sim_compute_performance-ego0": 0.0019252287269613073}}
set_robot_commands_max0.00226519505182902
set_robot_commands_mean0.0021596857726859393
set_robot_commands_median0.00216602957651761
set_robot_commands_min0.0020414888858795166
sim_compute_performance-ego0_max0.002031578431053767
sim_compute_performance-ego0_mean0.001949608687601327
sim_compute_performance-ego0_median0.00196514037581461
sim_compute_performance-ego0_min0.0018365755677223208
sim_compute_sim_state_max0.011333397456577844
sim_compute_sim_state_mean0.009100207654719935
sim_compute_sim_state_median0.009393365477973764
sim_compute_sim_state_min0.006280702206354375
sim_render-ego0_max0.0038357069568028526
sim_render-ego0_mean0.003702390554525107
sim_render-ego0_median0.003747318575427707
sim_render-ego0_min0.0034792181104421616
simulation-passed1
step_physics_max0.1746907066553831
step_physics_mean0.13753598762246833
step_physics_median0.12949475396591176
step_physics_min0.11646373590266672
survival_time_max43.94999999999964
survival_time_mean23.437499999999993
survival_time_min6.349999999999985
No reset possible
54095LFv-simsuccessyes0:19:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54092LFv-simsuccessyes0:21:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49144LFv-simerrorno0:09:29
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140330273817312
- M:video_aido:cmdline(in:/;out:/) 140330273820000
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43397LFv-simsuccessno0:07:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43396LFv-simhost-errorno0:00:25
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43393LFv-simsuccessno0:12:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43391LFv-simhost-errorno0:00:20
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible