Duckietown Challenges Home Challenges Submissions

Submission 10828

Submission10828
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57782
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57782

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57782LFv-simsuccessyes0:17:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9506120905239176
survival_time_median17.525000000000112
deviation-center-line_median0.5383305787781117
in-drivable-lane_median7.624999999999939


other stats
agent_compute-ego0_max0.014385091575756997
agent_compute-ego0_mean0.013919698488714334
agent_compute-ego0_median0.014030681424417615
agent_compute-ego0_min0.013232339530265104
complete-iteration_max0.24197951287425035
complete-iteration_mean0.2053225716517379
complete-iteration_median0.2010483511587768
complete-iteration_min0.1772140714151476
deviation-center-line_max2.101169817977628
deviation-center-line_mean0.8350698214158518
deviation-center-line_min0.1624483101295559
deviation-heading_max11.302439989394324
deviation-heading_mean4.4410253302504215
deviation-heading_median2.792058766960538
deviation-heading_min0.8775437976862893
driven_any_max9.161070186385126
driven_any_mean3.894132212851777
driven_any_median2.4535666914551513
driven_any_min1.5083252821116786
driven_lanedir_consec_max7.61328390463054
driven_lanedir_consec_mean2.5335817750022027
driven_lanedir_consec_min0.619819014330435
driven_lanedir_max7.61328390463054
driven_lanedir_mean2.5338839724696394
driven_lanedir_median0.9512164854587912
driven_lanedir_min0.619819014330435
get_duckie_state_max2.0207812607550934e-06
get_duckie_state_mean1.809814642715934e-06
get_duckie_state_median1.8102281151773175e-06
get_duckie_state_min1.5980210797540072e-06
get_robot_state_max0.004248262887977692
get_robot_state_mean0.004154707213144685
get_robot_state_median0.004172001064013841
get_robot_state_min0.004026563836573363
get_state_dump_max0.005410791493722521
get_state_dump_mean0.0052573035187118014
get_state_dump_median0.005289217234565815
get_state_dump_min0.005039988111993058
get_ui_image_max0.03882624031688674
get_ui_image_mean0.03253500230140685
get_ui_image_median0.031946685812164505
get_ui_image_min0.02742039726441165
in-drivable-lane_max12.550000000000145
in-drivable-lane_mean8.675000000000015
in-drivable-lane_min6.900000000000038
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 9.161070186385126, "get_ui_image": 0.028994368077515563, "step_physics": 0.1069424346920652, "survival_time": 59.99999999999873, "driven_lanedir": 7.61328390463054, "get_state_dump": 0.005039988111993058, "get_robot_state": 0.004026563836573363, "sim_render-ego0": 0.004165312729707666, "get_duckie_state": 1.74853525789056e-06, "in-drivable-lane": 7.599999999999827, "deviation-heading": 11.302439989394324, "agent_compute-ego0": 0.013232339530265104, "complete-iteration": 0.1772140714151476, "set_robot_commands": 0.0023777417398114487, "deviation-center-line": 2.101169817977628, "driven_lanedir_consec": 7.61328390463054, "sim_compute_sim_state": 0.010064125061035156, "sim_compute_performance-ego0": 0.0022599842030241726}, "LF-norm-zigzag-000-ego0": {"driven_any": 3.206665065163666, "get_ui_image": 0.03882624031688674, "step_physics": 0.15792489682000113, "survival_time": 22.650000000000187, "driven_lanedir": 1.2555706639880972, "get_state_dump": 0.005410791493722521, "get_robot_state": 0.004210486811163142, "sim_render-ego0": 0.004385901442708423, "get_duckie_state": 2.0207812607550934e-06, "in-drivable-lane": 12.550000000000145, "deviation-heading": 4.689623104124674, "agent_compute-ego0": 0.014385091575756997, "complete-iteration": 0.24197951287425035, "set_robot_commands": 0.002486452132069592, "deviation-center-line": 0.9005979481042476, "driven_lanedir_consec": 1.2543618741183504, "sim_compute_sim_state": 0.011850120212538126, "sim_compute_performance-ego0": 0.00237342176983535}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.5083252821116786, "get_ui_image": 0.03489900354681344, "step_physics": 0.13904460545243888, "survival_time": 11.55000000000003, "driven_lanedir": 0.619819014330435, "get_state_dump": 0.005387689532904789, "get_robot_state": 0.00413351531686454, "sim_render-ego0": 0.004411643949048272, "get_duckie_state": 1.5980210797540072e-06, "in-drivable-lane": 6.900000000000038, "deviation-heading": 0.8944944297964016, "agent_compute-ego0": 0.014372513212006667, "complete-iteration": 0.2170354888356965, "set_robot_commands": 0.0024631413920172328, "deviation-center-line": 0.1624483101295559, "driven_lanedir_consec": 0.619819014330435, "sim_compute_sim_state": 0.009914001514171732, "sim_compute_performance-ego0": 0.0022979779490109147}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.7004683177466364, "get_ui_image": 0.02742039726441165, "step_physics": 0.1189974982096967, "survival_time": 12.40000000000004, "driven_lanedir": 0.646862306929485, "get_state_dump": 0.005190744936226841, "get_robot_state": 0.004248262887977692, "sim_render-ego0": 0.00440309229624798, "get_duckie_state": 1.871920972464075e-06, "in-drivable-lane": 7.65000000000005, "deviation-heading": 0.8775437976862893, "agent_compute-ego0": 0.013688849636828563, "complete-iteration": 0.1850612134818571, "set_robot_commands": 0.002595121123225814, "deviation-center-line": 0.17606320945197587, "driven_lanedir_consec": 0.646862306929485, "sim_compute_sim_state": 0.0060056385745006395, "sim_compute_performance-ego0": 0.002395198048357983}}
set_robot_commands_max0.002595121123225814
set_robot_commands_mean0.002480614096781022
set_robot_commands_median0.0024747967620434123
set_robot_commands_min0.0023777417398114487
sim_compute_performance-ego0_max0.002395198048357983
sim_compute_performance-ego0_mean0.0023316454925571047
sim_compute_performance-ego0_median0.0023356998594231326
sim_compute_performance-ego0_min0.0022599842030241726
sim_compute_sim_state_max0.011850120212538126
sim_compute_sim_state_mean0.009458471340561414
sim_compute_sim_state_median0.009989063287603444
sim_compute_sim_state_min0.0060056385745006395
sim_render-ego0_max0.004411643949048272
sim_render-ego0_mean0.0043414876044280855
sim_render-ego0_median0.004394496869478202
sim_render-ego0_min0.004165312729707666
simulation-passed1
step_physics_max0.15792489682000113
step_physics_mean0.1307273587935505
step_physics_median0.1290210518310678
step_physics_min0.1069424346920652
survival_time_max59.99999999999873
survival_time_mean26.64999999999975
survival_time_min11.55000000000003
No reset possible
51560LFv-simerrorno0:05:54
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139855297669392
- M:video_aido:cmdline(in:/;out:/) 139855297669872
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51557LFv-simerrorno0:08:33
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140354644928784
- M:video_aido:cmdline(in:/;out:/) 140354643998656
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40890LFv-simsuccessno0:09:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40889LFv-simsuccessno0:09:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38061LFv-simsuccessno0:11:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible