Duckietown Challenges Home Challenges Submissions

Submission 11456

Submission11456
Competingyes
Challengeaido5-LF-sim-validation
UserMo Kleit 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54545
Next
User labelreal-exercise-1-fallback
Admin priority50
Blessingn/a
User priority50

54545

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54545LFv-simsuccessyes0:20:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4803787475750393
survival_time_median21.900000000000176
deviation-center-line_median0.6329685520423263
in-drivable-lane_median13.375000000000156


other stats
agent_compute-ego0_max0.012922136790890942
agent_compute-ego0_mean0.012465012366448418
agent_compute-ego0_median0.012468275530055289
agent_compute-ego0_min0.012001361614792148
complete-iteration_max0.3453382609726547
complete-iteration_mean0.3034045320326713
complete-iteration_median0.30915280963310976
complete-iteration_min0.24997424789181105
deviation-center-line_max0.6959443550502984
deviation-center-line_mean0.5719886646092519
deviation-center-line_min0.3260731993020567
deviation-heading_max5.878141564086586
deviation-heading_mean3.6695622428442936
deviation-heading_median3.586681669102713
deviation-heading_min1.626744069085163
driven_any_max3.0230607461928347
driven_any_mean1.750349376797719
driven_any_median1.4577218484880516
driven_any_min1.0628930640219365
driven_lanedir_consec_max0.6406760173617714
driven_lanedir_consec_mean0.46708330866502346
driven_lanedir_consec_min0.26689972214824387
driven_lanedir_max0.6406760173617714
driven_lanedir_mean0.46708330866502346
driven_lanedir_median0.4803787475750393
driven_lanedir_min0.26689972214824387
get_duckie_state_max1.5238374015971423e-06
get_duckie_state_mean1.4360008757398394e-06
get_duckie_state_median1.4347701662388902e-06
get_duckie_state_min1.350625768884436e-06
get_robot_state_max0.004161201121245684
get_robot_state_mean0.003896413317112184
get_robot_state_median0.003867903609797972
get_robot_state_min0.003688644927607105
get_state_dump_max0.005099653104029664
get_state_dump_mean0.004848487654007628
get_state_dump_median0.0047832188720551205
get_state_dump_min0.004727859767890605
get_ui_image_max0.0363426635791729
get_ui_image_mean0.03190844343973665
get_ui_image_median0.0317210840684629
get_ui_image_min0.02784894204284792
in-drivable-lane_max37.7999999999997
in-drivable-lane_mean17.700000000000024
in-drivable-lane_min6.250000000000089
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.0230607461928347, "get_ui_image": 0.02843898796847063, "step_physics": 0.20851212450992967, "survival_time": 44.29999999999962, "driven_lanedir": 0.26689972214824387, "get_state_dump": 0.004819625394320138, "get_robot_state": 0.003874348949015208, "sim_render-ego0": 0.003946723368149062, "get_duckie_state": 1.4912584857446096e-06, "in-drivable-lane": 37.7999999999997, "deviation-heading": 5.227393545584871, "agent_compute-ego0": 0.012659601858799193, "complete-iteration": 0.27731446053290987, "set_robot_commands": 0.002305717586530236, "deviation-center-line": 0.6959443550502984, "driven_lanedir_consec": 0.26689972214824387, "sim_compute_sim_state": 0.010524283939176643, "sim_compute_performance-ego0": 0.0021352770632256783}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.269042606354128, "get_ui_image": 0.0363426635791729, "step_physics": 0.26893474838950415, "survival_time": 19.200000000000134, "driven_lanedir": 0.3522920386226194, "get_state_dump": 0.004746812349790103, "get_robot_state": 0.0038614582705807374, "sim_render-ego0": 0.003849770806052468, "get_duckie_state": 1.350625768884436e-06, "in-drivable-lane": 11.450000000000095, "deviation-heading": 5.878141564086586, "agent_compute-ego0": 0.012276949201311385, "complete-iteration": 0.3453382609726547, "set_robot_commands": 0.0022172921663754946, "deviation-center-line": 0.6742705330320855, "driven_lanedir_consec": 0.3522920386226194, "sim_compute_sim_state": 0.010937862272386427, "sim_compute_performance-ego0": 0.002077441401295848}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0628930640219365, "get_ui_image": 0.03500318016845516, "step_physics": 0.2650889752472577, "survival_time": 16.300000000000097, "driven_lanedir": 0.6406760173617714, "get_state_dump": 0.005099653104029664, "get_robot_state": 0.004161201121245684, "sim_render-ego0": 0.004264911744937255, "get_duckie_state": 1.5238374015971423e-06, "in-drivable-lane": 6.250000000000089, "deviation-heading": 1.9459697926205544, "agent_compute-ego0": 0.012922136790890942, "complete-iteration": 0.3409911587333096, "set_robot_commands": 0.0024502160716858845, "deviation-center-line": 0.3260731993020567, "driven_lanedir_consec": 0.6406760173617714, "sim_compute_sim_state": 0.009563065450125878, "sim_compute_performance-ego0": 0.002333885303695632}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6464010906219757, "get_ui_image": 0.02784894204284792, "step_physics": 0.1883817207498918, "survival_time": 24.600000000000215, "driven_lanedir": 0.6084654565274592, "get_state_dump": 0.004727859767890605, "get_robot_state": 0.003688644927607105, "sim_render-ego0": 0.003755270104862855, "get_duckie_state": 1.3782818467331703e-06, "in-drivable-lane": 15.300000000000216, "deviation-heading": 1.626744069085163, "agent_compute-ego0": 0.012001361614792148, "complete-iteration": 0.24997424789181105, "set_robot_commands": 0.002244096740506, "deviation-center-line": 0.5916665710525671, "driven_lanedir_consec": 0.6084654565274592, "sim_compute_sim_state": 0.005271960465467967, "sim_compute_performance-ego0": 0.001963059752272786}}
set_robot_commands_max0.0024502160716858845
set_robot_commands_mean0.0023043306412744036
set_robot_commands_median0.002274907163518118
set_robot_commands_min0.0022172921663754946
sim_compute_performance-ego0_max0.002333885303695632
sim_compute_performance-ego0_mean0.002127415880122486
sim_compute_performance-ego0_median0.002106359232260763
sim_compute_performance-ego0_min0.001963059752272786
sim_compute_sim_state_max0.010937862272386427
sim_compute_sim_state_mean0.00907429303178923
sim_compute_sim_state_median0.01004367469465126
sim_compute_sim_state_min0.005271960465467967
sim_render-ego0_max0.004264911744937255
sim_render-ego0_mean0.00395416900600041
sim_render-ego0_median0.0038982470871007646
sim_render-ego0_min0.003755270104862855
simulation-passed1
step_physics_max0.26893474838950415
step_physics_mean0.23272939222414585
step_physics_median0.2368005498785937
step_physics_min0.1883817207498918
survival_time_max44.29999999999962
survival_time_mean26.100000000000016
survival_time_min16.300000000000097
No reset possible
49717LFv-simerrorno0:08:26
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140107308563760
- M:video_aido:cmdline(in:/;out:/) 140107142980272
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49715LFv-simerrorno0:03:49
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140645980154224
- M:video_aido:cmdline(in:/;out:/) 140645980152784
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49708LFv-simhost-errorno0:08:05
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49702LFv-simerrorno0:04:19
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140621786216672
- M:video_aido:cmdline(in:/;out:/) 140621786216720
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42449LFv-simsuccessno0:07:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42448LFv-simsuccessno0:10:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42447LFv-simsuccessno0:10:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible