Duckietown Challenges Home Challenges Submissions

Submission 11645

Submission11645
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54172
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54172

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54172LFv-simsuccessyes0:28:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.3923370788362437
survival_time_median59.99999999999873
deviation-center-line_median0.519442662053919
in-drivable-lane_median28.224999999999213


other stats
agent_compute-ego0_max0.013071798265824011
agent_compute-ego0_mean0.0125018312697566
agent_compute-ego0_median0.012584193957834618
agent_compute-ego0_min0.011767138897533148
complete-iteration_max0.2427350901048646
complete-iteration_mean0.21538766216464805
complete-iteration_median0.2188757886872169
complete-iteration_min0.18106398117929373
deviation-center-line_max3.8902278242735218
deviation-center-line_mean1.3180426942513714
deviation-center-line_min0.34305762862412664
deviation-heading_max12.684303198996874
deviation-heading_mean5.689734838937544
deviation-heading_median4.1322166345832665
deviation-heading_min1.8102028875867675
driven_any_max13.363467970997872
driven_any_mean9.239237846932348
driven_any_median11.246351923265458
driven_any_min1.100779570200606
driven_lanedir_consec_max12.199185269549552
driven_lanedir_consec_mean3.958659708334872
driven_lanedir_consec_min0.8507794061174484
driven_lanedir_max12.199185269549552
driven_lanedir_mean3.963081339209427
driven_lanedir_median1.3923370788362437
driven_lanedir_min0.8684659296156699
get_duckie_state_max1.4587008486580197e-06
get_duckie_state_mean1.3472532832055358e-06
get_duckie_state_median1.337208616842735e-06
get_duckie_state_min1.2558950504786532e-06
get_robot_state_max0.004059072537386447
get_robot_state_mean0.003831525269933233
get_robot_state_median0.0038841935021990447
get_robot_state_min0.0034986415379483935
get_state_dump_max0.0050531414327375295
get_state_dump_mean0.004845174808354903
get_state_dump_median0.004930774834034941
get_state_dump_min0.004466008132612201
get_ui_image_max0.03511630985098825
get_ui_image_mean0.03192054317787429
get_ui_image_median0.03263168301213095
get_ui_image_min0.02730249683624699
in-drivable-lane_max53.59999999999883
in-drivable-lane_mean27.87499999999931
in-drivable-lane_min1.4499999999999948
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 13.363467970997872, "get_ui_image": 0.030581553313853243, "step_physics": 0.12924344037395036, "survival_time": 59.99999999999873, "driven_lanedir": 12.199185269549552, "get_state_dump": 0.004927803138015074, "get_robot_state": 0.003913514520802367, "sim_render-ego0": 0.0039894455775531705, "get_duckie_state": 1.3550751215214534e-06, "in-drivable-lane": 4.799999999999727, "deviation-heading": 12.684303198996874, "agent_compute-ego0": 0.012726740277280022, "complete-iteration": 0.200101642981854, "set_robot_commands": 0.0023304862245532693, "deviation-center-line": 3.8902278242735218, "driven_lanedir_consec": 12.199185269549552, "sim_compute_sim_state": 0.010146874373004796, "sim_compute_performance-ego0": 0.002151507521350616}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.100779570200606, "get_ui_image": 0.03511630985098825, "step_physics": 0.16574414850960317, "survival_time": 7.049999999999983, "driven_lanedir": 0.952472791431456, "get_state_dump": 0.004466008132612201, "get_robot_state": 0.0034986415379483935, "sim_render-ego0": 0.0035575493960313394, "get_duckie_state": 1.2558950504786532e-06, "in-drivable-lane": 1.4499999999999948, "deviation-heading": 1.8102028875867675, "agent_compute-ego0": 0.011767138897533148, "complete-iteration": 0.23764993439257984, "set_robot_commands": 0.002060002004596549, "deviation-center-line": 0.34305762862412664, "driven_lanedir_consec": 0.952472791431456, "sim_compute_sim_state": 0.00951951154520814, "sim_compute_performance-ego0": 0.0018419044118531992}, "LF-norm-techtrack-000-ego0": {"driven_any": 10.511193948661155, "get_ui_image": 0.03468181271040866, "step_physics": 0.1634624891733746, "survival_time": 59.99999999999873, "driven_lanedir": 0.8684659296156699, "get_state_dump": 0.0050531414327375295, "get_robot_state": 0.004059072537386447, "sim_render-ego0": 0.004210427043638459, "get_duckie_state": 1.4587008486580197e-06, "in-drivable-lane": 53.59999999999883, "deviation-heading": 5.47884898160915, "agent_compute-ego0": 0.013071798265824011, "complete-iteration": 0.2427350901048646, "set_robot_commands": 0.0024978245029243003, "deviation-center-line": 0.5152359418061916, "driven_lanedir_consec": 0.8507794061174484, "sim_compute_sim_state": 0.013288538620732012, "sim_compute_performance-ego0": 0.0023086209976107353}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.98150989786976, "get_ui_image": 0.02730249683624699, "step_physics": 0.11770608145231808, "survival_time": 59.99999999999873, "driven_lanedir": 1.8322013662410317, "get_state_dump": 0.004933746530054808, "get_robot_state": 0.0038548724835957217, "sim_render-ego0": 0.003940428623450388, "get_duckie_state": 1.319342112164017e-06, "in-drivable-lane": 51.6499999999987, "deviation-heading": 2.7855842875573833, "agent_compute-ego0": 0.012441647638389211, "complete-iteration": 0.18106398117929373, "set_robot_commands": 0.0022593767418650963, "deviation-center-line": 0.5236493823016465, "driven_lanedir_consec": 1.8322013662410317, "sim_compute_sim_state": 0.006467081128707238, "sim_compute_performance-ego0": 0.0020712541600845934}}
set_robot_commands_max0.0024978245029243003
set_robot_commands_mean0.0022869223684848037
set_robot_commands_median0.002294931483209183
set_robot_commands_min0.002060002004596549
sim_compute_performance-ego0_max0.0023086209976107353
sim_compute_performance-ego0_mean0.002093321772724786
sim_compute_performance-ego0_median0.0021113808407176044
sim_compute_performance-ego0_min0.0018419044118531992
sim_compute_sim_state_max0.013288538620732012
sim_compute_sim_state_mean0.009855501416913048
sim_compute_sim_state_median0.009833192959106467
sim_compute_sim_state_min0.006467081128707238
sim_render-ego0_max0.004210427043638459
sim_render-ego0_mean0.003924462660168339
sim_render-ego0_median0.003964937100501779
sim_render-ego0_min0.0035575493960313394
simulation-passed1
step_physics_max0.16574414850960317
step_physics_mean0.14403903987731156
step_physics_median0.1463529647736625
step_physics_min0.11770608145231808
survival_time_max59.99999999999873
survival_time_mean46.76249999999904
survival_time_min7.049999999999983
No reset possible
54171LFv-simsuccessyes0:26:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54170LFv-simsuccessyes0:20:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54169LFv-simsuccessyes0:24:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54165LFv-simsuccessyes0:28:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54161LFv-simsuccessyes0:19:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49272LFv-simerrorno0:04:21
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139829379669344
- M:video_aido:cmdline(in:/;out:/) 139829378719072
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49263LFv-simerrorno0:08:11
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140185310418304
- M:video_aido:cmdline(in:/;out:/) 140185310417392
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49251LFv-simerrorno0:04:47
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140007659590560
- M:video_aido:cmdline(in:/;out:/) 140007659145872
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43244LFv-simsuccessno0:12:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43243LFv-simsuccessno0:08:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43242LFv-simsuccessno0:11:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43241LFv-simsuccessno0:12:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible