Duckietown Challenges Home Challenges Submissions

Submission 11609

Submission11609
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54275
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54275

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54275LFv-simsuccessyes0:37:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median9.06183097530914
survival_time_median59.99999999999873
deviation-center-line_median3.5653287304348713
in-drivable-lane_median5.599999999999994


other stats
agent_compute-ego0_max0.01307834951605626
agent_compute-ego0_mean0.012550836250645035
agent_compute-ego0_median0.01251998362592813
agent_compute-ego0_min0.012085028234667623
complete-iteration_max0.23557097508845776
complete-iteration_mean0.20964109709022644
complete-iteration_median0.2153338483727842
complete-iteration_min0.17232571652687956
deviation-center-line_max3.946518525850359
deviation-center-line_mean3.439267004213012
deviation-center-line_min2.6798920301319455
deviation-heading_max17.974946995375802
deviation-heading_mean15.593539583950909
deviation-heading_median16.513839500171006
deviation-heading_min11.37153234008582
driven_any_max12.342940439371118
driven_any_mean11.848201933098732
driven_any_median11.91853928269911
driven_any_min11.212788727625592
driven_lanedir_consec_max11.35000605591078
driven_lanedir_consec_mean8.749001720379933
driven_lanedir_consec_min5.5223388749906785
driven_lanedir_max11.35000605591078
driven_lanedir_mean9.696043940726714
driven_lanedir_median10.231713816839871
driven_lanedir_min6.970742073316334
get_duckie_state_max1.422570805863278e-06
get_duckie_state_mean1.3014259783056357e-06
get_duckie_state_median1.3004830238920369e-06
get_duckie_state_min1.1821670595751912e-06
get_robot_state_max0.003926660496428249
get_robot_state_mean0.003679246777400287
get_robot_state_median0.0036675998511461294
get_robot_state_min0.0034551269108806424
get_state_dump_max0.004961880914972386
get_state_dump_mean0.0046930874416373555
get_state_dump_median0.00471379169318003
get_state_dump_min0.004382885465216974
get_ui_image_max0.03391227257638847
get_ui_image_mean0.030211951512281943
get_ui_image_median0.03096981151812678
get_ui_image_min0.02499591043648573
in-drivable-lane_max23.04999999999904
in-drivable-lane_mean8.812499999999742
in-drivable-lane_min0.9999999999999432
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.342940439371118, "get_ui_image": 0.02905613536342395, "step_physics": 0.1345540953119232, "survival_time": 59.99999999999873, "driven_lanedir": 11.35000605591078, "get_state_dump": 0.004791000304273721, "get_robot_state": 0.0037808271371553978, "sim_render-ego0": 0.0038391745755515625, "get_duckie_state": 1.3802867447903115e-06, "in-drivable-lane": 0.9999999999999432, "deviation-heading": 17.974946995375802, "agent_compute-ego0": 0.012825246853792696, "complete-iteration": 0.20290619864451895, "set_robot_commands": 0.0022306992152052855, "deviation-center-line": 3.742513154588324, "driven_lanedir_consec": 11.35000605591078, "sim_compute_sim_state": 0.009708032719201587, "sim_compute_performance-ego0": 0.0020374318741441864}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.212788727625592, "get_ui_image": 0.03288348767282961, "step_physics": 0.15432378791154772, "survival_time": 59.99999999999873, "driven_lanedir": 6.970742073316334, "get_state_dump": 0.004382885465216974, "get_robot_state": 0.0034551269108806424, "sim_render-ego0": 0.003541028866859201, "get_duckie_state": 1.1821670595751912e-06, "in-drivable-lane": 23.04999999999904, "deviation-heading": 11.37153234008582, "agent_compute-ego0": 0.012085028234667623, "complete-iteration": 0.22776149810104945, "set_robot_commands": 0.00204461440754175, "deviation-center-line": 2.6798920301319455, "driven_lanedir_consec": 6.969432812577214, "sim_compute_sim_state": 0.013111588956910704, "sim_compute_performance-ego0": 0.0018580765847262496}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.240116758860555, "get_ui_image": 0.03391227257638847, "step_physics": 0.15759142471491347, "survival_time": 59.99999999999873, "driven_lanedir": 11.154229138041066, "get_state_dump": 0.004961880914972386, "get_robot_state": 0.003926660496428249, "sim_render-ego0": 0.004032164191723267, "get_duckie_state": 1.422570805863278e-06, "in-drivable-lane": 2.3499999999999917, "deviation-heading": 16.304623408550242, "agent_compute-ego0": 0.01307834951605626, "complete-iteration": 0.23557097508845776, "set_robot_commands": 0.0023496641306754054, "deviation-center-line": 3.388144306281419, "driven_lanedir_consec": 11.154229138041066, "sim_compute_sim_state": 0.01345014075851758, "sim_compute_performance-ego0": 0.0021799869283252115}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.596961806537662, "get_ui_image": 0.02499591043648573, "step_physics": 0.11318555580984048, "survival_time": 59.99999999999873, "driven_lanedir": 9.30919849563868, "get_state_dump": 0.004636583082086339, "get_robot_state": 0.003554372565136861, "sim_render-ego0": 0.0036448876526234647, "get_duckie_state": 1.2206793029937616e-06, "in-drivable-lane": 8.849999999999998, "deviation-heading": 16.72305559179177, "agent_compute-ego0": 0.012214720398063565, "complete-iteration": 0.17232571652687956, "set_robot_commands": 0.002151805494945313, "deviation-center-line": 3.946518525850359, "driven_lanedir_consec": 5.5223388749906785, "sim_compute_sim_state": 0.005979548683769995, "sim_compute_performance-ego0": 0.0018846770309587995}}
set_robot_commands_max0.0023496641306754054
set_robot_commands_mean0.0021941958120919387
set_robot_commands_median0.0021912523550752993
set_robot_commands_min0.00204461440754175
sim_compute_performance-ego0_max0.0021799869283252115
sim_compute_performance-ego0_mean0.0019900431045386116
sim_compute_performance-ego0_median0.001961054452551493
sim_compute_performance-ego0_min0.0018580765847262496
sim_compute_sim_state_max0.01345014075851758
sim_compute_sim_state_mean0.010562327779599966
sim_compute_sim_state_median0.011409810838056146
sim_compute_sim_state_min0.005979548683769995
sim_render-ego0_max0.004032164191723267
sim_render-ego0_mean0.0037643138216893737
sim_render-ego0_median0.0037420311140875136
sim_render-ego0_min0.003541028866859201
simulation-passed1
step_physics_max0.15759142471491347
step_physics_mean0.13991371593705623
step_physics_median0.14443894161173546
step_physics_min0.11318555580984048
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
54268LFv-simsuccessyes0:30:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54264LFv-simsuccessyes0:37:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49374LFv-simerrorno0:10:59
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139911290193088
- M:video_aido:cmdline(in:/;out:/) 139908160118640
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49359LFv-simhost-errorno0:11:19
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49356LFv-simerrorno0:09:53
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139956732157232
- M:video_aido:cmdline(in:/;out:/) 139956732199504
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43119LFv-simsuccessno0:11:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43118LFv-simsuccessno0:07:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible