Duckietown Challenges Home Challenges Submissions

Submission 10978

Submission10978
Competingyes
Challengeaido5-LF-sim-validation
UserPhilippe Reddy 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57245
Next
User labelsim-exercise-1
Admin priority50
Blessingn/a
User priority50

57245

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57245LFv-simsuccessyes0:24:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.617251978226594
survival_time_median35.10000000000008
deviation-center-line_median1.8876731189062808
in-drivable-lane_median11.699999999999925


other stats
agent_compute-ego0_max0.04393496537449384
agent_compute-ego0_mean0.03644428131569861
agent_compute-ego0_median0.034523479628798634
agent_compute-ego0_min0.03279520063070332
complete-iteration_max0.23257929022167248
complete-iteration_mean0.1995473562360667
complete-iteration_median0.19312950962751096
complete-iteration_min0.17935111546757246
deviation-center-line_max3.218194136012909
deviation-center-line_mean1.7738257505182806
deviation-center-line_min0.10176262824765164
deviation-heading_max9.04686578393731
deviation-heading_mean4.570592852914481
deviation-heading_median4.313602159635396
deviation-heading_min0.6083013084498217
driven_any_max12.506589783206987
driven_any_mean7.155841011217437
driven_any_median7.134915952372982
driven_any_min1.8469423569167949
driven_lanedir_consec_max7.3300229152234815
driven_lanedir_consec_mean4.317384610060815
driven_lanedir_consec_min0.7050115685665883
driven_lanedir_max8.632565237314804
driven_lanedir_mean4.646544789639323
driven_lanedir_median4.6243011763379505
driven_lanedir_min0.7050115685665883
get_duckie_state_max1.8059065895811108e-06
get_duckie_state_mean1.665328478835354e-06
get_duckie_state_median1.663350195117977e-06
get_duckie_state_min1.5287069355243512e-06
get_robot_state_max0.003613133513857979
get_robot_state_mean0.003558621670046832
get_robot_state_median0.003565069144999628
get_robot_state_min0.003491214876330093
get_state_dump_max0.004610719947877787
get_state_dump_mean0.0045491559002961745
get_state_dump_median0.004582097088635213
get_state_dump_min0.004421709476036482
get_ui_image_max0.03506884527246124
get_ui_image_mean0.029483757519251422
get_ui_image_median0.02847395589695262
get_ui_image_min0.025918273010639228
in-drivable-lane_max16.699999999999633
in-drivable-lane_mean11.524999999999874
in-drivable-lane_min6.000000000000011
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.146380130956326, "get_ui_image": 0.026922387981295436, "step_physics": 0.09881113229018726, "survival_time": 39.89999999999987, "driven_lanedir": 6.15340638184437, "get_state_dump": 0.004421709476036482, "get_robot_state": 0.003491214876330093, "sim_render-ego0": 0.003692012256913549, "get_duckie_state": 1.637300054480943e-06, "in-drivable-lane": 9.349999999999667, "deviation-heading": 4.583507563187711, "agent_compute-ego0": 0.03400357434985336, "complete-iteration": 0.18434545901302105, "set_robot_commands": 0.002132872616095895, "deviation-center-line": 2.395368592656406, "driven_lanedir_consec": 6.15340638184437, "sim_compute_sim_state": 0.008932001152086318, "sim_compute_performance-ego0": 0.0018548828191840751}, "LF-norm-zigzag-000-ego0": {"driven_any": 12.506589783206987, "get_ui_image": 0.03506884527246124, "step_physics": 0.1324647185605928, "survival_time": 59.99999999999873, "driven_lanedir": 8.632565237314804, "get_state_dump": 0.0045750772426964145, "get_robot_state": 0.003613133513857979, "sim_render-ego0": 0.003916269734340544, "get_duckie_state": 1.8059065895811108e-06, "in-drivable-lane": 16.699999999999633, "deviation-heading": 9.04686578393731, "agent_compute-ego0": 0.03504338490774391, "complete-iteration": 0.23257929022167248, "set_robot_commands": 0.0022292291989036645, "deviation-center-line": 3.218194136012909, "driven_lanedir_consec": 7.3300229152234815, "sim_compute_sim_state": 0.013588269088389376, "sim_compute_performance-ego0": 0.001990581928542214}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.123451773789637, "get_ui_image": 0.030025523812609804, "step_physics": 0.11095557063572488, "survival_time": 30.300000000000296, "driven_lanedir": 3.0951959708315315, "get_state_dump": 0.004610719947877787, "get_robot_state": 0.0035787779573946453, "sim_render-ego0": 0.0037937451037386495, "get_duckie_state": 1.5287069355243512e-06, "in-drivable-lane": 14.05000000000018, "deviation-heading": 4.043696756083081, "agent_compute-ego0": 0.03279520063070332, "complete-iteration": 0.20191356024200088, "set_robot_commands": 0.0021568578000516437, "deviation-center-line": 1.3799776451561552, "driven_lanedir_consec": 3.08109757460882, "sim_compute_sim_state": 0.01195987720269544, "sim_compute_performance-ego0": 0.001954978930302075}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.8469423569167949, "get_ui_image": 0.025918273010639228, "step_physics": 0.08825855182878899, "survival_time": 9.850000000000003, "driven_lanedir": 0.7050115685665883, "get_state_dump": 0.004589116934574012, "get_robot_state": 0.0035513603326046105, "sim_render-ego0": 0.003820990071152196, "get_duckie_state": 1.689400335755011e-06, "in-drivable-lane": 6.000000000000011, "deviation-heading": 0.6083013084498217, "agent_compute-ego0": 0.04393496537449384, "complete-iteration": 0.17935111546757246, "set_robot_commands": 0.0022290848722361554, "deviation-center-line": 0.10176262824765164, "driven_lanedir_consec": 0.7050115685665883, "sim_compute_sim_state": 0.005052009014168171, "sim_compute_performance-ego0": 0.0019112536401459663}}
set_robot_commands_max0.0022292291989036645
set_robot_commands_mean0.0021870111218218395
set_robot_commands_median0.0021929713361438993
set_robot_commands_min0.002132872616095895
sim_compute_performance-ego0_max0.001990581928542214
sim_compute_performance-ego0_mean0.0019279243295435829
sim_compute_performance-ego0_median0.0019331162852240207
sim_compute_performance-ego0_min0.0018548828191840751
sim_compute_sim_state_max0.013588269088389376
sim_compute_sim_state_mean0.009883039114334826
sim_compute_sim_state_median0.01044593917739088
sim_compute_sim_state_min0.005052009014168171
sim_render-ego0_max0.003916269734340544
sim_render-ego0_mean0.003805754291536234
sim_render-ego0_median0.003807367587445423
sim_render-ego0_min0.003692012256913549
simulation-passed1
step_physics_max0.1324647185605928
step_physics_mean0.1076224933288235
step_physics_median0.10488335146295608
step_physics_min0.08825855182878899
survival_time_max59.99999999999873
survival_time_mean35.01249999999972
survival_time_min9.850000000000003
No reset possible
51026LFv-simerrorno0:03:47
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140375821673376
- M:video_aido:cmdline(in:/;out:/) 140375821675008
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51016LFv-simerrorno0:08:15
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140282829376528
- M:video_aido:cmdline(in:/;out:/) 140282829210000
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40508LFv-simsuccessno0:09:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40507LFv-simsuccessno0:09:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40506LFv-simsuccessno0:11:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40504LFv-simsuccessno0:10:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39796LFv-simsuccessno0:09:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38587LFv-simsuccessno0:11:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38586LFv-simsuccessno0:08:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible