Duckietown Challenges Home Challenges Submissions

Submission 11003

Submission11003
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 57148
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

57148

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
57148LFv-simsuccessyes0:24:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.899984276601852
survival_time_median56.199999999998944
deviation-center-line_median0.6377443223208161
in-drivable-lane_median20.499999999999552


other stats
agent_compute-ego0_max0.013301128427452765
agent_compute-ego0_mean0.012789154544187446
agent_compute-ego0_median0.012894448285595163
agent_compute-ego0_min0.012066593178106684
complete-iteration_max0.23895910680294036
complete-iteration_mean0.1858435607640029
complete-iteration_median0.1769294407047905
complete-iteration_min0.1505562548434903
deviation-center-line_max3.266032124079493
deviation-center-line_mean1.1714515367762628
deviation-center-line_min0.14428537838392566
deviation-heading_max10.967165439583836
deviation-heading_mean4.310307159007528
deviation-heading_median2.270738830356514
deviation-heading_min1.7325855357332487
driven_any_max8.535375240815826
driven_any_mean5.695626220723407
driven_any_median6.994741754874882
driven_any_min0.25764613232803946
driven_lanedir_consec_max7.573223521619112
driven_lanedir_consec_mean2.865413315545683
driven_lanedir_consec_min0.08846118735991526
driven_lanedir_max7.573223521619112
driven_lanedir_mean2.865413315545683
driven_lanedir_median1.899984276601852
driven_lanedir_min0.08846118735991526
get_duckie_state_max1.4991982592631142e-06
get_duckie_state_mean1.3784146791451624e-06
get_duckie_state_median1.363500065084549e-06
get_duckie_state_min1.2874603271484376e-06
get_robot_state_max0.003768739300301463
get_robot_state_mean0.00367481942900294
get_robot_state_median0.003669263044066671
get_robot_state_min0.003592012327576954
get_state_dump_max0.004798296093940735
get_state_dump_mean0.004679358341157567
get_state_dump_median0.0046962607661585574
get_state_dump_min0.004526615738372422
get_ui_image_max0.035384070873260495
get_ui_image_mean0.030599133042299748
get_ui_image_median0.03075218638120577
get_ui_image_min0.025508088533526953
in-drivable-lane_max46.799999999998676
in-drivable-lane_mean21.949999999999445
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.535375240815826, "get_ui_image": 0.02876269112617944, "step_physics": 0.0953805416847248, "survival_time": 59.99999999999873, "driven_lanedir": 2.035591220153016, "get_state_dump": 0.00468621901131788, "get_robot_state": 0.003737642604246624, "sim_render-ego0": 0.004011607785506808, "get_duckie_state": 1.4991982592631142e-06, "in-drivable-lane": 46.799999999998676, "deviation-heading": 1.7325855357332487, "agent_compute-ego0": 0.012811314354927512, "complete-iteration": 0.16325183276034314, "set_robot_commands": 0.002255810190497787, "deviation-center-line": 0.5560012830039653, "driven_lanedir_consec": 2.035591220153016, "sim_compute_sim_state": 0.009493631089756037, "sim_compute_performance-ego0": 0.002023943854212066}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.25764613232803946, "get_ui_image": 0.035384070873260495, "step_physics": 0.16460329294204712, "survival_time": 3.949999999999994, "driven_lanedir": 0.08846118735991526, "get_state_dump": 0.004798296093940735, "get_robot_state": 0.003600883483886719, "sim_render-ego0": 0.0036840349435806274, "get_duckie_state": 1.2874603271484376e-06, "in-drivable-lane": 1.649999999999994, "deviation-heading": 1.967986461393192, "agent_compute-ego0": 0.012977582216262818, "complete-iteration": 0.23895910680294036, "set_robot_commands": 0.002318769693374634, "deviation-center-line": 0.14428537838392566, "driven_lanedir_consec": 0.08846118735991526, "sim_compute_sim_state": 0.009635597467422484, "sim_compute_performance-ego0": 0.001872238516807556}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.252363800808376, "get_ui_image": 0.032741681636232096, "step_physics": 0.11571978477208926, "survival_time": 52.39999999999916, "driven_lanedir": 1.7643773330506884, "get_state_dump": 0.004706302520999236, "get_robot_state": 0.003768739300301463, "sim_render-ego0": 0.00403043744220861, "get_duckie_state": 1.3534629538129691e-06, "in-drivable-lane": 39.34999999999911, "deviation-heading": 2.5734911993198355, "agent_compute-ego0": 0.013301128427452765, "complete-iteration": 0.1906070486492379, "set_robot_commands": 0.0022425362902442196, "deviation-center-line": 0.7194873616376668, "driven_lanedir_consec": 1.7643773330506884, "sim_compute_sim_state": 0.011978634433364504, "sim_compute_performance-ego0": 0.002028155940504047}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.737119708941387, "get_ui_image": 0.025508088533526953, "step_physics": 0.0907875217069297, "survival_time": 59.99999999999873, "driven_lanedir": 7.573223521619112, "get_state_dump": 0.004526615738372422, "get_robot_state": 0.003592012327576954, "sim_render-ego0": 0.003796644353747467, "get_duckie_state": 1.373537176356129e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.967165439583836, "agent_compute-ego0": 0.012066593178106684, "complete-iteration": 0.1505562548434903, "set_robot_commands": 0.0021379944883119454, "deviation-center-line": 3.266032124079493, "driven_lanedir_consec": 7.573223521619112, "sim_compute_sim_state": 0.006133182956019011, "sim_compute_performance-ego0": 0.001921570172020041}}
set_robot_commands_max0.002318769693374634
set_robot_commands_mean0.0022387776656071466
set_robot_commands_median0.002249173240371003
set_robot_commands_min0.0021379944883119454
sim_compute_performance-ego0_max0.002028155940504047
sim_compute_performance-ego0_mean0.0019614771208859272
sim_compute_performance-ego0_median0.0019727570131160537
sim_compute_performance-ego0_min0.001872238516807556
sim_compute_sim_state_max0.011978634433364504
sim_compute_sim_state_mean0.00931026148664051
sim_compute_sim_state_median0.00956461427858926
sim_compute_sim_state_min0.006133182956019011
sim_render-ego0_max0.00403043744220861
sim_render-ego0_mean0.003880681131260878
sim_render-ego0_median0.0039041260696271378
sim_render-ego0_min0.0036840349435806274
simulation-passed1
step_physics_max0.16460329294204712
step_physics_mean0.11662278527644772
step_physics_median0.10555016322840705
step_physics_min0.0907875217069297
survival_time_max59.99999999999873
survival_time_mean44.08749999999915
survival_time_min3.949999999999994
No reset possible
50942LFv-simerrorno0:08:38
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140631180599600
- M:video_aido:cmdline(in:/;out:/) 140631180600416
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50940LFv-simhost-errorno0:08:07
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50939LFv-simerrorno0:07:24
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140062064821680
- M:video_aido:cmdline(in:/;out:/) 140062064823600
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50926LFv-simerrorno0:08:31
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139623855751952
- M:video_aido:cmdline(in:/;out:/) 139623855755024
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40473LFv-simsuccessno0:10:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40472LFv-simsuccessno0:09:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40471LFv-simsuccessno0:11:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39764LFv-simsuccessno0:14:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38637LFv-simsuccessno0:08:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38636LFv-simsuccessno0:08:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible