Duckietown Challenges Home Challenges Submissions

Submission 11678

Submission11678
Competingyes
Challengeaido5-LF-sim-validation
UserHimanshu Arora 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54059
Next
User labelreal-exercise-2
Admin priority50
Blessingn/a
User priority50

54059

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54059LFv-simsuccessyes0:29:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.597298911455903
survival_time_median55.274999999999
deviation-center-line_median3.66960477884479
in-drivable-lane_median7.500000000000069


other stats
agent_compute-ego0_max0.013091211498019253
agent_compute-ego0_mean0.01261304880166351
agent_compute-ego0_median0.01251759269056868
agent_compute-ego0_min0.012325798327497427
complete-iteration_max0.2220669447829781
complete-iteration_mean0.1998361860432484
complete-iteration_median0.2000653639814816
complete-iteration_min0.17714707142705227
deviation-center-line_max5.211252447654935
deviation-center-line_mean3.2911592991274836
deviation-center-line_min0.6141751911654193
deviation-heading_max17.77155664202554
deviation-heading_mean10.237414244303094
deviation-heading_median10.44573426810929
deviation-heading_min2.286631798968259
driven_any_max8.33794915898882
driven_any_mean6.69926941576376
driven_any_median7.621651868874784
driven_any_min3.2158247663166515
driven_lanedir_consec_max7.787545868746939
driven_lanedir_consec_mean5.215235392634572
driven_lanedir_consec_min1.8787978788795408
driven_lanedir_max7.787545868746939
driven_lanedir_mean5.215235392634572
driven_lanedir_median5.597298911455903
driven_lanedir_min1.8787978788795408
get_duckie_state_max1.3931903315027192e-06
get_duckie_state_mean1.3499916000822831e-06
get_duckie_state_median1.3556512515636296e-06
get_duckie_state_min1.295473565699154e-06
get_robot_state_max0.004011891104958274
get_robot_state_mean0.0038355827385945984
get_robot_state_median0.003813820020245275
get_robot_state_min0.00370279980892957
get_state_dump_max0.005075487930312929
get_state_dump_mean0.004864089727486155
get_state_dump_median0.004822919708207486
get_state_dump_min0.00473503156321672
get_ui_image_max0.03731510879395904
get_ui_image_mean0.03181529734323692
get_ui_image_median0.03142375710541421
get_ui_image_min0.0270985663681602
in-drivable-lane_max19.349999999999145
in-drivable-lane_mean8.587499999999821
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.2158247663166515, "get_ui_image": 0.028051367439175048, "step_physics": 0.11346718631839356, "survival_time": 24.050000000000207, "driven_lanedir": 1.8787978788795408, "get_state_dump": 0.00473503156321672, "get_robot_state": 0.00370279980892957, "sim_render-ego0": 0.0036502834177610786, "get_duckie_state": 1.295473565699154e-06, "in-drivable-lane": 9.80000000000014, "deviation-heading": 2.286631798968259, "agent_compute-ego0": 0.012325798327497427, "complete-iteration": 0.1802108624169441, "set_robot_commands": 0.002142446664359065, "deviation-center-line": 0.6141751911654193, "driven_lanedir_consec": 1.8787978788795408, "sim_compute_sim_state": 0.010081549897728123, "sim_compute_performance-ego0": 0.001974406578728767}, "LF-norm-zigzag-000-ego0": {"driven_any": 8.33794915898882, "get_ui_image": 0.03731510879395904, "step_physics": 0.1417622355795423, "survival_time": 59.99999999999873, "driven_lanedir": 7.787545868746939, "get_state_dump": 0.004805736795849447, "get_robot_state": 0.003826928277694613, "sim_render-ego0": 0.003878345298925903, "get_duckie_state": 1.3931903315027192e-06, "in-drivable-lane": 0.0, "deviation-heading": 17.77155664202554, "agent_compute-ego0": 0.01254433934436452, "complete-iteration": 0.2220669447829781, "set_robot_commands": 0.0022697798119893577, "deviation-center-line": 4.9906080444304965, "driven_lanedir_consec": 7.787545868746939, "sim_compute_sim_state": 0.013444086792665556, "sim_compute_performance-ego0": 0.0021310942456883057}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.905454077189991, "get_ui_image": 0.034796146771653366, "step_physics": 0.1393148788350373, "survival_time": 50.549999999999265, "driven_lanedir": 3.909899743009927, "get_state_dump": 0.005075487930312929, "get_robot_state": 0.004011891104958274, "sim_render-ego0": 0.004145996843873277, "get_duckie_state": 1.3734983361285666e-06, "in-drivable-lane": 19.349999999999145, "deviation-heading": 9.594609833784508, "agent_compute-ego0": 0.013091211498019253, "complete-iteration": 0.2199198655460192, "set_robot_commands": 0.0024101387841899405, "deviation-center-line": 2.3486015132590836, "driven_lanedir_consec": 3.909899743009927, "sim_compute_sim_state": 0.014785452322526413, "sim_compute_performance-ego0": 0.00219519878093433}, "LF-norm-small_loop-000-ego0": {"driven_any": 8.337849660559577, "get_ui_image": 0.0270985663681602, "step_physics": 0.11453661533517702, "survival_time": 59.99999999999873, "driven_lanedir": 7.284698079901879, "get_state_dump": 0.004840102620565524, "get_robot_state": 0.0038007117627959367, "sim_render-ego0": 0.003757826791615609, "get_duckie_state": 1.3378041669986925e-06, "in-drivable-lane": 5.199999999999999, "deviation-heading": 11.29685870243407, "agent_compute-ego0": 0.012490846036772843, "complete-iteration": 0.17714707142705227, "set_robot_commands": 0.0022277085608387866, "deviation-center-line": 5.211252447654935, "driven_lanedir_consec": 7.284698079901879, "sim_compute_sim_state": 0.006324546124714797, "sim_compute_performance-ego0": 0.00198575062716037}}
set_robot_commands_max0.0024101387841899405
set_robot_commands_mean0.0022625184553442873
set_robot_commands_median0.002248744186414072
set_robot_commands_min0.002142446664359065
sim_compute_performance-ego0_max0.00219519878093433
sim_compute_performance-ego0_mean0.002071612558127943
sim_compute_performance-ego0_median0.002058422436424338
sim_compute_performance-ego0_min0.001974406578728767
sim_compute_sim_state_max0.014785452322526413
sim_compute_sim_state_mean0.01115890878440872
sim_compute_sim_state_median0.01176281834519684
sim_compute_sim_state_min0.006324546124714797
sim_render-ego0_max0.004145996843873277
sim_render-ego0_mean0.003858113088043967
sim_render-ego0_median0.003818086045270756
sim_render-ego0_min0.0036502834177610786
simulation-passed1
step_physics_max0.1417622355795423
step_physics_mean0.12727022901703752
step_physics_median0.12692574708510715
step_physics_min0.11346718631839356
survival_time_max59.99999999999873
survival_time_mean48.64999999999923
survival_time_min24.050000000000207
No reset possible
54056LFv-simsuccessyes0:30:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49056LFv-simerrorno0:09:51
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139768450521792
- M:video_aido:cmdline(in:/;out:/) 139768450416304
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49055LFv-simerrorno0:04:52
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140436692020624
- M:video_aido:cmdline(in:/;out:/) 140436555747872
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43462LFv-simsuccessno0:09:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43461LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible