Duckietown Challenges Home Challenges Submissions

Submission 11688

Submission11688
Competingyes
Challengeaido5-LF-sim-validation
UserBhavya Patwa 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54028
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54028

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54028LFv-simsuccessyes0:27:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.495623327747763
survival_time_median49.64999999999932
deviation-center-line_median0.9843450763659172
in-drivable-lane_median13.34999999999978


other stats
agent_compute-ego0_max0.013368362089881751
agent_compute-ego0_mean0.01297844363236472
agent_compute-ego0_median0.01293515576112664
agent_compute-ego0_min0.01267510091732384
complete-iteration_max0.2752425155615413
complete-iteration_mean0.2471241829069063
complete-iteration_median0.24720948153062933
complete-iteration_min0.2188352530048253
deviation-center-line_max3.861841229258786
deviation-center-line_mean1.4975535335948154
deviation-center-line_min0.15968275238864166
deviation-heading_max17.6794517895913
deviation-heading_mean7.726744089655876
deviation-heading_median6.050468720054242
deviation-heading_min1.1265871289237184
driven_any_max7.920619797734549
driven_any_mean5.461815632524498
driven_any_median6.494675973874732
driven_any_min0.93729078461398
driven_lanedir_consec_max5.963858745737858
driven_lanedir_consec_mean2.3644361307645134
driven_lanedir_consec_min0.5026391218246693
driven_lanedir_max5.963858745737858
driven_lanedir_mean2.3644361307645134
driven_lanedir_median1.495623327747763
driven_lanedir_min0.5026391218246693
get_duckie_state_max2.1779269203357556e-06
get_duckie_state_mean2.1004463786287434e-06
get_duckie_state_median2.110717533929979e-06
get_duckie_state_min2.0024235263192582e-06
get_robot_state_max0.003992467853589767
get_robot_state_mean0.003763573463702489
get_robot_state_median0.003769997256086033
get_robot_state_min0.003521831489048121
get_state_dump_max0.00494837306506146
get_state_dump_mean0.004815127311906661
get_state_dump_median0.004826413602455768
get_state_dump_min0.004659308977653644
get_ui_image_max0.03461518609450639
get_ui_image_mean0.03140978835663591
get_ui_image_median0.03178606338269073
get_ui_image_min0.027451840566655776
in-drivable-lane_max55.54999999999873
in-drivable-lane_mean21.38749999999957
in-drivable-lane_min3.299999999999989
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.920619797734549, "get_ui_image": 0.02912916092948056, "step_physics": 0.14968709008679798, "survival_time": 59.99999999999873, "driven_lanedir": 5.963858745737858, "get_state_dump": 0.004841671895226472, "get_robot_state": 0.0037607449873798, "sim_render-ego0": 0.003796259032796563, "get_duckie_state": 2.1779269203357556e-06, "in-drivable-lane": 10.149999999999777, "deviation-heading": 17.6794517895913, "agent_compute-ego0": 0.013182659133288584, "complete-iteration": 0.2188352530048253, "set_robot_commands": 0.0022759902090156805, "deviation-center-line": 3.861841229258786, "driven_lanedir_consec": 5.963858745737858, "sim_compute_sim_state": 0.010028965566477909, "sim_compute_performance-ego0": 0.0020435821206047575}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.93729078461398, "get_ui_image": 0.03461518609450639, "step_physics": 0.1811184546698822, "survival_time": 8.09999999999998, "driven_lanedir": 0.5136169066087972, "get_state_dump": 0.004659308977653644, "get_robot_state": 0.003521831489048121, "sim_render-ego0": 0.003694076479578311, "get_duckie_state": 2.0024235263192582e-06, "in-drivable-lane": 3.299999999999989, "deviation-heading": 1.8697818667080792, "agent_compute-ego0": 0.012687652388964694, "complete-iteration": 0.2549227307910568, "set_robot_commands": 0.0021756865495553045, "deviation-center-line": 0.15968275238864166, "driven_lanedir_consec": 0.5136169066087972, "sim_compute_sim_state": 0.010451833163302369, "sim_compute_performance-ego0": 0.001912822021297151}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.072631104578288, "get_ui_image": 0.034442965835900896, "step_physics": 0.1966869046454811, "survival_time": 39.299999999999905, "driven_lanedir": 2.477629748886729, "get_state_dump": 0.00494837306506146, "get_robot_state": 0.003992467853589767, "sim_render-ego0": 0.004070122935720441, "get_duckie_state": 2.097901728434969e-06, "in-drivable-lane": 16.549999999999788, "deviation-heading": 10.231155573400406, "agent_compute-ego0": 0.013368362089881751, "complete-iteration": 0.2752425155615413, "set_robot_commands": 0.002409940450588357, "deviation-center-line": 1.7645153168290388, "driven_lanedir_consec": 2.477629748886729, "sim_compute_sim_state": 0.013042830148524826, "sim_compute_performance-ego0": 0.00218670977116237}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.916720843171176, "get_ui_image": 0.027451840566655776, "step_physics": 0.17590861157711896, "survival_time": 59.99999999999873, "driven_lanedir": 0.5026391218246693, "get_state_dump": 0.004811155309685065, "get_robot_state": 0.003779249524792267, "sim_render-ego0": 0.003910806157209792, "get_duckie_state": 2.1235333394249907e-06, "in-drivable-lane": 55.54999999999873, "deviation-heading": 1.1265871289237184, "agent_compute-ego0": 0.01267510091732384, "complete-iteration": 0.23949623227020184, "set_robot_commands": 0.0022930233405094955, "deviation-center-line": 0.2041748359027956, "driven_lanedir_consec": 0.5026391218246693, "sim_compute_sim_state": 0.006540079696490108, "sim_compute_performance-ego0": 0.0020326861334680818}}
set_robot_commands_max0.002409940450588357
set_robot_commands_mean0.002288660137417209
set_robot_commands_median0.002284506774762588
set_robot_commands_min0.0021756865495553045
sim_compute_performance-ego0_max0.00218670977116237
sim_compute_performance-ego0_mean0.00204395001163309
sim_compute_performance-ego0_median0.0020381341270364196
sim_compute_performance-ego0_min0.001912822021297151
sim_compute_sim_state_max0.013042830148524826
sim_compute_sim_state_mean0.010015927143698802
sim_compute_sim_state_median0.010240399364890138
sim_compute_sim_state_min0.006540079696490108
sim_render-ego0_max0.004070122935720441
sim_render-ego0_mean0.0038678161513262766
sim_render-ego0_median0.0038535325950031776
sim_render-ego0_min0.003694076479578311
simulation-passed1
step_physics_max0.1966869046454811
step_physics_mean0.17585026524482006
step_physics_median0.17851353312350057
step_physics_min0.14968709008679798
survival_time_max59.99999999999873
survival_time_mean41.84999999999933
survival_time_min8.09999999999998
No reset possible
48979LFv-simerrorno0:03:18
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140057272314608
- M:video_aido:cmdline(in:/;out:/) 140057272315616
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43549LFv-simsuccessno0:08:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43548LFv-simsuccessno0:10:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43547LFv-simsuccessno0:08:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43546LFv-simsuccessno0:06:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43545LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43544LFv-simhost-errorno0:00:14
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43543LFv-simhost-errorno0:00:12
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43542LFv-simsuccessno0:06:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43529LFv-simsuccessno0:09:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43528LFv-simsuccessno0:08:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43527LFv-simsuccessno0:09:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible