Duckietown Challenges Home Challenges Submissions

Submission 2499

Submission2499
Competingyes
Challengeaido2-LFVI-sim-testing
UserLiam Paull 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22938
Next
User labelchallenge-aido_LF-template-pytorch
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22938step1-simulationsuccessyes0:18:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.23356919140029575
survival_time_median1.5500000000000007
deviation-center-line_median0.1056904917896022
in-drivable-lane_median0


other stats
agent_compute-ego_max0.19315692901611328
agent_compute-ego_mean0.1737490270390965
agent_compute-ego_median0.17483253479003907
agent_compute-ego_min0.1381561448497157
deviation-center-line_max0.14784202605609845
deviation-center-line_mean0.1016079030861103
deviation-center-line_min0.05974673579480917
deviation-heading_max1.0307177433761603
deviation-heading_mean0.6222654881876274
deviation-heading_median0.6903017356270803
deviation-heading_min0.38203308049776047
driven_any_max0.4279219069518986
driven_any_mean0.2793720663780102
driven_any_median0.2876357061084567
driven_any_min0.16104946337865889
driven_lanedir_consec_max0.34351695401031046
driven_lanedir_consec_mean0.2327139818594781
driven_lanedir_consec_min0.12786081395086368
driven_lanedir_max0.34351695401031046
driven_lanedir_mean0.2327139818594781
driven_lanedir_median0.23356919140029575
driven_lanedir_min0.12786081395086368
in-drivable-lane_max0.05000000000000005
in-drivable-lane_mean0.013333333333333345
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 0.29308239412902776, "sim_physics": 0.15178110522608604, "survival_time": 1.5500000000000007, "driven_lanedir": 0.23356919140029575, "sim_render-ego": 0.04900947693855532, "in-drivable-lane": 0, "agent_compute-ego": 0.1381561448497157, "deviation-heading": 0.7439854412495074, "set_robot_commands": 0.07121364531978484, "deviation-center-line": 0.1056904917896022, "driven_lanedir_consec": 0.23356919140029575, "sim_compute_sim_state": 0.0302642083937122, "sim_compute_performance-ego": 0.05538477436188729, "sim_compute_robot_state-ego": 0.05721196051566831, "sim_compute_robot_state-npc0": 0.053957085455617594, "sim_compute_robot_state-npc1": 0.05319519196787188, "sim_compute_robot_state-npc2": 0.053082681471301664, "sim_compute_robot_state-npc3": 0.05312667354460685}, "udem1-1-0": {"driven_any": 0.2284815620051714, "sim_physics": 0.2225224494934082, "survival_time": 1.2500000000000004, "driven_lanedir": 0.15821868998250865, "sim_render-ego": 0.06909192085266114, "in-drivable-lane": 0, "agent_compute-ego": 0.1816263866424561, "deviation-heading": 0.8174248432739096, "set_robot_commands": 0.09649744987487792, "deviation-center-line": 0.08450918276400904, "driven_lanedir_consec": 0.15821868998250865, "sim_compute_sim_state": 0.039737672805786134, "sim_compute_performance-ego": 0.06993693351745606, "sim_compute_robot_state-ego": 0.07385391235351563, "sim_compute_robot_state-npc0": 0.07545166969299316, "sim_compute_robot_state-npc1": 0.07532422065734863, "sim_compute_robot_state-npc2": 0.07405117034912109, "sim_compute_robot_state-npc3": 0.07711867332458495}, "udem1-2-0": {"driven_any": 0.26355228945583303, "sim_physics": 0.24589232740731076, "survival_time": 1.4500000000000006, "driven_lanedir": 0.2301646997373283, "sim_render-ego": 0.06320523393565211, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.17011315247108197, "deviation-heading": 0.38203308049776047, "set_robot_commands": 0.09684808500881852, "deviation-center-line": 0.12401551369024284, "driven_lanedir_consec": 0.2301646997373283, "sim_compute_sim_state": 0.03904153560769969, "sim_compute_performance-ego": 0.06739442101840315, "sim_compute_robot_state-ego": 0.07309449130091174, "sim_compute_robot_state-npc0": 0.0728066954119452, "sim_compute_robot_state-npc1": 0.07532511908432533, "sim_compute_robot_state-npc2": 0.07289402238253889, "sim_compute_robot_state-npc3": 0.07165672039163523}, "udem1-3-0": {"driven_any": 0.2876357061084567, "sim_physics": 0.22014321050336283, "survival_time": 1.5500000000000007, "driven_lanedir": 0.25578312034093775, "sim_render-ego": 0.06620401720846852, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.18328944329292543, "deviation-heading": 0.390587270603202, "set_robot_commands": 0.0965855352340206, "deviation-center-line": 0.1211910163215702, "driven_lanedir_consec": 0.25578312034093775, "sim_compute_sim_state": 0.037820285366427515, "sim_compute_performance-ego": 0.07151488334901872, "sim_compute_robot_state-ego": 0.07867107852812737, "sim_compute_robot_state-npc0": 0.06912718280669182, "sim_compute_robot_state-npc1": 0.06860716881290559, "sim_compute_robot_state-npc2": 0.07100420613442698, "sim_compute_robot_state-npc3": 0.06763729741496424}, "udem1-4-0": {"driven_any": 0.2322239410415205, "sim_physics": 0.20612090110778808, "survival_time": 1.2500000000000004, "driven_lanedir": 0.18608610279804896, "sim_render-ego": 0.06591239929199219, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.19315692901611328, "deviation-heading": 0.5694330133359564, "set_robot_commands": 0.10228744506835938, "deviation-center-line": 0.07334749754760075, "driven_lanedir_consec": 0.18608610279804896, "sim_compute_sim_state": 0.03869771957397461, "sim_compute_performance-ego": 0.07424585342407226, "sim_compute_robot_state-ego": 0.0864449405670166, "sim_compute_robot_state-npc0": 0.06945788383483886, "sim_compute_robot_state-npc1": 0.06712491989135742, "sim_compute_robot_state-npc2": 0.0689844799041748, "sim_compute_robot_state-npc3": 0.0685588264465332}, "udem1-5-0": {"driven_any": 0.4279219069518986, "sim_physics": 0.1786985009215599, "survival_time": 2.1500000000000004, "driven_lanedir": 0.34351695401031046, "sim_render-ego": 0.05645311710446379, "in-drivable-lane": 0, "agent_compute-ego": 0.165311430775842, "deviation-heading": 1.0307177433761603, "set_robot_commands": 0.0973985749621724, "deviation-center-line": 0.14784202605609845, "driven_lanedir_consec": 0.34351695401031046, "sim_compute_sim_state": 0.03771707623503929, "sim_compute_performance-ego": 0.06643812600956407, "sim_compute_robot_state-ego": 0.0688355800717376, "sim_compute_robot_state-npc0": 0.06621160063632699, "sim_compute_robot_state-npc1": 0.06224532460057458, "sim_compute_robot_state-npc2": 0.06674935651379962, "sim_compute_robot_state-npc3": 0.06371896765952886}, "udem1-6-0": {"driven_any": 0.32791765141157314, "sim_physics": 0.20442749559879303, "survival_time": 1.6000000000000008, "driven_lanedir": 0.2976733937831195, "sim_render-ego": 0.05369427055120468, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.15281646698713303, "deviation-heading": 0.3942546474000477, "set_robot_commands": 0.08179318904876709, "deviation-center-line": 0.1297403120666079, "driven_lanedir_consec": 0.2976733937831195, "sim_compute_sim_state": 0.0409713014960289, "sim_compute_performance-ego": 0.06084734946489334, "sim_compute_robot_state-ego": 0.06588172167539597, "sim_compute_robot_state-npc0": 0.0634879544377327, "sim_compute_robot_state-npc1": 0.060547031462192535, "sim_compute_robot_state-npc2": 0.062170274555683136, "sim_compute_robot_state-npc3": 0.06109467148780823}, "udem1-7-0": {"driven_any": 0.29971256233919347, "sim_physics": 0.20614001967690207, "survival_time": 1.6500000000000008, "driven_lanedir": 0.27496127583922836, "sim_render-ego": 0.06523409034266617, "in-drivable-lane": 0, "agent_compute-ego": 0.187471071879069, "deviation-heading": 0.4837785304442438, "set_robot_commands": 0.09528779983520508, "deviation-center-line": 0.12453589118076168, "driven_lanedir_consec": 0.27496127583922836, "sim_compute_sim_state": 0.040794574853145714, "sim_compute_performance-ego": 0.07122788284764145, "sim_compute_robot_state-ego": 0.07654815731626569, "sim_compute_robot_state-npc0": 0.06592676133820505, "sim_compute_robot_state-npc1": 0.06942692669955167, "sim_compute_robot_state-npc2": 0.06921252337369052, "sim_compute_robot_state-npc3": 0.06927501794063684}, "udem1-8-0": {"driven_any": 0.35146341821273697, "sim_physics": 0.201917957615208, "survival_time": 1.850000000000001, "driven_lanedir": 0.2995063827009914, "sim_render-ego": 0.06301201356423868, "in-drivable-lane": 0, "agent_compute-ego": 0.17779716929873904, "deviation-heading": 0.7416122555313746, "set_robot_commands": 0.0966817366110312, "deviation-center-line": 0.08542788847761074, "driven_lanedir_consec": 0.2995063827009914, "sim_compute_sim_state": 0.03813396273432551, "sim_compute_performance-ego": 0.07025727710208378, "sim_compute_robot_state-ego": 0.0805643893576957, "sim_compute_robot_state-npc0": 0.06974825987944731, "sim_compute_robot_state-npc1": 0.07146040168968407, "sim_compute_robot_state-npc2": 0.07025064004434121, "sim_compute_robot_state-npc3": 0.06851888991690971}, "udem1-9-0": {"driven_any": 0.24672877535585777, "sim_physics": 0.22735318770775428, "survival_time": 1.3000000000000005, "driven_lanedir": 0.19458025238182872, "sim_render-ego": 0.06296314642979549, "in-drivable-lane": 0, "agent_compute-ego": 0.18431086723621076, "deviation-heading": 0.6903017356270803, "set_robot_commands": 0.09465212088364822, "deviation-center-line": 0.07463114417353597, "driven_lanedir_consec": 0.19458025238182872, "sim_compute_sim_state": 0.04153126936692458, "sim_compute_performance-ego": 0.07225681268251859, "sim_compute_robot_state-ego": 0.07885127801161546, "sim_compute_robot_state-npc0": 0.06738584775191087, "sim_compute_robot_state-npc1": 0.07496245090778057, "sim_compute_robot_state-npc2": 0.06989763333247258, "sim_compute_robot_state-npc3": 0.07143503886002761}, "udem1-10-0": {"driven_any": 0.18146533780324453, "sim_physics": 0.20863033377605936, "survival_time": 1.1500000000000004, "driven_lanedir": 0.12786081395086368, "sim_render-ego": 0.060758352279663086, "in-drivable-lane": 0, "agent_compute-ego": 0.16582918167114258, "deviation-heading": 0.7309471410531787, "set_robot_commands": 0.09323986716892407, "deviation-center-line": 0.05974673579480917, "driven_lanedir_consec": 0.12786081395086368, "sim_compute_sim_state": 0.03956064970596977, "sim_compute_performance-ego": 0.06477921942005986, "sim_compute_robot_state-ego": 0.07108133772145146, "sim_compute_robot_state-npc0": 0.06969419769618822, "sim_compute_robot_state-npc1": 0.06980616113413936, "sim_compute_robot_state-npc2": 0.07084088740141495, "sim_compute_robot_state-npc3": 0.06603725060172703}, "udem1-11-0": {"driven_any": 0.16104946337865889, "sim_physics": 0.18767731189727785, "survival_time": 1.0000000000000002, "driven_lanedir": 0.13489778064275493, "sim_render-ego": 0.06099039316177368, "in-drivable-lane": 0, "agent_compute-ego": 0.17483253479003907, "deviation-heading": 0.4069992131871475, "set_robot_commands": 0.0911582350730896, "deviation-center-line": 0.09231785175860296, "driven_lanedir_consec": 0.13489778064275493, "sim_compute_sim_state": 0.03781082630157471, "sim_compute_performance-ego": 0.0656701922416687, "sim_compute_robot_state-ego": 0.07020400762557984, "sim_compute_robot_state-npc0": 0.06629047393798829, "sim_compute_robot_state-npc1": 0.07088220119476318, "sim_compute_robot_state-npc2": 0.06545730829238891, "sim_compute_robot_state-npc3": 0.06628654003143311}, "udem1-12-0": {"driven_any": 0.32501390100994193, "sim_physics": 0.21169635908944265, "survival_time": 1.7500000000000009, "driven_lanedir": 0.26857354569688074, "sim_render-ego": 0.06142142159598214, "in-drivable-lane": 0, "agent_compute-ego": 0.1719043459211077, "deviation-heading": 0.771831903546438, "set_robot_commands": 0.09874271665300642, "deviation-center-line": 0.11733222491398004, "driven_lanedir_consec": 0.26857354569688074, "sim_compute_sim_state": 0.03743924413408552, "sim_compute_performance-ego": 0.06603930337088448, "sim_compute_robot_state-ego": 0.07026177133832659, "sim_compute_robot_state-npc0": 0.06925198010035924, "sim_compute_robot_state-npc1": 0.06869684628077916, "sim_compute_robot_state-npc2": 0.0698432377406529, "sim_compute_robot_state-npc3": 0.07042653220040458}, "udem1-13-0": {"driven_any": 0.2190551355517376, "sim_physics": 0.22305987431452823, "survival_time": 1.3000000000000005, "driven_lanedir": 0.19209629605724743, "sim_render-ego": 0.07014789031102107, "in-drivable-lane": 0, "agent_compute-ego": 0.19122484097113976, "deviation-heading": 0.4223293331536395, "set_robot_commands": 0.1044158752147968, "deviation-center-line": 0.10577969460776926, "driven_lanedir_consec": 0.19209629605724743, "sim_compute_sim_state": 0.041993911449725814, "sim_compute_performance-ego": 0.07708454132080078, "sim_compute_robot_state-ego": 0.08367727353022648, "sim_compute_robot_state-npc0": 0.07515120506286621, "sim_compute_robot_state-npc1": 0.07596665162306565, "sim_compute_robot_state-npc2": 0.07563622181232159, "sim_compute_robot_state-npc3": 0.07381323667672965}, "udem1-14-0": {"driven_any": 0.3452769509153013, "sim_physics": 0.18660100301106772, "survival_time": 1.800000000000001, "driven_lanedir": 0.29322122856982724, "sim_render-ego": 0.06274625990125868, "in-drivable-lane": 0, "agent_compute-ego": 0.1683954397837321, "deviation-heading": 0.7577461705347642, "set_robot_commands": 0.0965947111447652, "deviation-center-line": 0.0780110751488531, "driven_lanedir_consec": 0.29322122856982724, "sim_compute_sim_state": 0.038612584273020424, "sim_compute_performance-ego": 0.06677275233798557, "sim_compute_robot_state-ego": 0.07030588388442993, "sim_compute_robot_state-npc0": 0.06657452715767755, "sim_compute_robot_state-npc1": 0.06857308414247301, "sim_compute_robot_state-npc2": 0.06744116544723511, "sim_compute_robot_state-npc3": 0.065569625960456}}
set_robot_commands_max0.1044158752147968
set_robot_commands_mean0.09422646580675116
set_robot_commands_median0.0965855352340206
set_robot_commands_min0.07121364531978484
sim_compute_performance-ego_max0.07708454132080078
sim_compute_performance-ego_mean0.06799002149792921
sim_compute_performance-ego_median0.06739442101840315
sim_compute_performance-ego_min0.05538477436188729
sim_compute_robot_state-ego_max0.0864449405670166
sim_compute_robot_state-ego_mean0.07369918558653096
sim_compute_robot_state-ego_median0.07309449130091174
sim_compute_robot_state-ego_min0.05721196051566831
sim_compute_robot_state-npc0_max0.07545166969299316
sim_compute_robot_state-npc0_mean0.06803488834671927
sim_compute_robot_state-npc0_median0.06912718280669182
sim_compute_robot_state-npc0_min0.053957085455617594
sim_compute_robot_state-npc1_max0.07596665162306565
sim_compute_robot_state-npc1_mean0.06880958000992085
sim_compute_robot_state-npc1_median0.06942692669955167
sim_compute_robot_state-npc1_min0.05319519196787188
sim_compute_robot_state-npc2_max0.07563622181232159
sim_compute_robot_state-npc2_mean0.0685010539170376
sim_compute_robot_state-npc2_median0.0698432377406529
sim_compute_robot_state-npc2_min0.053082681471301664
sim_compute_robot_state-npc3_max0.07711867332458495
sim_compute_robot_state-npc3_mean0.06761826416386575
sim_compute_robot_state-npc3_median0.06851888991690971
sim_compute_robot_state-npc3_min0.05312667354460685
sim_compute_sim_state_max0.041993911449725814
sim_compute_sim_state_mean0.038675121486496024
sim_compute_sim_state_median0.03869771957397461
sim_compute_sim_state_min0.0302642083937122
sim_physics_max0.24589232740731076
sim_physics_mean0.2055108024897699
sim_physics_median0.20614001967690207
sim_physics_min0.15178110522608604
sim_render-ego_max0.07014789031102107
sim_render-ego_mean0.06205626689795979
sim_render-ego_median0.06296314642979549
sim_render-ego_min0.04900947693855532
simulation-passed1
survival_time_max2.1500000000000004
survival_time_mean1.5066666666666673
survival_time_min1.0000000000000002
No reset possible
21731step1-simulationsuccessno0:53:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21237step1-simulationsuccessno0:14:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20527step1-simulationsuccessno0:08:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19975step1-simulationsuccessno0:18:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19795step1-simulationsuccessno0:07:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19566step1-simulationsuccessno0:11:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18823step1-simulationsuccessno0:08:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18463step1-simulationsuccessno0:10:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17838step1-simulationsuccessno0:12:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17800step1-simulationtimeoutno0:25:14
I can see how the jo [...]
I can see how the job 17800 is timeout because passed 1514 seconds and the timeout is 1200.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17645step1-simulationsuccessno0:18:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17635step1-simulationhost-errorno0:02:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown-shell/lib/dt_shell/env_checks.py", line 60, in check_docker_environment
    containers = client.containers.list(filters=dict(status='running'))
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 925, in list
    containers.append(self.get(r['Id']))
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 862, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/container.py", line 755, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/containers/641e105b013a6ab4a60444f587bed6ece194f036ad1787d4e20e026b45fc10cb/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 665, in run_one
    client = check_docker_environment()
  File "/project/src/duckietown-shell/lib/dt_shell/env_checks.py", line 67, in check_docker_environment
    raise InvalidEnvironment(msg)
dt_shell.exceptions.InvalidEnvironment: I cannot communicate with Docker:
404 Client Error: Not Found for url: http+docker://localhost/v1.35/containers/641e105b013a6ab4a60444f587bed6ece194f036ad1787d4e20e026b45fc10cb/json

Make sure the docker service is running.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible