Duckietown Challenges Home Challenges Submissions

Submission 2375

Submission2375
Competingyes
Challengeaido2-LF-sim-testing
UserLiam Paull 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 23022
Next
User labelminimal_agent_python2 (Python 2)
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
23022step1-simulationsuccessyes0:20:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6560989890090594
survival_time_median3.649999999999995
deviation-center-line_median0.14678748740599404
in-drivable-lane_median0.34999999999999876


other stats
agent_compute-ego_max0.07209179825978736
agent_compute-ego_mean0.06758519305273342
agent_compute-ego_median0.06859314228807177
agent_compute-ego_min0.051842579970488675
deviation-center-line_max0.4005818627784723
deviation-center-line_mean0.19535396696552737
deviation-center-line_min0.12422801598242109
deviation-heading_max1.0645521116851462
deviation-heading_mean0.5821209778469638
deviation-heading_median0.5423458590238992
deviation-heading_min0.29987290367335917
driven_any_max2.1648273185719797
driven_any_mean0.9348502469008624
driven_any_median0.9948273195001768
driven_any_min0.2750262368649079
driven_lanedir_consec_max2.0050790504729665
driven_lanedir_consec_mean0.738495147174259
driven_lanedir_consec_min0.2671322113623069
driven_lanedir_max2.0050790504729665
driven_lanedir_mean0.738495147174259
driven_lanedir_median0.6560989890090594
driven_lanedir_min0.2671322113623069
in-drivable-lane_max1.5499999999999945
in-drivable-lane_mean0.5433333333333313
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9948273195001778, "sim_physics": 0.12533506301984396, "survival_time": 3.649999999999995, "driven_lanedir": 0.6707441932890994, "sim_render-ego": 0.06623670499618739, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.0707444426131575, "deviation-heading": 0.5590533399929921, "set_robot_commands": 0.10569999969168883, "deviation-center-line": 0.14435558631904788, "driven_lanedir_consec": 0.6707441932890994, "sim_compute_sim_state": 0.0417018034686781, "sim_compute_performance-ego": 0.07593215654974114, "sim_compute_robot_state-ego": 0.08581738929225974}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.009827319290899, "sim_physics": 0.111672221003352, "survival_time": 3.699999999999995, "driven_lanedir": 0.8760331754635476, "sim_render-ego": 0.06383419036865234, "in-drivable-lane": 0.1999999999999993, "agent_compute-ego": 0.06460965001905286, "deviation-heading": 1.047774623333474, "set_robot_commands": 0.0966546277742128, "deviation-center-line": 0.29642020115014733, "driven_lanedir_consec": 0.8760331754635476, "sim_compute_sim_state": 0.04164726991911192, "sim_compute_performance-ego": 0.0712896069964847, "sim_compute_robot_state-ego": 0.0771898063453468}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.2750262368649079, "sim_physics": 0.13439103126525878, "survival_time": 1.2500000000000004, "driven_lanedir": 0.2671322113623069, "sim_render-ego": 0.06533352851867676, "in-drivable-lane": 0, "agent_compute-ego": 0.06736084938049317, "deviation-heading": 0.3001803644882976, "set_robot_commands": 0.09313840866088868, "deviation-center-line": 0.12422801598242109, "driven_lanedir_consec": 0.2671322113623069, "sim_compute_sim_state": 0.03871442794799805, "sim_compute_performance-ego": 0.07005731582641601, "sim_compute_robot_state-ego": 0.07454339981079101}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4548365617710419, "sim_physics": 0.1236608479474042, "survival_time": 1.850000000000001, "driven_lanedir": 0.4333629680464388, "sim_render-ego": 0.061958061682211386, "in-drivable-lane": 0, "agent_compute-ego": 0.06768019134933884, "deviation-heading": 0.5707543029776994, "set_robot_commands": 0.09602139447186445, "deviation-center-line": 0.12501321046442526, "driven_lanedir_consec": 0.4333629680464388, "sim_compute_sim_state": 0.041993353817913984, "sim_compute_performance-ego": 0.07078346046241554, "sim_compute_robot_state-ego": 0.07428789783168484}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.5798273185719856, "sim_physics": 0.12039540920938764, "survival_time": 5.599999999999988, "driven_lanedir": 1.1257539445961453, "sim_render-ego": 0.06481583416461945, "in-drivable-lane": 1.499999999999995, "agent_compute-ego": 0.06859314228807177, "deviation-heading": 0.31699480744097813, "set_robot_commands": 0.099961485181536, "deviation-center-line": 0.4005818627784723, "driven_lanedir_consec": 1.1257539445961453, "sim_compute_sim_state": 0.04112331782068525, "sim_compute_performance-ego": 0.07156882115772792, "sim_compute_robot_state-ego": 0.07865693739482335}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.454836561771035, "sim_physics": 0.14149695473748283, "survival_time": 1.850000000000001, "driven_lanedir": 0.4295977528045558, "sim_render-ego": 0.0682466610057934, "in-drivable-lane": 0, "agent_compute-ego": 0.06962898615244273, "deviation-heading": 0.6191860504142409, "set_robot_commands": 0.10470951570046914, "deviation-center-line": 0.13177329190693754, "driven_lanedir_consec": 0.4295977528045558, "sim_compute_sim_state": 0.04156313715754328, "sim_compute_performance-ego": 0.07380382434741871, "sim_compute_robot_state-ego": 0.07786806854041847}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0248273191286392, "sim_physics": 0.1286488978068034, "survival_time": 3.7499999999999942, "driven_lanedir": 0.5418367454355217, "sim_render-ego": 0.06740750948588053, "in-drivable-lane": 1.5499999999999945, "agent_compute-ego": 0.07059236208597819, "deviation-heading": 0.5423458590238992, "set_robot_commands": 0.09984731356302898, "deviation-center-line": 0.14678748740599404, "driven_lanedir_consec": 0.5418367454355217, "sim_compute_sim_state": 0.042226123809814456, "sim_compute_performance-ego": 0.07454467455546061, "sim_compute_robot_state-ego": 0.0785951042175293}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3049465552081734, "sim_physics": 0.12590600826122142, "survival_time": 1.3500000000000003, "driven_lanedir": 0.29745564201515107, "sim_render-ego": 0.06826369850723832, "in-drivable-lane": 0, "agent_compute-ego": 0.06691541495146575, "deviation-heading": 0.29987290367335917, "set_robot_commands": 0.10202311586450648, "deviation-center-line": 0.1306820468362013, "driven_lanedir_consec": 0.29745564201515107, "sim_compute_sim_state": 0.040814744101630315, "sim_compute_performance-ego": 0.07954993954411259, "sim_compute_robot_state-ego": 0.0890966256459554}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.7098274382089755, "sim_physics": 0.12025599126462584, "survival_time": 2.6999999999999984, "driven_lanedir": 0.7032085791835883, "sim_render-ego": 0.06493922957667599, "in-drivable-lane": 0, "agent_compute-ego": 0.0695848994784885, "deviation-heading": 0.36899467884091897, "set_robot_commands": 0.0996176799138387, "deviation-center-line": 0.18371297048277177, "driven_lanedir_consec": 0.7032085791835883, "sim_compute_sim_state": 0.04079538804513437, "sim_compute_performance-ego": 0.07327796353234185, "sim_compute_robot_state-ego": 0.07564389705657959}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.9948273195001768, "sim_physics": 0.13081961135341696, "survival_time": 3.649999999999995, "driven_lanedir": 0.6560989890090594, "sim_render-ego": 0.06792803986431802, "in-drivable-lane": 0.8999999999999968, "agent_compute-ego": 0.06745528848203894, "deviation-heading": 0.6658599650846626, "set_robot_commands": 0.10067318563591944, "deviation-center-line": 0.1561741058626541, "driven_lanedir_consec": 0.6560989890090594, "sim_compute_sim_state": 0.041726102567698854, "sim_compute_performance-ego": 0.07684120413375227, "sim_compute_robot_state-ego": 0.08318378827343248}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5798273185720009, "sim_physics": 0.12437527520315989, "survival_time": 5.599999999999988, "driven_lanedir": 1.2014600726933682, "sim_render-ego": 0.0656459629535675, "in-drivable-lane": 1.1999999999999955, "agent_compute-ego": 0.06711894273757935, "deviation-heading": 0.5156620482368139, "set_robot_commands": 0.09772745413439614, "deviation-center-line": 0.20858216306835017, "driven_lanedir_consec": 1.2014600726933682, "sim_compute_sim_state": 0.04127084357397897, "sim_compute_performance-ego": 0.07285428898675102, "sim_compute_robot_state-ego": 0.0795153294290815}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.0098273192909009, "sim_physics": 0.08866000497663343, "survival_time": 3.699999999999995, "driven_lanedir": 0.6405535448452877, "sim_render-ego": 0.0508708889419968, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.051842579970488675, "deviation-heading": 1.0645521116851462, "set_robot_commands": 0.07455145346151816, "deviation-center-line": 0.2228681204106694, "driven_lanedir_consec": 0.6405535448452877, "sim_compute_sim_state": 0.030837854823550663, "sim_compute_performance-ego": 0.05634660656387742, "sim_compute_robot_state-ego": 0.05690912620441334}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.9948273195001752, "sim_physics": 0.11515083704909232, "survival_time": 3.649999999999995, "driven_lanedir": 0.7695653387571403, "sim_render-ego": 0.06251164658428872, "in-drivable-lane": 0.6499999999999977, "agent_compute-ego": 0.07209179825978736, "deviation-heading": 0.4813700122068389, "set_robot_commands": 0.09772268060135517, "deviation-center-line": 0.1442443284046606, "driven_lanedir_consec": 0.7695653387571403, "sim_compute_sim_state": 0.03991619854757231, "sim_compute_performance-ego": 0.06994176890752087, "sim_compute_robot_state-ego": 0.0747600744848382}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.4698344777618675, "sim_physics": 0.14442910646137438, "survival_time": 1.900000000000001, "driven_lanedir": 0.4595449996397094, "sim_render-ego": 0.06771891368062873, "in-drivable-lane": 0, "agent_compute-ego": 0.07054565454784192, "deviation-heading": 0.398367781186276, "set_robot_commands": 0.10598785626260858, "deviation-center-line": 0.1252701007981349, "driven_lanedir_consec": 0.4595449996397094, "sim_compute_sim_state": 0.042283171101620325, "sim_compute_performance-ego": 0.08167983356275056, "sim_compute_robot_state-ego": 0.0916382764515124}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1648273185719797, "sim_physics": 0.12519784636844863, "survival_time": 7.549999999999981, "driven_lanedir": 2.0050790504729665, "sim_render-ego": 0.06655987524828375, "in-drivable-lane": 0.34999999999999876, "agent_compute-ego": 0.06901369347477591, "deviation-heading": 0.9808458191188588, "set_robot_commands": 0.1024602675280034, "deviation-center-line": 0.3896160126120228, "driven_lanedir_consec": 2.0050790504729665, "sim_compute_sim_state": 0.04174732056674579, "sim_compute_performance-ego": 0.07515954813420378, "sim_compute_robot_state-ego": 0.08142730573944698}}
set_robot_commands_max0.10598785626260858
set_robot_commands_mean0.098453095896389
set_robot_commands_median0.09984731356302898
set_robot_commands_min0.07455145346151816
sim_compute_performance-ego_max0.08167983356275056
sim_compute_performance-ego_mean0.07290873421739835
sim_compute_performance-ego_median0.07327796353234185
sim_compute_performance-ego_min0.05634660656387742
sim_compute_robot_state-ego_max0.0916382764515124
sim_compute_robot_state-ego_mean0.07860886844787424
sim_compute_robot_state-ego_median0.0785951042175293
sim_compute_robot_state-ego_min0.05690912620441334
sim_compute_sim_state_max0.042283171101620325
sim_compute_sim_state_mean0.04055740381797844
sim_compute_sim_state_median0.04156313715754328
sim_compute_sim_state_min0.030837854823550663
sim_physics_max0.14442910646137438
sim_physics_mean0.12402634039516704
sim_physics_median0.12519784636844863
sim_physics_min0.08866000497663343
sim_render-ego_max0.06826369850723832
sim_render-ego_mean0.06481804970526796
sim_render-ego_median0.0656459629535675
sim_render-ego_min0.0508708889419968
simulation-passed1
survival_time_max7.549999999999981
survival_time_mean3.4499999999999957
survival_time_min1.2500000000000004
No reset possible
21364step1-simulationsuccessno0:18:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20641step1-simulationsuccessno0:04:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20640step1-simulationsuccessno0:05:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20613step1-simulationhost-errorno0:01:14
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20613-86732', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20613-86732', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20613-86732', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19666step1-simulationsuccessno0:07:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18915step1-simulationsuccessno0:08:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18544step1-simulationsuccessno0:11:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18248step1-simulationsuccessno0:07:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17969step1-simulationsuccessno0:06:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17877step1-simulationsuccessno0:03:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17546step1-simulationsuccessno0:04:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible