Job ID submission user user label challenge step status up to date evaluator date started date completed duration message 8461216837Lin Wei-Chih template-ros mooc-modconsim success no nogpu-production-b-spot-1-022021-08-01 12:12:14+00:00 2021-08-01 12:24:15+00:00 0:12:01 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
in-drivable-lane_median 30.549999999999866 deviation-center-line_median 0.4139659671224809 driven_lanedir_consec_median 0.5319592549778875 survival_time_median 34.72499999999973
other stats agent_compute-ego0_max 0.015905068268128975 agent_compute-ego0_mean 0.010685231895484535 agent_compute-ego0_median 0.010685231895484535 agent_compute-ego0_min 0.0054653955228400955 complete-iteration_max 0.18769144125320208 complete-iteration_mean 0.1838757005226648 complete-iteration_median 0.1838757005226648 complete-iteration_min 0.18005995979212752 deviation-center-line_max 0.547054754526366 deviation-center-line_mean 0.4139659671224809 deviation-center-line_min 0.2808771797185957 deviation-heading_max 1.6658678596787482 deviation-heading_mean 1.636499156089971 deviation-heading_median 1.636499156089971 deviation-heading_min 1.607130452501194 distance-from-start_max 6.66453407839656 distance-from-start_mean 4.553190921312204 distance-from-start_median 4.553190921312204 distance-from-start_min 2.441847764227848 driven_any_max 6.800812134112666 driven_any_mean 4.700889598067616 driven_any_median 4.700889598067616 driven_any_min 2.6009670620225656 driven_lanedir_consec_max 0.7081264423278091 driven_lanedir_consec_mean 0.5319592549778875 driven_lanedir_consec_min 0.355792067627966 driven_lanedir_max 0.7081264423278091 driven_lanedir_mean 0.5319592549778875 driven_lanedir_median 0.5319592549778875 driven_lanedir_min 0.355792067627966 get_duckie_state_max 9.91002477780737e-07 get_duckie_state_mean 9.794070886346955e-07 get_duckie_state_median 9.794070886346955e-07 get_duckie_state_min 9.678116994886542e-07 get_robot_state_max 0.0030537620939389623 get_robot_state_mean 0.0030347432961809317 get_robot_state_median 0.0030347432961809317 get_robot_state_min 0.0030157244984229006 get_state_dump_max 0.003902391953901811 get_state_dump_mean 0.003878176882148279 get_state_dump_median 0.003878176882148279 get_state_dump_min 0.0038539618103947473 get_ui_image_max 0.06242911020914713 get_ui_image_mean 0.061950390342891314 get_ui_image_median 0.061950390342891314 get_ui_image_min 0.061471670476635494 in-drivable-lane_max 44.34999999999962 in-drivable-lane_mean 30.549999999999866 in-drivable-lane_min 16.750000000000103 per-episodes details {"LF-full-loop-000-ego0": {"driven_any": 6.800812134112666, "get_ui_image": 0.061471670476635494, "step_physics": 0.07180605701465703, "survival_time": 49.699999999999314, "driven_lanedir": 0.7081264423278091, "get_state_dump": 0.0038539618103947473, "get_robot_state": 0.0030157244984229006, "sim_render-ego0": 0.0030643271441435694, "get_duckie_state": 9.678116994886542e-07, "in-drivable-lane": 44.34999999999962, "deviation-heading": 1.607130452501194, "agent_compute-ego0": 0.015905068268128975, "complete-iteration": 0.18769144125320208, "set_robot_commands": 0.0017553451672271268, "distance-from-start": 6.66453407839656, "deviation-center-line": 0.547054754526366, "driven_lanedir_consec": 0.7081264423278091, "sim_compute_sim_state": 0.02520365738988522, "sim_compute_performance-ego0": 0.001542720363367742}, "LF-full-loop-001-ego0": {"driven_any": 2.6009670620225656, "get_ui_image": 0.06242911020914713, "step_physics": 0.07375510533650716, "survival_time": 19.750000000000146, "driven_lanedir": 0.355792067627966, "get_state_dump": 0.003902391953901811, "get_robot_state": 0.0030537620939389623, "sim_render-ego0": 0.0031480048642014012, "get_duckie_state": 9.91002477780737e-07, "in-drivable-lane": 16.750000000000103, "deviation-heading": 1.6658678596787482, "agent_compute-ego0": 0.0054653955228400955, "complete-iteration": 0.18005995979212752, "set_robot_commands": 0.0017713537119855784, "distance-from-start": 2.441847764227848, "deviation-center-line": 0.2808771797185957, "driven_lanedir_consec": 0.355792067627966, "sim_compute_sim_state": 0.02487158715122878, "sim_compute_performance-ego0": 0.0015882482432355784}}set_robot_commands_max 0.0017713537119855784 set_robot_commands_mean 0.0017633494396063526 set_robot_commands_median 0.0017633494396063526 set_robot_commands_min 0.0017553451672271268 sim_compute_performance-ego0_max 0.0015882482432355784 sim_compute_performance-ego0_mean 0.00156548430330166 sim_compute_performance-ego0_median 0.00156548430330166 sim_compute_performance-ego0_min 0.001542720363367742 sim_compute_sim_state_max 0.02520365738988522 sim_compute_sim_state_mean 0.025037622270557 sim_compute_sim_state_median 0.025037622270557 sim_compute_sim_state_min 0.02487158715122878 sim_render-ego0_max 0.0031480048642014012 sim_render-ego0_mean 0.0031061660041724853 sim_render-ego0_median 0.0031061660041724853 sim_render-ego0_min 0.0030643271441435694 simulation-passed 1 step_physics_max 0.07375510533650716 step_physics_mean 0.07278058117558209 step_physics_median 0.07278058117558209 step_physics_min 0.07180605701465703 survival_time_max 49.699999999999314 survival_time_mean 34.72499999999973 survival_time_min 19.750000000000146
No reset possible 8432116658Bea Baselines baseline-solution mooc-planning-cc-static-valistep1-1of4 success yes nogpu-production-b-spot-1-022021-07-30 17:42:38+00:00 2021-07-30 17:48:42+00:00 0:06:04 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes 1.0 success_ratio 1.0 duration 36.598764110266906 complexity 114.16666666666669 avg_min_distance 0.1936511494140563
other stats No reset possible 8430816661Bea Baselines baseline-solution mooc-planning-cc-dynamic-teststep1-0of4 success yes nogpu-production-b-spot-1-022021-07-30 17:36:44+00:00 2021-07-30 17:42:17+00:00 0:05:33 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes 1.0 success_ratio 1.0 duration 30.59995140807227 complexity 88.66666666666667 avg_min_distance 0.016524361787006125
other stats No reset possible 8429816655Bea Baselines baseline-solution mooc-planning-dd-dynamic-teststep1-0of4 success yes nogpu-production-b-spot-1-022021-07-30 17:30:58+00:00 2021-07-30 17:36:26+00:00 0:05:28 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes 0.5 success_ratio 1.0 duration 32.30249940990253 complexity 92.33333333333331 avg_min_distance 0.022206701485941265
other stats No reset possible 8420816631Arwa Alabdulkarim base-image-ml mooc-visservoingsim success no nogpu-production-b-spot-1-022021-07-30 13:33:31+00:00 2021-07-30 13:49:03+00:00 0:15:32 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
in-drivable-lane_median 0.0 deviation-center-line_median 5.2969981341196 driven_lanedir_consec_median 6.141876998076276 survival_time_median 59.99999999999873
other stats agent_compute-ego0_max 0.005349533841770753 agent_compute-ego0_mean 0.005256282399834245 agent_compute-ego0_median 0.005256282399834245 agent_compute-ego0_min 0.005163030957897736 complete-iteration_max 0.1155915820132088 complete-iteration_mean 0.11411166191101074 complete-iteration_median 0.11411166191101074 complete-iteration_min 0.11263174180881268 deviation-center-line_max 5.398618713169436 deviation-center-line_mean 5.2969981341196 deviation-center-line_min 5.195377555069764 deviation-heading_max 10.33064373730566 deviation-heading_mean 8.898948456202337 deviation-heading_median 8.898948456202337 deviation-heading_min 7.467253175099015 distance-from-start_max 1.619585673173333 distance-from-start_mean 1.3421060707079462 distance-from-start_median 1.3421060707079462 distance-from-start_min 1.0646264682425597 driven_any_max 6.248288158309491 driven_any_mean 6.245714048063916 driven_any_median 6.245714048063916 driven_any_min 6.2431399378183405 driven_lanedir_consec_max 6.175672149118538 driven_lanedir_consec_mean 6.141876998076276 driven_lanedir_consec_min 6.108081847034014 driven_lanedir_max 6.175672149118538 driven_lanedir_mean 6.141876998076276 driven_lanedir_median 6.141876998076276 driven_lanedir_min 6.108081847034014 get_duckie_state_max 1.130155679288256e-06 get_duckie_state_mean 1.0389372470674666e-06 get_duckie_state_median 1.0389372470674666e-06 get_duckie_state_min 9.477188148466772e-07 get_robot_state_max 0.0033407237110884363 get_robot_state_mean 0.0033257893976025737 get_robot_state_median 0.0033257893976025737 get_robot_state_min 0.003310855084116711 get_state_dump_max 0.004239215739660716 get_state_dump_mean 0.004199218888961703 get_state_dump_median 0.004199218888961703 get_state_dump_min 0.0041592220382626905 get_ui_image_max 0.03770640966397936 get_ui_image_mean 0.036815944063375634 get_ui_image_median 0.036815944063375634 get_ui_image_min 0.03592547846277191 in-drivable-lane_max 0.0 in-drivable-lane_mean 0.0 in-drivable-lane_min 0.0 per-episodes details {"LF-small-loop-000-ego0": {"driven_any": 6.248288158309491, "get_ui_image": 0.03592547846277191, "step_physics": 0.05287679526132906, "survival_time": 59.99999999999873, "driven_lanedir": 6.175672149118538, "get_state_dump": 0.004239215739660716, "get_robot_state": 0.0033407237110884363, "sim_render-ego0": 0.00335482991208244, "get_duckie_state": 1.130155679288256e-06, "in-drivable-lane": 0.0, "deviation-heading": 7.467253175099015, "agent_compute-ego0": 0.005349533841770753, "complete-iteration": 0.11263174180881268, "set_robot_commands": 0.0019949064167413388, "distance-from-start": 1.0646264682425597, "deviation-center-line": 5.195377555069764, "driven_lanedir_consec": 6.175672149118538, "sim_compute_sim_state": 0.003588794768601035, "sim_compute_performance-ego0": 0.0018778937146824463}, "LF-small-loop-001-ego0": {"driven_any": 6.2431399378183405, "get_ui_image": 0.03770640966397936, "step_physics": 0.05479619028566283, "survival_time": 59.99999999999873, "driven_lanedir": 6.108081847034014, "get_state_dump": 0.0041592220382626905, "get_robot_state": 0.003310855084116711, "sim_render-ego0": 0.00322262293889461, "get_duckie_state": 9.477188148466772e-07, "in-drivable-lane": 0.0, "deviation-heading": 10.33064373730566, "agent_compute-ego0": 0.005163030957897736, "complete-iteration": 0.1155915820132088, "set_robot_commands": 0.001900971283225791, "distance-from-start": 1.619585673173333, "deviation-center-line": 5.398618713169436, "driven_lanedir_consec": 6.108081847034014, "sim_compute_sim_state": 0.0035750665434393456, "sim_compute_performance-ego0": 0.0016816345281545368}}set_robot_commands_max 0.0019949064167413388 set_robot_commands_mean 0.0019479388499835648 set_robot_commands_median 0.0019479388499835648 set_robot_commands_min 0.001900971283225791 sim_compute_performance-ego0_max 0.0018778937146824463 sim_compute_performance-ego0_mean 0.0017797641214184917 sim_compute_performance-ego0_median 0.0017797641214184917 sim_compute_performance-ego0_min 0.0016816345281545368 sim_compute_sim_state_max 0.003588794768601035 sim_compute_sim_state_mean 0.0035819306560201905 sim_compute_sim_state_median 0.0035819306560201905 sim_compute_sim_state_min 0.0035750665434393456 sim_render-ego0_max 0.00335482991208244 sim_render-ego0_mean 0.003288726425488525 sim_render-ego0_median 0.003288726425488525 sim_render-ego0_min 0.00322262293889461 simulation-passed 1 step_physics_max 0.05479619028566283 step_physics_mean 0.05383649277349595 step_physics_median 0.05383649277349595 step_physics_min 0.05287679526132906 survival_time_max 59.99999999999873 survival_time_mean 59.99999999999873 survival_time_min 59.99999999999873
No reset possible 8419916587Bea Baselines planning-exercise mooc-planning-cc-dynamic-valistep1-3of4 success yes nogpu-production-b-spot-1-022021-07-30 13:30:31+00:00 2021-07-30 13:33:25+00:00 0:02:54 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes 1.0 success_ratio 1.0 duration 22.0 complexity 8.0 avg_min_distance 0.0
other stats No reset possible 8419016598Bea Baselines baseline-solution mooc-planning-cc-static-teststep1-2of4 success yes nogpu-production-b-spot-1-022021-07-30 13:25:59+00:00 2021-07-30 13:30:13+00:00 0:04:14 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes 1.0 success_ratio 1.0 duration 24.94836262025912 complexity 76.66666666666667 avg_min_distance 0.21483334054841263
other stats No reset possible 8417416592Bea Baselines baseline-solution mooc-planning-dd-static-teststep1-1of4 success yes nogpu-production-b-spot-1-022021-07-30 13:20:26+00:00 2021-07-30 13:25:27+00:00 0:05:01 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes 0.0 success_ratio 1.0 duration 33.00657813759664 complexity 97.5 avg_min_distance 0.19126888599160857
other stats No reset possible 8416816588Bea Baselines planning-exercise mooc-planning-cc-dynamic-teststep1-1of4 success yes nogpu-production-b-spot-1-022021-07-30 13:17:02+00:00 2021-07-30 13:20:13+00:00 0:03:11 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes 1.0 success_ratio 1.0 duration 22.0 complexity 8.0 avg_min_distance 0.0
other stats No reset possible 8415413798Nicholas Kostelnik template-random aido-hello-sim-validation370 aborted yes nogpu-production-b-spot-1-022021-07-30 13:16:06+00:00 2021-07-30 13:16:29+00:00 0:00:23 Uncaught exception:
[...] Uncaught exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
response.raise_for_status()
File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=2021_03_25_17_23_41&fromImage=docker.io%2Fnitaigao%2Faido-submissions
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 93, in docker_pull
pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 430, in pull
self._raise_for_status(response)
File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=2021_03_25_17_23_41&fromImage=docker.io%2Fnitaigao%2Faido-submissions: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 112, in docker_pull
raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691 tag 2021_03_25_17_23_41
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
cr = run_single(
File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 51, in docker_pull_retry
raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull nitaigao/aido-submissions:2021_03_25_17_23_41@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 8413215873Bea Baselines dummy_collision_checker mooc-collision-check-teststep1 success yes nogpu-production-b-spot-1-022021-07-30 13:11:03+00:00 2021-07-30 13:11:36+00:00 0:00:33 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
false_negative_rate 0.53 false_positive_rate 0.59
other stats No reset possible 8410215873Bea Baselines dummy_collision_checker mooc-collision-check-teststep1 success yes nogpu-production-b-spot-1-022021-07-30 13:09:09+00:00 2021-07-30 13:10:50+00:00 0:01:41 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
false_negative_rate 0.55 false_positive_rate 0.56
other stats No reset possible
Lin Wei-Chih
Bea Baselines 🐤
Arwa Alabdulkarim
Nicholas Kostelnik