Duckietown Challenges Home Challenges Submissions

Evaluator 5309

ID5309
evaluatornogpu-production-b-spot-0-05
ownerI don't have one 😀
machinenogpu-production-b-spot-0_e77cba509819
processnogpu-production-b-spot-0-05_e77cba509819
version6.2.7
first heard
last heard
statusinactive
# evaluating
# success2 79210
# timeout
# failed
# error
# aborted1 79188
# host-error
arm0
x86_641
Mac0
gpu available0
Number of processors96
Processor frequency (MHz)0.0 GHz
Free % of processors99%
RAM total (MB)184.7 GB
RAM free (MB)176.0 GB
Disk (MB)969.3 GB
Disk available (MB)922.4 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
7957016072Evanthia Fasoulaexercises_braitenbergmooc-BV1sim-0of5successnonogpu-production-b-spot-0-050:01:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean2.904665548896716


other stats
agent_compute-ego0_max0.010814087450011703
agent_compute-ego0_mean0.010814087450011703
agent_compute-ego0_median0.010814087450011703
agent_compute-ego0_min0.010814087450011703
complete-iteration_max0.23310888897288928
complete-iteration_mean0.23310888897288928
complete-iteration_median0.23310888897288928
complete-iteration_min0.23310888897288928
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max2.904665548896716
distance-from-start_median2.904665548896716
distance-from-start_min2.904665548896716
driven_any_max2.9087124182635193
driven_any_mean2.9087124182635193
driven_any_median2.9087124182635193
driven_any_min2.9087124182635193
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.07823779957353576
get_duckie_state_mean0.07823779957353576
get_duckie_state_median0.07823779957353576
get_duckie_state_min0.07823779957353576
get_robot_state_max0.0032903419053258976
get_robot_state_mean0.0032903419053258976
get_robot_state_median0.0032903419053258976
get_robot_state_min0.0032903419053258976
get_state_dump_max0.01621300129851034
get_state_dump_mean0.01621300129851034
get_state_dump_median0.01621300129851034
get_state_dump_min0.01621300129851034
get_ui_image_max0.04014809466590566
get_ui_image_mean0.04014809466590566
get_ui_image_median0.04014809466590566
get_ui_image_min0.04014809466590566
in-drivable-lane_max5.999999999999987
in-drivable-lane_mean5.999999999999987
in-drivable-lane_median5.999999999999987
in-drivable-lane_min5.999999999999987
per-episodes
details{"d45-ego0": {"driven_any": 2.9087124182635193, "get_ui_image": 0.04014809466590566, "step_physics": 0.06979823506568089, "survival_time": 5.999999999999987, "driven_lanedir": 0.0, "get_state_dump": 0.01621300129851034, "get_robot_state": 0.0032903419053258976, "sim_render-ego0": 0.0031756940952017286, "get_duckie_state": 0.07823779957353576, "in-drivable-lane": 5.999999999999987, "deviation-heading": 0.0, "agent_compute-ego0": 0.010814087450011703, "complete-iteration": 0.23310888897288928, "set_robot_commands": 0.0019538166109195425, "distance-from-start": 2.904665548896716, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.007738223745803203, "sim_compute_performance-ego0": 0.0016504673918416677}}
set_robot_commands_max0.0019538166109195425
set_robot_commands_mean0.0019538166109195425
set_robot_commands_median0.0019538166109195425
set_robot_commands_min0.0019538166109195425
sim_compute_performance-ego0_max0.0016504673918416677
sim_compute_performance-ego0_mean0.0016504673918416677
sim_compute_performance-ego0_median0.0016504673918416677
sim_compute_performance-ego0_min0.0016504673918416677
sim_compute_sim_state_max0.007738223745803203
sim_compute_sim_state_mean0.007738223745803203
sim_compute_sim_state_median0.007738223745803203
sim_compute_sim_state_min0.007738223745803203
sim_render-ego0_max0.0031756940952017286
sim_render-ego0_mean0.0031756940952017286
sim_render-ego0_median0.0031756940952017286
sim_render-ego0_min0.0031756940952017286
simulation-passed1
step_physics_max0.06979823506568089
step_physics_mean0.06979823506568089
step_physics_median0.06979823506568089
step_physics_min0.06979823506568089
survival_time_max5.999999999999987
survival_time_mean5.999999999999987
survival_time_median5.999999999999987
survival_time_min5.999999999999987
No reset possible
7921015955Alessandro Zanardi 🇨🇭minimal-agent-fullaido-LFVI_multi_full-sim-validationsim-1of4successyesnogpu-production-b-spot-0-050:24:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median59.99999999999873
in-drivable-lane_median0.0
driven_lanedir_consec_median2.789237059087774
deviation-center-line_median0.8149470111780187


other stats
agent_compute-ego0_max0.02723802416449681
agent_compute-ego0_mean0.02723802416449681
agent_compute-ego0_median0.02723802416449681
agent_compute-ego0_min0.02723802416449681
agent_compute-ego1_max0.033259710006967966
agent_compute-ego1_mean0.033259710006967966
agent_compute-ego1_median0.033259710006967966
agent_compute-ego1_min0.033259710006967966
agent_compute-ego2_max0.02704832496293677
agent_compute-ego2_mean0.02704832496293677
agent_compute-ego2_median0.02704832496293677
agent_compute-ego2_min0.02704832496293677
agent_compute-ego3_max0.03381150747516769
agent_compute-ego3_mean0.03381150747516769
agent_compute-ego3_median0.03381150747516769
agent_compute-ego3_min0.03381150747516769
complete-iteration_max0.4705794637745167
complete-iteration_mean0.4705794637745167
complete-iteration_median0.4705794637745167
complete-iteration_min0.4705794637745167
deviation-center-line_max3.345336382076788
deviation-center-line_mean1.2965621179694184
deviation-center-line_min0.21101806744484797
deviation-heading_max29.009133619514312
deviation-heading_mean10.658006965481896
deviation-heading_median5.830211270516446
deviation-heading_min1.9624717013803823
distance-from-start_max3.515438928989946
distance-from-start_mean1.591831360221949
distance-from-start_median1.425943255948925
distance-from-start_min0.0
driven_any_max5.928215603737186
driven_any_mean2.9536774446556624
driven_any_median2.943247087442732
driven_any_min0.0
driven_lanedir_consec_max5.916338856038844
driven_lanedir_consec_mean2.8737032435535985
driven_lanedir_consec_min0.0
driven_lanedir_max5.9163390359320305
driven_lanedir_mean2.9479859918217457
driven_lanedir_median2.9378024656774766
driven_lanedir_min0.0
get_duckie_state_max2.326218909169117e-06
get_duckie_state_mean2.326218909169117e-06
get_duckie_state_median2.326218909169117e-06
get_duckie_state_min2.326218909169117e-06
get_robot_state_max0.01393946223612332
get_robot_state_mean0.01393946223612332
get_robot_state_median0.01393946223612332
get_robot_state_min0.01393946223612332
get_state_dump_max0.008285888525766696
get_state_dump_mean0.008285888525766696
get_state_dump_median0.008285888525766696
get_state_dump_min0.008285888525766696
get_ui_image_max0.05423903207199262
get_ui_image_mean0.05423903207199262
get_ui_image_median0.05423903207199262
get_ui_image_min0.05423903207199262
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LFVI_multi-full-udem1-000-ego0": {"driven_any": 5.95808196779335e-13, "get_ui_image": 0.05423903207199262, "step_physics": 0.21602272669738973, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.008285888525766696, "get_robot_state": 0.01393946223612332, "sim_render-ego0": 0.003635486893411679, "sim_render-ego1": 0.003223661975400037, "sim_render-ego2": 0.003242527416207014, "sim_render-ego3": 0.0032478361502972174, "get_duckie_state": 2.326218909169117e-06, "in-drivable-lane": 0.0, "deviation-heading": 9.544591376009151, "agent_compute-ego0": 0.02723802416449681, "agent_compute-ego1": 0.033259710006967966, "agent_compute-ego2": 0.02704832496293677, "agent_compute-ego3": 0.03381150747516769, "complete-iteration": 0.4705794637745167, "set_robot_commands": 0.002076370134440985, "distance-from-start": 4.965068306494546e-16, "deviation-center-line": 3.345336382076788, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.02791250615592404, "sim_compute_performance-ego0": 0.0022005211006691813, "sim_compute_performance-ego1": 0.001655215923236272, "sim_compute_performance-ego2": 0.0016515435624579208, "sim_compute_performance-ego3": 0.0016590093792130012}, "LFVI_multi-full-udem1-000-ego1": {"driven_any": 5.928215603737186, "get_ui_image": 0.05423903207199262, "step_physics": 0.21602272669738973, "survival_time": 59.99999999999873, "driven_lanedir": 5.9163390359320305, "get_state_dump": 0.008285888525766696, "get_robot_state": 0.01393946223612332, "sim_render-ego0": 0.003635486893411679, "sim_render-ego1": 0.003223661975400037, "sim_render-ego2": 0.003242527416207014, "sim_render-ego3": 0.0032478361502972174, "get_duckie_state": 2.326218909169117e-06, "in-drivable-lane": 0.0, "deviation-heading": 2.1158311650237387, "agent_compute-ego0": 0.02723802416449681, "agent_compute-ego1": 0.033259710006967966, "agent_compute-ego2": 0.02704832496293677, "agent_compute-ego3": 0.03381150747516769, "complete-iteration": 0.4705794637745167, "set_robot_commands": 0.002076370134440985, "distance-from-start": 2.85188651189785, "deviation-center-line": 0.21101806744484797, "driven_lanedir_consec": 5.916338856038844, "sim_compute_sim_state": 0.02791250615592404, "sim_compute_performance-ego0": 0.0022005211006691813, "sim_compute_performance-ego1": 0.001655215923236272, "sim_compute_performance-ego2": 0.0016515435624579208, "sim_compute_performance-ego3": 0.0016590093792130012}, "LFVI_multi-full-udem1-000-ego2": {"driven_any": 0.0, "get_ui_image": 0.05423903207199262, "step_physics": 0.21602272669738973, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.008285888525766696, "get_robot_state": 0.01393946223612332, "sim_render-ego0": 0.003635486893411679, "sim_render-ego1": 0.003223661975400037, "sim_render-ego2": 0.003242527416207014, "sim_render-ego3": 0.0032478361502972174, "get_duckie_state": 2.326218909169117e-06, "in-drivable-lane": 0.0, "deviation-heading": 29.009133619514312, "agent_compute-ego0": 0.02723802416449681, "agent_compute-ego1": 0.033259710006967966, "agent_compute-ego2": 0.02704832496293677, "agent_compute-ego3": 0.03381150747516769, "complete-iteration": 0.4705794637745167, "set_robot_commands": 0.002076370134440985, "distance-from-start": 0.0, "deviation-center-line": 1.410755081959342, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.02791250615592404, "sim_compute_performance-ego0": 0.0022005211006691813, "sim_compute_performance-ego1": 0.001655215923236272, "sim_compute_performance-ego2": 0.0016515435624579208, "sim_compute_performance-ego3": 0.0016590093792130012}, "LFVI_multi-full-udem1-000-ego3": {"driven_any": 5.886494174884868, "get_ui_image": 0.05423903207199262, "step_physics": 0.21602272669738973, "survival_time": 59.99999999999873, "driven_lanedir": 5.875604931354953, "get_state_dump": 0.008285888525766696, "get_robot_state": 0.01393946223612332, "sim_render-ego0": 0.003635486893411679, "sim_render-ego1": 0.003223661975400037, "sim_render-ego2": 0.003242527416207014, "sim_render-ego3": 0.0032478361502972174, "get_duckie_state": 2.326218909169117e-06, "in-drivable-lane": 0.0, "deviation-heading": 1.9624717013803823, "agent_compute-ego0": 0.02723802416449681, "agent_compute-ego1": 0.033259710006967966, "agent_compute-ego2": 0.02704832496293677, "agent_compute-ego3": 0.03381150747516769, "complete-iteration": 0.4705794637745167, "set_robot_commands": 0.002076370134440985, "distance-from-start": 3.515438928989946, "deviation-center-line": 0.21913894039669535, "driven_lanedir_consec": 5.578474118175549, "sim_compute_sim_state": 0.02791250615592404, "sim_compute_performance-ego0": 0.0022005211006691813, "sim_compute_performance-ego1": 0.001655215923236272, "sim_compute_performance-ego2": 0.0016515435624579208, "sim_compute_performance-ego3": 0.0016590093792130012}}
set_robot_commands_max0.002076370134440985
set_robot_commands_mean0.002076370134440985
set_robot_commands_median0.002076370134440985
set_robot_commands_min0.002076370134440985
sim_compute_performance-ego0_max0.0022005211006691813
sim_compute_performance-ego0_mean0.0022005211006691813
sim_compute_performance-ego0_median0.0022005211006691813
sim_compute_performance-ego0_min0.0022005211006691813
sim_compute_performance-ego1_max0.001655215923236272
sim_compute_performance-ego1_mean0.001655215923236272
sim_compute_performance-ego1_median0.001655215923236272
sim_compute_performance-ego1_min0.001655215923236272
sim_compute_performance-ego2_max0.0016515435624579208
sim_compute_performance-ego2_mean0.0016515435624579208
sim_compute_performance-ego2_median0.0016515435624579208
sim_compute_performance-ego2_min0.0016515435624579208
sim_compute_performance-ego3_max0.0016590093792130012
sim_compute_performance-ego3_mean0.0016590093792130012
sim_compute_performance-ego3_median0.0016590093792130012
sim_compute_performance-ego3_min0.0016590093792130012
sim_compute_sim_state_max0.02791250615592404
sim_compute_sim_state_mean0.02791250615592404
sim_compute_sim_state_median0.02791250615592404
sim_compute_sim_state_min0.02791250615592404
sim_render-ego0_max0.003635486893411679
sim_render-ego0_mean0.003635486893411679
sim_render-ego0_median0.003635486893411679
sim_render-ego0_min0.003635486893411679
sim_render-ego1_max0.003223661975400037
sim_render-ego1_mean0.003223661975400037
sim_render-ego1_median0.003223661975400037
sim_render-ego1_min0.003223661975400037
sim_render-ego2_max0.003242527416207014
sim_render-ego2_mean0.003242527416207014
sim_render-ego2_median0.003242527416207014
sim_render-ego2_min0.003242527416207014
sim_render-ego3_max0.0032478361502972174
sim_render-ego3_mean0.0032478361502972174
sim_render-ego3_median0.0032478361502972174
sim_render-ego3_min0.0032478361502972174
simulation-passed1
step_physics_max0.21602272669738973
step_physics_mean0.21602272669738973
step_physics_median0.21602272669738973
step_physics_min0.21602272669738973
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
7918813798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortednonogpu-production-b-spot-0-050:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible