Duckietown Challenges Home Challenges Submissions

Evaluator 5371

ID5371
evaluatorgpu-production-spot-0-02
ownerI don't have one 😀
machinegpu-production-spot-0_7d601d8a4727
processgpu-production-spot-0-02_7d601d8a4727
version6.2.7
first heard
last heard
statusinactive
# evaluating
# success4 79762
# timeout
# failed
# error
# aborted1 79843
# host-error
arm0
x86_641
Mac0
gpu available1
Number of processors64
Processor frequency (MHz)0.0 GHz
Free % of processors100%
RAM total (MB)249.0 GB
RAM free (MB)245.9 GB
Disk (MB)969.3 GB
Disk available (MB)853.8 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
7984313798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortednogpu-production-spot-0-020:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7982216100Richard Belangerobjdet exercisemooc-objdetsimsuccessnogpu-production-spot-0-020:06:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median18.325000000000124
driven_lanedir_consec_median4.810667741392846


other stats
agent_compute-ego0_max0.010580028658327849
agent_compute-ego0_mean0.0098223590717406
agent_compute-ego0_median0.0098223590717406
agent_compute-ego0_min0.009064689485153349
complete-iteration_max0.19324677819791047
complete-iteration_mean0.18761826176813745
complete-iteration_median0.18761826176813745
complete-iteration_min0.1819897453383644
deviation-center-line_max0.27010230231285143
deviation-center-line_mean0.2168967593908318
deviation-center-line_median0.2168967593908318
deviation-center-line_min0.16369121646881216
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max6.73934300422758
distance-from-start_mean4.8106218624120345
distance-from-start_median4.8106218624120345
distance-from-start_min2.88190072059649
driven_any_max6.73934300422758
driven_any_mean4.8106218624120345
driven_any_median4.8106218624120345
driven_any_min2.88190072059649
driven_lanedir_consec_max6.739416375324249
driven_lanedir_consec_mean4.810667741392846
driven_lanedir_consec_min2.8819191074614428
driven_lanedir_max6.739416375324249
driven_lanedir_mean4.810667741392846
driven_lanedir_median4.810667741392846
driven_lanedir_min2.8819191074614428
get_duckie_state_max0.02206300963526187
get_duckie_state_mean0.021016896273932783
get_duckie_state_median0.021016896273932783
get_duckie_state_min0.0199707829126037
get_robot_state_max0.003913635792939559
get_robot_state_mean0.003750713750097806
get_robot_state_median0.003750713750097806
get_robot_state_min0.0035877917072560527
get_state_dump_max0.00838613510131836
get_state_dump_mean0.008146664411714761
get_state_dump_median0.008146664411714761
get_state_dump_min0.007907193722111164
get_ui_image_max0.03177795306495998
get_ui_image_mean0.031004710198475068
get_ui_image_median0.031004710198475068
get_ui_image_min0.030231467331990156
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_median0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-long-loop-with-duckies-000-ego0": {"driven_any": 6.73934300422758, "get_ui_image": 0.030231467331990156, "step_physics": 0.07587162197226345, "survival_time": 25.200000000000223, "driven_lanedir": 6.739416375324249, "get_state_dump": 0.007907193722111164, "get_robot_state": 0.0035877917072560527, "sim_render-ego0": 0.003598090445641244, "get_duckie_state": 0.0199707829126037, "in-drivable-lane": 0.0, "deviation-heading": 0.0, "agent_compute-ego0": 0.009064689485153349, "complete-iteration": 0.1819897453383644, "set_robot_commands": 0.0022246450480848255, "distance-from-start": 6.73934300422758, "deviation-center-line": 0.16369121646881216, "driven_lanedir_consec": 6.739416375324249, "sim_compute_sim_state": 0.027520333186234577, "sim_compute_performance-ego0": 0.0019238538081103032}, "LF-long-loop-with-duckies-001-ego0": {"driven_any": 2.88190072059649, "get_ui_image": 0.03177795306495998, "step_physics": 0.0740797467853712, "survival_time": 11.450000000000028, "driven_lanedir": 2.8819191074614428, "get_state_dump": 0.00838613510131836, "get_robot_state": 0.003913635792939559, "sim_render-ego0": 0.003916349618331246, "get_duckie_state": 0.02206300963526187, "in-drivable-lane": 0.0, "deviation-heading": 0.0, "agent_compute-ego0": 0.010580028658327849, "complete-iteration": 0.19324677819791047, "set_robot_commands": 0.0024193235065626062, "distance-from-start": 2.88190072059649, "deviation-center-line": 0.27010230231285143, "driven_lanedir_consec": 2.8819191074614428, "sim_compute_sim_state": 0.03381657600402832, "sim_compute_performance-ego0": 0.0021902550821718964}}
set_robot_commands_max0.0024193235065626062
set_robot_commands_mean0.002321984277323716
set_robot_commands_median0.002321984277323716
set_robot_commands_min0.0022246450480848255
sim_compute_performance-ego0_max0.0021902550821718964
sim_compute_performance-ego0_mean0.0020570544451411
sim_compute_performance-ego0_median0.0020570544451411
sim_compute_performance-ego0_min0.0019238538081103032
sim_compute_sim_state_max0.03381657600402832
sim_compute_sim_state_mean0.03066845459513145
sim_compute_sim_state_median0.03066845459513145
sim_compute_sim_state_min0.027520333186234577
sim_render-ego0_max0.003916349618331246
sim_render-ego0_mean0.0037572200319862447
sim_render-ego0_median0.0037572200319862447
sim_render-ego0_min0.003598090445641244
simulation-passed1
step_physics_max0.07587162197226345
step_physics_mean0.07497568437881733
step_physics_median0.07497568437881733
step_physics_min0.0740797467853712
survival_time_max25.200000000000223
survival_time_mean18.325000000000124
survival_time_min11.450000000000028
No reset possible
7981616103Asoka Vardhan Sexercises_braitenbergmooc-BV1sim-2of5successnogpu-production-spot-0-020:06:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean2.1156000000001285


other stats
agent_compute-ego0_max0.011795148232944074
agent_compute-ego0_mean0.011795148232944074
agent_compute-ego0_median0.011795148232944074
agent_compute-ego0_min0.011795148232944074
complete-iteration_max0.2217661268609426
complete-iteration_mean0.2217661268609426
complete-iteration_median0.2217661268609426
complete-iteration_min0.2217661268609426
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max2.1156000000001285
distance-from-start_median2.1156000000001285
distance-from-start_min2.1156000000001285
driven_any_max2.115600000000152
driven_any_mean2.115600000000152
driven_any_median2.115600000000152
driven_any_min2.115600000000152
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.08451704211543318
get_duckie_state_mean0.08451704211543318
get_duckie_state_median0.08451704211543318
get_duckie_state_min0.08451704211543318
get_robot_state_max0.003926737272427859
get_robot_state_mean0.003926737272427859
get_robot_state_median0.003926737272427859
get_robot_state_min0.003926737272427859
get_state_dump_max0.018047684980389833
get_state_dump_mean0.018047684980389833
get_state_dump_median0.018047684980389833
get_state_dump_min0.018047684980389833
get_ui_image_max0.015405578167271254
get_ui_image_mean0.015405578167271254
get_ui_image_median0.015405578167271254
get_ui_image_min0.015405578167271254
in-drivable-lane_max36.300000000000075
in-drivable-lane_mean36.300000000000075
in-drivable-lane_median36.300000000000075
in-drivable-lane_min36.300000000000075
per-episodes
details{"d40-ego0": {"driven_any": 2.115600000000152, "get_ui_image": 0.015405578167271254, "step_physics": 0.06941108788879899, "survival_time": 36.300000000000075, "driven_lanedir": 0.0, "get_state_dump": 0.018047684980389833, "get_robot_state": 0.003926737272427859, "sim_render-ego0": 0.003730628316635279, "get_duckie_state": 0.08451704211543318, "in-drivable-lane": 36.300000000000075, "deviation-heading": 0.0, "agent_compute-ego0": 0.011795148232944074, "complete-iteration": 0.2217661268609426, "set_robot_commands": 0.002426303564436334, "distance-from-start": 2.1156000000001285, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.01040982150637956, "sim_compute_performance-ego0": 0.0020040157244654764}}
set_robot_commands_max0.002426303564436334
set_robot_commands_mean0.002426303564436334
set_robot_commands_median0.002426303564436334
set_robot_commands_min0.002426303564436334
sim_compute_performance-ego0_max0.0020040157244654764
sim_compute_performance-ego0_mean0.0020040157244654764
sim_compute_performance-ego0_median0.0020040157244654764
sim_compute_performance-ego0_min0.0020040157244654764
sim_compute_sim_state_max0.01040982150637956
sim_compute_sim_state_mean0.01040982150637956
sim_compute_sim_state_median0.01040982150637956
sim_compute_sim_state_min0.01040982150637956
sim_render-ego0_max0.003730628316635279
sim_render-ego0_mean0.003730628316635279
sim_render-ego0_median0.003730628316635279
sim_render-ego0_min0.003730628316635279
simulation-passed1
step_physics_max0.06941108788879899
step_physics_mean0.06941108788879899
step_physics_median0.06941108788879899
step_physics_min0.06941108788879899
survival_time_max36.300000000000075
survival_time_mean36.300000000000075
survival_time_median36.300000000000075
survival_time_min36.300000000000075
No reset possible
7977916121Lucas Hagelobjdet exercisemooc-objdetsimsuccessnogpu-production-spot-0-020:14:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median42.59999999999947
driven_lanedir_consec_median3.9495111712571753


other stats
agent_compute-ego0_max0.02028400137660704
agent_compute-ego0_mean0.0199086671761956
agent_compute-ego0_median0.0199086671761956
agent_compute-ego0_min0.019533332975784152
complete-iteration_max0.19537077279610995
complete-iteration_mean0.19070036953747077
complete-iteration_median0.19070036953747077
complete-iteration_min0.18602996627883156
deviation-center-line_max1.4153832435607612
deviation-center-line_mean0.7895372300147867
deviation-center-line_median0.7895372300147867
deviation-center-line_min0.16369121646881216
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max6.737558440716519
distance-from-start_mean3.9494919989265087
distance-from-start_median3.9494919989265087
distance-from-start_min1.1614255571364986
driven_any_max6.737558440716519
driven_any_mean3.9494919989265087
driven_any_median3.9494919989265087
driven_any_min1.1614255571364986
driven_lanedir_consec_max6.737628467702284
driven_lanedir_consec_mean3.9495111712571753
driven_lanedir_consec_min1.161393874812067
driven_lanedir_max6.737628467702284
driven_lanedir_mean3.9495111712571753
driven_lanedir_median3.9495111712571753
driven_lanedir_min1.161393874812067
get_duckie_state_max0.02113814397616549
get_duckie_state_mean0.021029190244075673
get_duckie_state_median0.021029190244075673
get_duckie_state_min0.020920236511985855
get_robot_state_max0.003757254467915734
get_robot_state_mean0.003749726328310967
get_robot_state_median0.003749726328310967
get_robot_state_min0.0037421981887062
get_state_dump_max0.008045242390565133
get_state_dump_mean0.008023720479615144
get_state_dump_median0.008023720479615144
get_state_dump_min0.008002198568665155
get_ui_image_max0.030740489109908
get_ui_image_mean0.030693803270215286
get_ui_image_median0.030693803270215286
get_ui_image_min0.030647117430522577
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_median0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-long-loop-with-duckies-000-ego0": {"driven_any": 6.737558440716519, "get_ui_image": 0.030740489109908, "step_physics": 0.06691647567371331, "survival_time": 25.200000000000223, "driven_lanedir": 6.737628467702284, "get_state_dump": 0.008002198568665155, "get_robot_state": 0.0037421981887062, "sim_render-ego0": 0.003692755368676516, "get_duckie_state": 0.020920236511985855, "in-drivable-lane": 0.0, "deviation-heading": 0.0, "agent_compute-ego0": 0.019533332975784152, "complete-iteration": 0.18602996627883156, "set_robot_commands": 0.0022944998032975905, "distance-from-start": 6.737558440716519, "deviation-center-line": 0.16369121646881216, "driven_lanedir_consec": 6.737628467702284, "sim_compute_sim_state": 0.028112882670789666, "sim_compute_performance-ego0": 0.001991995726481523}, "LF-long-loop-with-duckies-001-ego0": {"driven_any": 1.1614255571364986, "get_ui_image": 0.030647117430522577, "step_physics": 0.06805453193277046, "survival_time": 59.99999999999873, "driven_lanedir": 1.161393874812067, "get_state_dump": 0.008045242390565133, "get_robot_state": 0.003757254467915734, "sim_render-ego0": 0.003814264499178338, "get_duckie_state": 0.02113814397616549, "in-drivable-lane": 0.0, "deviation-heading": 0.0, "agent_compute-ego0": 0.02028400137660704, "complete-iteration": 0.19537077279610995, "set_robot_commands": 0.0023738853540349065, "distance-from-start": 1.1614255571364986, "deviation-center-line": 1.4153832435607612, "driven_lanedir_consec": 1.161393874812067, "sim_compute_sim_state": 0.03512309056138317, "sim_compute_performance-ego0": 0.0020482254266540375}}
set_robot_commands_max0.0023738853540349065
set_robot_commands_mean0.0023341925786662483
set_robot_commands_median0.0023341925786662483
set_robot_commands_min0.0022944998032975905
sim_compute_performance-ego0_max0.0020482254266540375
sim_compute_performance-ego0_mean0.00202011057656778
sim_compute_performance-ego0_median0.00202011057656778
sim_compute_performance-ego0_min0.001991995726481523
sim_compute_sim_state_max0.03512309056138317
sim_compute_sim_state_mean0.03161798661608642
sim_compute_sim_state_median0.03161798661608642
sim_compute_sim_state_min0.028112882670789666
sim_render-ego0_max0.003814264499178338
sim_render-ego0_mean0.003753509933927427
sim_render-ego0_median0.003753509933927427
sim_render-ego0_min0.003692755368676516
simulation-passed1
step_physics_max0.06805453193277046
step_physics_mean0.06748550380324189
step_physics_median0.06748550380324189
step_physics_min0.06691647567371331
survival_time_max59.99999999999873
survival_time_mean42.59999999999947
survival_time_min25.200000000000223
No reset possible
7976216126Arwa Alabdulkarimexercises_braitenbergmooc-BV1sim-3of5successnogpu-production-spot-0-020:12:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean4.315629317619417


other stats
agent_compute-ego0_max0.011734661710550149
agent_compute-ego0_mean0.011734661710550149
agent_compute-ego0_median0.011734661710550149
agent_compute-ego0_min0.011734661710550149
complete-iteration_max0.19100533853859628
complete-iteration_mean0.19100533853859628
complete-iteration_median0.19100533853859628
complete-iteration_min0.19100533853859628
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max4.315629317619417
distance-from-start_median4.315629317619417
distance-from-start_min4.315629317619417
driven_any_max5.824709055384445
driven_any_mean5.824709055384445
driven_any_median5.824709055384445
driven_any_min5.824709055384445
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.06266814862361657
get_duckie_state_mean0.06266814862361657
get_duckie_state_median0.06266814862361657
get_duckie_state_min0.06266814862361657
get_robot_state_max0.0038509841366274767
get_robot_state_mean0.0038509841366274767
get_robot_state_median0.0038509841366274767
get_robot_state_min0.0038509841366274767
get_state_dump_max0.014730335969313496
get_state_dump_mean0.014730335969313496
get_state_dump_median0.014730335969313496
get_state_dump_min0.014730335969313496
get_ui_image_max0.011442641235211808
get_ui_image_mean0.011442641235211808
get_ui_image_median0.011442641235211808
get_ui_image_min0.011442641235211808
in-drivable-lane_max59.99999999999873
in-drivable-lane_mean59.99999999999873
in-drivable-lane_median59.99999999999873
in-drivable-lane_min59.99999999999873
per-episodes
details{"d30-ego0": {"driven_any": 5.824709055384445, "get_ui_image": 0.011442641235211808, "step_physics": 0.06818932756396952, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.014730335969313496, "get_robot_state": 0.0038509841366274767, "sim_render-ego0": 0.0037405973270870463, "get_duckie_state": 0.06266814862361657, "in-drivable-lane": 59.99999999999873, "deviation-heading": 0.0, "agent_compute-ego0": 0.011734661710550149, "complete-iteration": 0.19100533853859628, "set_robot_commands": 0.0023105666599702477, "distance-from-start": 4.315629317619417, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.010250049070156584, "sim_compute_performance-ego0": 0.0019914081551252456}}
set_robot_commands_max0.0023105666599702477
set_robot_commands_mean0.0023105666599702477
set_robot_commands_median0.0023105666599702477
set_robot_commands_min0.0023105666599702477
sim_compute_performance-ego0_max0.0019914081551252456
sim_compute_performance-ego0_mean0.0019914081551252456
sim_compute_performance-ego0_median0.0019914081551252456
sim_compute_performance-ego0_min0.0019914081551252456
sim_compute_sim_state_max0.010250049070156584
sim_compute_sim_state_mean0.010250049070156584
sim_compute_sim_state_median0.010250049070156584
sim_compute_sim_state_min0.010250049070156584
sim_render-ego0_max0.0037405973270870463
sim_render-ego0_mean0.0037405973270870463
sim_render-ego0_median0.0037405973270870463
sim_render-ego0_min0.0037405973270870463
simulation-passed1
step_physics_max0.06818932756396952
step_physics_mean0.06818932756396952
step_physics_median0.06818932756396952
step_physics_min0.06818932756396952
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_median59.99999999999873
survival_time_min59.99999999999873
No reset possible