Duckietown Challenges Home Challenges Submissions

Evaluator 5356

ID5356
evaluatorgpu-production-spot-0-01
ownerI don't have one 😀
machinegpu-production-spot-0_8203b861464a
processgpu-production-spot-0-01_8203b861464a
version6.2.7
first heard
last heard
statusinactive
# evaluating
# success1 79725
# timeout
# failed
# error
# aborted4 79735
# host-error
arm0
x86_641
Mac0
gpu available1
Number of processors64
Processor frequency (MHz)0.0 GHz
Free % of processors100%
RAM total (MB)249.0 GB
RAM free (MB)245.9 GB
Disk (MB)969.3 GB
Disk available (MB)876.3 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
7973813798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortedyesgpu-production-spot-0-010:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7973713798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortedyesgpu-production-spot-0-010:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7973613798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortedyesgpu-production-spot-0-010:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7973513798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortedyesgpu-production-spot-0-010:00:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7972516095SEYEONG JANGbase-image-mlmooc-visservoingsimsuccessnogpu-production-spot-0-010:17:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
in-drivable-lane_median4.449999999999795
deviation-center-line_median2.2153112723672357
driven_lanedir_consec_median5.631945338468869
survival_time_median59.99999999999873


other stats
agent_compute-ego0_max0.006566325988896582
agent_compute-ego0_mean0.006543112138625883
agent_compute-ego0_median0.006543112138625883
agent_compute-ego0_min0.006519898288355183
complete-iteration_max0.12130490866032966
complete-iteration_mean0.12047055371099467
complete-iteration_median0.12047055371099467
complete-iteration_min0.11963619876165968
deviation-center-line_max3.2584365991730437
deviation-center-line_mean2.2153112723672357
deviation-center-line_min1.1721859455614276
deviation-heading_max9.243866930134278
deviation-heading_mean7.916238713245321
deviation-heading_median7.916238713245321
deviation-heading_min6.588610496356365
distance-from-start_max1.672298267772573
distance-from-start_mean1.4128921939057837
distance-from-start_median1.4128921939057837
distance-from-start_min1.153486120038994
driven_any_max6.248350473834752
driven_any_mean6.2456765298963575
driven_any_median6.2456765298963575
driven_any_min6.243002585957963
driven_lanedir_consec_max6.133244584399884
driven_lanedir_consec_mean5.631945338468869
driven_lanedir_consec_min5.130646092537854
driven_lanedir_max6.133244584399884
driven_lanedir_mean5.631945338468869
driven_lanedir_median5.631945338468869
driven_lanedir_min5.130646092537854
get_duckie_state_max1.2790432182775746e-06
get_duckie_state_mean1.2622685555514449e-06
get_duckie_state_median1.2622685555514449e-06
get_duckie_state_min1.2454938928253149e-06
get_robot_state_max0.003607227641478069
get_robot_state_mean0.003571350806757969
get_robot_state_median0.003571350806757969
get_robot_state_min0.003535473972037869
get_state_dump_max0.004656748608883771
get_state_dump_mean0.004598536757406446
get_state_dump_median0.004598536757406446
get_state_dump_min0.004540324905929121
get_ui_image_max0.028361168034765544
get_ui_image_mean0.0283551963540934
get_ui_image_median0.0283551963540934
get_ui_image_min0.02834922467342126
in-drivable-lane_max8.89999999999959
in-drivable-lane_mean4.449999999999795
in-drivable-lane_min0.0
per-episodes
details{"LF-small-loop-000-ego0": {"driven_any": 6.243002585957963, "get_ui_image": 0.028361168034765544, "step_physics": 0.06646830612773402, "survival_time": 59.99999999999873, "driven_lanedir": 6.133244584399884, "get_state_dump": 0.004656748608883771, "get_robot_state": 0.003607227641478069, "sim_render-ego0": 0.003569369907680896, "get_duckie_state": 1.2454938928253149e-06, "in-drivable-lane": 0.0, "deviation-heading": 6.588610496356365, "agent_compute-ego0": 0.006566325988896582, "complete-iteration": 0.12130490866032966, "set_robot_commands": 0.0023073163457357515, "distance-from-start": 1.153486120038994, "deviation-center-line": 1.1721859455614276, "driven_lanedir_consec": 6.133244584399884, "sim_compute_sim_state": 0.00379981268057716, "sim_compute_performance-ego0": 0.0018888381398984731}, "LF-small-loop-001-ego0": {"driven_any": 6.248350473834752, "get_ui_image": 0.02834922467342126, "step_physics": 0.06523673480793797, "survival_time": 59.99999999999873, "driven_lanedir": 5.130646092537854, "get_state_dump": 0.004540324905929121, "get_robot_state": 0.003535473972037869, "sim_render-ego0": 0.003480455659013505, "get_duckie_state": 1.2790432182775746e-06, "in-drivable-lane": 8.89999999999959, "deviation-heading": 9.243866930134278, "agent_compute-ego0": 0.006519898288355183, "complete-iteration": 0.11963619876165968, "set_robot_commands": 0.0022883097595418123, "distance-from-start": 1.672298267772573, "deviation-center-line": 3.2584365991730437, "driven_lanedir_consec": 5.130646092537854, "sim_compute_sim_state": 0.0037474540945493014, "sim_compute_performance-ego0": 0.0018605050397455245}}
set_robot_commands_max0.0023073163457357515
set_robot_commands_mean0.002297813052638782
set_robot_commands_median0.002297813052638782
set_robot_commands_min0.0022883097595418123
sim_compute_performance-ego0_max0.0018888381398984731
sim_compute_performance-ego0_mean0.001874671589821999
sim_compute_performance-ego0_median0.001874671589821999
sim_compute_performance-ego0_min0.0018605050397455245
sim_compute_sim_state_max0.00379981268057716
sim_compute_sim_state_mean0.003773633387563231
sim_compute_sim_state_median0.003773633387563231
sim_compute_sim_state_min0.0037474540945493014
sim_render-ego0_max0.003569369907680896
sim_render-ego0_mean0.0035249127833472003
sim_render-ego0_median0.0035249127833472003
sim_render-ego0_min0.003480455659013505
simulation-passed1
step_physics_max0.06646830612773402
step_physics_mean0.06585252046783599
step_physics_median0.06585252046783599
step_physics_min0.06523673480793797
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible