Duckietown Challenges Home Challenges Submissions

Evaluator 5447

ID5447
evaluatornogpu-production-b-spot-1-01
ownerI don't have one 😀
machinenogpu-production-b-spot-1
processnogpu-production-b-spot-1-01
version6.2.32
first heard
last heard
statusinactive
# evaluating
# success14 84104
# timeout
# failed
# error
# aborted2 84150
# host-error
arm0
x86_641
Mac0
gpu available0
Number of processors96
Processor frequency (MHz)0.0 GHz
Free % of processors100%
RAM total (MB)184.7 GB
RAM free (MB)177.0 GB
Disk (MB)969.3 GB
Disk available (MB)946.0 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
8450916778Lin Wei-Chihcollision-check-exercisemooc-collision-check-valistep1successyesnogpu-production-b-spot-1-010:00:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
false_negative_rate0.0
false_positive_rate1.0


other stats
nsamples200
No reset possible
8432216658Bea Baselines 🐤baseline-solutionmooc-planning-cc-static-valistep1-0of4successyesnogpu-production-b-spot-1-010:05:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes1.0
success_ratio1.0
duration35.8793189266521
complexity111.0
avg_min_distance0.20324295479561583


other stats
nfeasible6
nsamples6
No reset possible
8430516661Bea Baselines 🐤baseline-solutionmooc-planning-cc-dynamic-teststep1-3of4successyesnogpu-production-b-spot-1-010:06:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes1.0
success_ratio1.0
duration33.617916094293854
complexity98.83333333333331
avg_min_distance0.0


other stats
nfeasible6
nsamples6
No reset possible
8429416653Bea Baselines 🐤baseline-solutionmooc-planning-dd-static-teststep1-2of4successyesnogpu-production-b-spot-1-010:04:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes0.0
success_ratio1.0
duration28.117412122371803
complexity87.16666666666667
avg_min_distance0.2254548034823704


other stats
nfeasible6
nsamples6
No reset possible
8424816015Dimitrios Dimopouloscollision-check-exercisemooc-collision-check-teststep1successyesnogpu-production-b-spot-1-010:00:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
false_negative_rate0.0
false_positive_rate0.12


other stats
nsamples200
No reset possible
8424216015Dimitrios Dimopouloscollision-check-exercisemooc-collision-check-teststep1successyesnogpu-production-b-spot-1-010:00:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
false_negative_rate0.0
false_positive_rate0.12


other stats
nsamples200
No reset possible
8422916580Bea Baselines 🐤planning-exercisemooc-planning-dd-static-teststep1-1of4successyesnogpu-production-b-spot-1-010:02:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes1.0
success_ratio1.0
duration22.0
complexity8.0
avg_min_distance0.0


other stats
nfeasible6
nsamples6
No reset possible
8421716581Bea Baselines 🐤planning-exercisemooc-planning-dd-dynamic-valistep1-1of4successyesnogpu-production-b-spot-1-010:03:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes1.0
success_ratio1.0
duration22.0
complexity8.0
avg_min_distance0.0


other stats
nfeasible6
nsamples6
No reset possible
8420416587Bea Baselines 🐤planning-exercisemooc-planning-cc-dynamic-valistep1-2of4successyesnogpu-production-b-spot-1-010:02:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes1.0
success_ratio1.0
duration22.0
complexity8.0
avg_min_distance0.003440326088530902


other stats
nfeasible6
nsamples6
No reset possible
8418516591Bea Baselines 🐤baseline-solutionmooc-planning-dd-static-valistep1-1of4successyesnogpu-production-b-spot-1-010:05:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes0.0
success_ratio1.0
duration37.97648914360376
complexity111.5
avg_min_distance0.18912656072552125


other stats
nfeasible6
nsamples6
No reset possible
8417516592Bea Baselines 🐤baseline-solutionmooc-planning-dd-static-teststep1-2of4successyesnogpu-production-b-spot-1-010:04:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes0.0
success_ratio1.0
duration28.117412122371803
complexity87.16666666666667
avg_min_distance0.2254548034823704


other stats
nfeasible6
nsamples6
No reset possible
8417116588Bea Baselines 🐤planning-exercisemooc-planning-cc-dynamic-teststep1-3of4successyesnogpu-production-b-spot-1-010:03:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
mistakes1.0
success_ratio1.0
duration22.0
complexity8.0
avg_min_distance0.0215035653855344


other stats
nfeasible6
nsamples6
No reset possible
8415313798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortedyesnogpu-production-b-spot-1-010:00:23
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=2021_03_25_17_23_41&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 93, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 430, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=2021_03_25_17_23_41&fromImage=docker.io%2Fnitaigao%2Faido-submissions: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 112, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  2021_03_25_17_23_41

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 51, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull nitaigao/aido-submissions:2021_03_25_17_23_41@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8415013798Nicholas Kostelniktemplate-randomaido-hello-sim-validation370abortedyesnogpu-production-b-spot-1-010:00:23
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=2021_03_25_17_23_41&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 93, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 430, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=2021_03_25_17_23_41&fromImage=docker.io%2Fnitaigao%2Faido-submissions: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 47, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet, credentials=credentials)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 112, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  2021_03_25_17_23_41

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 836, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1038, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5, credentials=credentials, quiet=False)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 51, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull nitaigao/aido-submissions:2021_03_25_17_23_41@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8412915872Bea Baselines 🐤dummy_collision_checkermooc-collision-check-valistep1successyesnogpu-production-b-spot-1-010:00:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
false_negative_rate0.54
false_positive_rate0.47


other stats
nsamples200
No reset possible
8410415872Bea Baselines 🐤dummy_collision_checkermooc-collision-check-valistep1successyesnogpu-production-b-spot-1-010:01:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
false_negative_rate0.45
false_positive_rate0.48


other stats
nsamples200
No reset possible