Duckietown Challenges Home Challenges Submissions

Evaluator 1464

ID1464
evaluatorp-2312
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processp-2312
versiond-c:4.0.30;d-c-r:4.0.47;d-s:4.0.22
first heard
last heard
statusinactive
# evaluating
# success2 23809
# timeout
# failed
# error3 23879
# aborted2 23772
# host-error1 23878
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
239953788Andrés Calderón 🇨🇴challenge-aido_LF-template-randomaido2-LFVI-sim-validation-offseasonstep1-simulationsuccessyesp-23120:07:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.493517638549009
survival_time_median2.3999999999999995
deviation-center-line_median0.09791133281082
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.14802444548833937
agent_compute-ego_mean0.14435878529275006
agent_compute-ego_median0.14394319208362436
agent_compute-ego_min0.14151752988497415
deviation-center-line_max0.2348196785686717
deviation-center-line_mean0.13547481992266058
deviation-center-line_min0.06216029860587694
deviation-heading_max0.8115272458314897
deviation-heading_mean0.49959867397150787
deviation-heading_median0.46504862964833654
deviation-heading_min0.30044835691402494
driven_any_max2.1864003300000694
driven_any_mean1.1068208197848532
driven_any_median0.9410513078477724
driven_any_min0.34749292376309954
driven_lanedir_consec_max1.826231945363564
driven_lanedir_consec_mean0.804744013449103
driven_lanedir_consec_min0.3325249708009643
driven_lanedir_max1.826231945363564
driven_lanedir_mean0.804828147320482
driven_lanedir_median0.493517638549009
driven_lanedir_min0.3325249708009643
in-drivable-lane_max1.3499999999999952
in-drivable-lane_mean0.5599999999999984
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.12303505366361596, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.06191552138026757, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.14394319208362436, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.07793401464631286, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.03503408311288568, "sim_compute_performance-ego": 0.06727266613441178, "sim_compute_robot_state-ego": 0.06306363661077959, "sim_compute_robot_state-npc0": 0.0690449430972715, "sim_compute_robot_state-npc1": 0.06534142735638196, "sim_compute_robot_state-npc2": 0.06333122675931907, "sim_compute_robot_state-npc3": 0.06334988074966624}, "udem1-1-0": {"driven_any": 0.47798123795353353, "sim_physics": 0.11886134317943028, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.06287203516278948, "in-drivable-lane": 0, "agent_compute-ego": 0.14456929479326522, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.07827351774488177, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.03498366049357823, "sim_compute_performance-ego": 0.0657938803945269, "sim_compute_robot_state-ego": 0.06284742695944649, "sim_compute_robot_state-npc0": 0.06998463187898908, "sim_compute_robot_state-npc1": 0.0657945019858224, "sim_compute_robot_state-npc2": 0.06276618582861763, "sim_compute_robot_state-npc3": 0.06157158953802926}, "udem1-2-0": {"driven_any": 2.1864003300000694, "sim_physics": 0.12377363269768872, "survival_time": 5.14999999999999, "driven_lanedir": 1.826231945363564, "sim_render-ego": 0.05881834956048762, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.1437394642135472, "deviation-heading": 0.8115272458314897, "set_robot_commands": 0.0801571109919872, "deviation-center-line": 0.1927316041609401, "driven_lanedir_consec": 1.826231945363564, "sim_compute_sim_state": 0.0352509230085947, "sim_compute_performance-ego": 0.06283545725553938, "sim_compute_robot_state-ego": 0.0640907681104049, "sim_compute_robot_state-npc0": 0.06808508715583282, "sim_compute_robot_state-npc1": 0.06510052634674368, "sim_compute_robot_state-npc2": 0.0629064805299333, "sim_compute_robot_state-npc3": 0.06249488673163849}, "udem1-3-0": {"driven_any": 0.9410513078477724, "sim_physics": 0.1527865727742513, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.055143311619758606, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.14151752988497415, "deviation-heading": 0.35154027506548424, "set_robot_commands": 0.08381109933058421, "deviation-center-line": 0.08975118546699407, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.03454114496707916, "sim_compute_performance-ego": 0.06274747848510742, "sim_compute_robot_state-ego": 0.06579576432704926, "sim_compute_robot_state-npc0": 0.06290708482265472, "sim_compute_robot_state-npc1": 0.061344568928082786, "sim_compute_robot_state-npc2": 0.06083444754282633, "sim_compute_robot_state-npc3": 0.061719988783200584}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.12453944342476982, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.06315420922778901, "in-drivable-lane": 0, "agent_compute-ego": 0.14802444548833937, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.07710511343819755, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.03647828102111816, "sim_compute_performance-ego": 0.06673861685253325, "sim_compute_robot_state-ego": 0.0644799414135161, "sim_compute_robot_state-npc0": 0.07183331534976051, "sim_compute_robot_state-npc1": 0.06828079904828753, "sim_compute_robot_state-npc2": 0.06567189806983584, "sim_compute_robot_state-npc3": 0.06588890438988096}}
set_robot_commands_max0.08381109933058421
set_robot_commands_mean0.07945617123039271
set_robot_commands_median0.07827351774488177
set_robot_commands_min0.07710511343819755
sim_compute_performance-ego_max0.06727266613441178
sim_compute_performance-ego_mean0.06507761982442375
sim_compute_performance-ego_median0.0657938803945269
sim_compute_performance-ego_min0.06274747848510742
sim_compute_robot_state-ego_max0.06579576432704926
sim_compute_robot_state-ego_mean0.06405550748423927
sim_compute_robot_state-ego_median0.0640907681104049
sim_compute_robot_state-ego_min0.06284742695944649
sim_compute_robot_state-npc0_max0.07183331534976051
sim_compute_robot_state-npc0_mean0.06837101246090173
sim_compute_robot_state-npc0_median0.0690449430972715
sim_compute_robot_state-npc0_min0.06290708482265472
sim_compute_robot_state-npc1_max0.06828079904828753
sim_compute_robot_state-npc1_mean0.06517236473306368
sim_compute_robot_state-npc1_median0.06534142735638196
sim_compute_robot_state-npc1_min0.061344568928082786
sim_compute_robot_state-npc2_max0.06567189806983584
sim_compute_robot_state-npc2_mean0.06310204774610643
sim_compute_robot_state-npc2_median0.0629064805299333
sim_compute_robot_state-npc2_min0.06083444754282633
sim_compute_robot_state-npc3_max0.06588890438988096
sim_compute_robot_state-npc3_mean0.06300505003848311
sim_compute_robot_state-npc3_median0.06249488673163849
sim_compute_robot_state-npc3_min0.06157158953802926
sim_compute_sim_state_max0.03647828102111816
sim_compute_sim_state_mean0.035257618520651186
sim_compute_sim_state_median0.03503408311288568
sim_compute_sim_state_min0.03454114496707916
sim_physics_max0.1527865727742513
sim_physics_mean0.12859920914795123
sim_physics_median0.12377363269768872
sim_physics_min0.11886134317943028
sim_render-ego_max0.06315420922778901
sim_render-ego_mean0.06038068539021846
sim_render-ego_median0.06191552138026757
sim_render-ego_min0.055143311619758606
simulation-passed1
survival_time_max5.14999999999999
survival_time_mean2.789999999999997
survival_time_min1.0500000000000005
No reset possible
239683767Anastasiya Nikolskaya 🇷🇺Boxed Solution 0.2aido2-LFVI-real-validationeval0-videos-autobotserroryesp-23123:00:30
Waited 10801 seconds [...]
Waited 10801 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
239413775Anastasiya Nikolskaya 🇷🇺Offset-based Oneaido2-LFV-real-validationeval0-videos-watchtowerserroryesp-23123:00:32
Waited 10802 seconds [...]
Waited 10802 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238793772Anastasiya Nikolskaya 🇷🇺Fine-tuned netaido2-LFV-real-validationeval0-videos-watchtowerserroryesp-23123:00:36
Waited 10800 seconds [...]
Waited 10800 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238783701jiang pengrobotarium test- cropaido2-LF-real-validationeval0-videos-autobotshost-erroryesp-23120:01:06
Could not access IPF [...]
Could not access IPFS data:
Traceback (most recent call last):
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 1184, in download_artefacts
    try_to_get_hash(client, sha256hex)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/timeout_decorator/timeout_decorator.py", line 81, in new_function
    return function(*args, **kwargs)
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 1133, in try_to_get_hash
    res = client.file_ls(qm)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/ipfsapi-0.4.4-py3.7.egg/ipfsapi/client/__init__.py", line 205, in wrapper
    return value(*args, **kwargs)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/ipfsapi-0.4.4-py3.7.egg/ipfsapi/client/__init__.py", line 242, in file_ls
    return self._client.request('/file/ls', args, decoder='json', **kwargs)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/ipfshttpclient-0.4.11-py3.7.egg/ipfshttpclient/http.py", line 44, in wrapper
    return func(self, *args, **merged)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/ipfshttpclient-0.4.11-py3.7.egg/ipfshttpclient/http.py", line 364, in request
    files, headers, data, timeout=timeout)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/ipfshttpclient-0.4.11-py3.7.egg/ipfshttpclient/http.py", line 282, in _request
    timeout=timeout)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/ipfshttpclient-0.4.11-py3.7.egg/ipfshttpclient/http.py", line 245, in _do_request
    return requests.request(*args, **kwargs)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/ipfshttpclient-0.4.11-py3.7.egg/ipfshttpclient/requests_wrapper.py", line 240, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/ipfshttpclient-0.4.11-py3.7.egg/ipfshttpclient/requests_wrapper.py", line 219, in request
    return super(Session, self).request(method, url, *args, **kwargs)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/home/linuxbrew/.linuxbrew/opt/python/lib/python3.7/http/client.py", line 1321, in getresponse
    response.begin()
  File "/home/linuxbrew/.linuxbrew/opt/python/lib/python3.7/http/client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "/home/linuxbrew/.linuxbrew/opt/python/lib/python3.7/http/client.py", line 257, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/home/linuxbrew/.linuxbrew/opt/python/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/timeout_decorator/timeout_decorator.py", line 72, in handler
    _raise_exception(timeout_exception, exception_message)
  File "/home/andrea/.local/share/virtualenvs/dt-env-developer-q48V9oeQ/lib/python3.7/site-packages/timeout_decorator/timeout_decorator.py", line 45, in _raise_exception
    raise exception()
timeout_decorator.timeout_decorator.TimeoutError: 'Timed Out'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 541, in get_cr
    allow_host_network=allow_host_network, use_ipfs=use_ipfs)
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 605, in run_single
    prepare_dir(wd, aws_config, steps2artefacts, use_ipfs=use_ipfs)
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 877, in prepare_dir
    download_artefacts(aws_config, steps2artefacts, previous_steps_dir, use_ipfs=use_ipfs)
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 1188, in download_artefacts
    raise IPFSException(msg)
duckietown_challenges_runner.runner.IPFSException: Could not get IPFS hash QmWdmWKpAqQpvsNPJiKNAbNkpo5iS8iKpmRA8DtA1ZRPp3
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238093772Anastasiya Nikolskaya 🇷🇺Fine-tuned netaido2-LFV-real-validationeval0-videos-autobotssuccessyesp-23122:17:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
237873761Julian ZillyBaseline-IL-logs-tensorflowaido2-LFVI-real-validationeval0-videos-autobotsabortedyesp-23121:16:56
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 820, in run_one
    time.sleep(5)
KeyboardInterrupt
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
237723754Anastasiya Nikolskaya 🇷🇺robotariumaido2-LF-real-validationeval0-videos-autobotsabortedyesp-23122:42:14
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/home/andrea/dt-env-developer/src/duckietown-challenges-runner/src/duckietown_challenges_runner/runner.py", line 820, in run_one
    time.sleep(5)
KeyboardInterrupt
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible