Duckietown Challenges Home Challenges Submissions

Evaluator 692

ID692
evaluatoridsc-rudolf-4752
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-4752
versiond-c:4.0.16;d-c-r:4.0.22;d-s:4.0.16
first heard
last heard
statusinactive
# evaluating
# success2 17721
# timeout
# failed
# error2 17732
# aborted
# host-error1 17715
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
177372540jiang pengtest for ppoaido2-LF-sim-testingstep1-simulationerrornoidsc-rudolf-47520:02:09
The container "solut [...]
The container "solution" exited with code 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
177362539Nikhil Khedekar 🇮🇳PredictorLastaido2-PREDstep1-simulationsuccessnoidsc-rudolf-47520:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.10321746472536174
error_L20.01704464565093361


No reset possible
177322517Liam Paull 🇨🇦challenge-aido_LF-baseline-duckietownaido2-LF-sim-validationstep1-simulationerrornoidsc-rudolf-47520:00:49
The container "solut [...]
The container "solution" exited with code 1.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
177212512Rami Al-Naim 🇷🇺challenge-aido_LF-template-rosaido2-LFVI-sim-testingstep1-simulationsuccessnoidsc-rudolf-47520:05:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.2448526305955765
survival_time_median1.0000000000000002
deviation-center-line_median0.02780956837444773
in-drivable-lane_median0.4500000000000001


other stats
agent_compute-ego_max0.1661499919313373
agent_compute-ego_mean0.15580808846495792
agent_compute-ego_median0.1566259264945984
agent_compute-ego_min0.14406946126152487
deviation-center-line_max0.1257230388091751
deviation-center-line_mean0.05577343615770758
deviation-center-line_min0.01951527185289407
deviation-heading_max0.5595345336423936
deviation-heading_mean0.32175424111925316
deviation-heading_median0.3074760381865353
deviation-heading_min0.16993642764665495
driven_any_max1.135775595781212
driven_any_mean0.7105521201006961
driven_any_median0.5457640496143795
driven_any_min0.3022738981004901
driven_lanedir_consec_max0.8171012478585276
driven_lanedir_consec_mean0.42515260445919045
driven_lanedir_consec_min0.164471242446655
driven_lanedir_max0.8171012478585276
driven_lanedir_mean0.4357084916634508
driven_lanedir_median0.2448526305955765
driven_lanedir_min0.164471242446655
in-drivable-lane_max0.6500000000000002
in-drivable-lane_mean0.38000000000000017
in-drivable-lane_min0.04999999999999999
per-episodes
details{"udem1-0-0": {"driven_any": 0.5457640496143795, "sim_physics": 0.027394092082977297, "survival_time": 1.0000000000000002, "driven_lanedir": 0.164471242446655, "sim_render-ego": 0.056502246856689455, "in-drivable-lane": 0.6500000000000002, "agent_compute-ego": 0.1580904245376587, "deviation-heading": 0.18758204257854744, "set_robot_commands": 0.07442598342895508, "deviation-center-line": 0.01951527185289407, "driven_lanedir_consec": 0.164471242446655, "sim_compute_sim_state": 0.03270809650421143, "sim_compute_performance-ego": 0.05502525568008423, "sim_compute_robot_state-ego": 0.056818366050720215, "sim_compute_robot_state-npc0": 0.05750241279602051, "sim_compute_robot_state-npc1": 0.05586925745010376, "sim_compute_robot_state-npc2": 0.05502208471298218, "sim_compute_robot_state-npc3": 0.05872974395751953}, "udem1-1-0": {"driven_any": 0.5108934139126888, "sim_physics": 0.02805841670316808, "survival_time": 0.8500000000000002, "driven_lanedir": 0.18120319035776863, "sim_render-ego": 0.06587739551768583, "in-drivable-lane": 0.4500000000000001, "agent_compute-ego": 0.14406946126152487, "deviation-heading": 0.3074760381865353, "set_robot_commands": 0.0754852715660544, "deviation-center-line": 0.02399913959382396, "driven_lanedir_consec": 0.1725821469418083, "sim_compute_sim_state": 0.03871561499202952, "sim_compute_performance-ego": 0.06605329233057358, "sim_compute_robot_state-ego": 0.06489955677705653, "sim_compute_robot_state-npc0": 0.06816332480486702, "sim_compute_robot_state-npc1": 0.06623571059283088, "sim_compute_robot_state-npc2": 0.06532495162066292, "sim_compute_robot_state-npc3": 0.06496520603404325}, "udem1-2-0": {"driven_any": 0.3022738981004901, "sim_physics": 0.03189492225646973, "survival_time": 0.49999999999999994, "driven_lanedir": 0.2448526305955765, "sim_render-ego": 0.06653938293457032, "in-drivable-lane": 0.04999999999999999, "agent_compute-ego": 0.1541046380996704, "deviation-heading": 0.16993642764665495, "set_robot_commands": 0.07594146728515624, "deviation-center-line": 0.02780956837444773, "driven_lanedir_consec": 0.2448526305955765, "sim_compute_sim_state": 0.03568048477172851, "sim_compute_performance-ego": 0.06298599243164063, "sim_compute_robot_state-ego": 0.06199483871459961, "sim_compute_robot_state-npc0": 0.07135567665100098, "sim_compute_robot_state-npc1": 0.06958041191101075, "sim_compute_robot_state-npc2": 0.060581374168396, "sim_compute_robot_state-npc3": 0.05883467197418213}, "udem1-3-0": {"driven_any": 1.135775595781212, "sim_physics": 0.025498956441879272, "survival_time": 2.000000000000001, "driven_lanedir": 0.7709141470587263, "sim_render-ego": 0.06501345038414001, "in-drivable-lane": 0.6000000000000003, "agent_compute-ego": 0.1566259264945984, "deviation-heading": 0.3842421635421346, "set_robot_commands": 0.07974670529365539, "deviation-center-line": 0.08182016215819705, "driven_lanedir_consec": 0.7267557544533849, "sim_compute_sim_state": 0.03468817472457886, "sim_compute_performance-ego": 0.06366730332374573, "sim_compute_robot_state-ego": 0.06670652031898498, "sim_compute_robot_state-npc0": 0.07100902199745178, "sim_compute_robot_state-npc1": 0.06633205413818359, "sim_compute_robot_state-npc2": 0.06443021893501281, "sim_compute_robot_state-npc3": 0.06284072399139404}, "udem1-4-0": {"driven_any": 1.05805364309471, "sim_physics": 0.026046760154492928, "survival_time": 1.6500000000000008, "driven_lanedir": 0.8171012478585276, "sim_render-ego": 0.07291125528740161, "in-drivable-lane": 0.15000000000000013, "agent_compute-ego": 0.1661499919313373, "deviation-heading": 0.5595345336423936, "set_robot_commands": 0.07698026570406827, "deviation-center-line": 0.1257230388091751, "driven_lanedir_consec": 0.8171012478585276, "sim_compute_sim_state": 0.040777184746482155, "sim_compute_performance-ego": 0.0700921867832993, "sim_compute_robot_state-ego": 0.06733589461355498, "sim_compute_robot_state-npc0": 0.07834141904657538, "sim_compute_robot_state-npc1": 0.07316590077949292, "sim_compute_robot_state-npc2": 0.07274520035946008, "sim_compute_robot_state-npc3": 0.07075382723952785}}
set_robot_commands_max0.07974670529365539
set_robot_commands_mean0.07651593865557789
set_robot_commands_median0.07594146728515624
set_robot_commands_min0.07442598342895508
sim_compute_performance-ego_max0.0700921867832993
sim_compute_performance-ego_mean0.0635648061098687
sim_compute_performance-ego_median0.06366730332374573
sim_compute_performance-ego_min0.05502525568008423
sim_compute_robot_state-ego_max0.06733589461355498
sim_compute_robot_state-ego_mean0.06355103529498327
sim_compute_robot_state-ego_median0.06489955677705653
sim_compute_robot_state-ego_min0.056818366050720215
sim_compute_robot_state-npc0_max0.07834141904657538
sim_compute_robot_state-npc0_mean0.06927437105918313
sim_compute_robot_state-npc0_median0.07100902199745178
sim_compute_robot_state-npc0_min0.05750241279602051
sim_compute_robot_state-npc1_max0.07316590077949292
sim_compute_robot_state-npc1_mean0.06623666697432437
sim_compute_robot_state-npc1_median0.06633205413818359
sim_compute_robot_state-npc1_min0.05586925745010376
sim_compute_robot_state-npc2_max0.07274520035946008
sim_compute_robot_state-npc2_mean0.0636207659593028
sim_compute_robot_state-npc2_median0.06443021893501281
sim_compute_robot_state-npc2_min0.05502208471298218
sim_compute_robot_state-npc3_max0.07075382723952785
sim_compute_robot_state-npc3_mean0.06322483463933336
sim_compute_robot_state-npc3_median0.06284072399139404
sim_compute_robot_state-npc3_min0.05872974395751953
sim_compute_sim_state_max0.040777184746482155
sim_compute_sim_state_mean0.0365139111478061
sim_compute_sim_state_median0.03568048477172851
sim_compute_sim_state_min0.03270809650421143
sim_physics_max0.03189492225646973
sim_physics_mean0.02777862952779746
sim_physics_median0.027394092082977297
sim_physics_min0.025498956441879272
sim_render-ego_max0.07291125528740161
sim_render-ego_mean0.06536874619609745
sim_render-ego_median0.06587739551768583
sim_render-ego_min0.056502246856689455
simulation-passed1
survival_time_max2.000000000000001
survival_time_mean1.2000000000000004
survival_time_min0.49999999999999994
No reset possible
177152532Liam Paull 🇨🇦baseline-IL-sim-tensorflowaido2-LFV-sim-testingstep1-simulationhost-errornoidsc-rudolf-47520:02:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown-shell/lib/dt_shell/env_checks.py", line 60, in check_docker_environment
    containers = client.containers.list(filters=dict(status='running'))
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 925, in list
    containers.append(self.get(r['Id']))
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 862, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/container.py", line 755, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/containers/12a4f6fb5a20451fbf78c360ef14ae6eb12ed99b964b98c3e723e947da9b4e61/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 678, in run_one
    client = check_docker_environment()
  File "/project/src/duckietown-shell/lib/dt_shell/env_checks.py", line 67, in check_docker_environment
    raise InvalidEnvironment(msg)
dt_shell.exceptions.InvalidEnvironment: I cannot communicate with Docker:
404 Client Error: Not Found for url: http+docker://localhost/v1.35/containers/12a4f6fb5a20451fbf78c360ef14ae6eb12ed99b964b98c3e723e947da9b4e61/json

Make sure the docker service is running.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible