Duckietown Challenges Home Challenges Submissions

Evaluator 2188

ID2188
evaluatormont05-7d821b41dde0-1
ownerI don't have one 😀
machineaido-ms-7b09
processmont05-7d821b41dde0-1
versiond-c:6.1.20;d-c-r:6.0.49
first heard
last heard
statusinactive
# evaluating
# success1 38001
# timeout1 38127
# failed
# error1 38006
# aborted
# host-error4 38109
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
381279370Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simtimeoutnomont05-7d821b41dde0-11:05:14
I can see how the jo [...]
I can see how the job 38127 is timeout because passed 3914 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
381229367Jean-Sébastien Grondin 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simhost-errornomont05-7d821b41dde0-10:00:01
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
381189370Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simhost-errornomont05-7d821b41dde0-10:00:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
381129364Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simhost-errornomont05-7d821b41dde0-10:00:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
381099371Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simhost-errornomont05-7d821b41dde0-10:00:01
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
380069982Bea Baselines 🐤template-tensorflowaido5-LFVI-sim-validationLFVIv-simerrornomont05-7d821b41dde0-11:00:33
Waited 3602 seconds [...]
Waited 3602 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
3800110809Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnomont05-7d821b41dde0-10:10:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9041412897845968
survival_time_median14.950000000000076
deviation-center-line_median0.3708527702246479
in-drivable-lane_median9.675000000000049


other stats
agent_compute-ego_max0.019686365922292073
agent_compute-ego_mean0.01909865293076368
agent_compute-ego_median0.019036011957540747
agent_compute-ego_min0.01863622188568115
complete-iteration_max0.25178424738840777
complete-iteration_mean0.24061065828550984
complete-iteration_median0.242268381913503
complete-iteration_min0.2261216219266256
deviation-center-line_max0.6550944862865037
deviation-center-line_mean0.4400829448556615
deviation-center-line_min0.3635317526868464
deviation-heading_max2.082242930954546
deviation-heading_mean1.9003905559711765
deviation-heading_median1.8754684109651345
deviation-heading_min1.7683824709998914
driven_any_max3.0631313314409176
driven_any_mean3.0291897562969363
driven_any_median3.0581368954344157
driven_any_min2.937353902877996
driven_lanedir_consec_max1.080500510287073
driven_lanedir_consec_mean0.9218498078844952
driven_lanedir_consec_min0.7986161416817149
driven_lanedir_max1.080500510287073
driven_lanedir_mean0.9218498078844952
driven_lanedir_median0.9041412897845968
driven_lanedir_min0.7986161416817149
get_duckie_state_max3.0843416849772137e-06
get_duckie_state_mean2.838188347500791e-06
get_duckie_state_median2.767244974772135e-06
get_duckie_state_min2.73392175548168e-06
get_robot_state_max0.017075302708855072
get_robot_state_mean0.01673229716934532
get_robot_state_median0.016693969964981077
get_robot_state_min0.016465946038564046
get_state_dump_max0.01422989077684356
get_state_dump_mean0.013960271346859817
get_state_dump_median0.013909687598546343
get_state_dump_min0.013791819413503012
get_ui_image_max0.04410774117978192
get_ui_image_mean0.04170775544214193
get_ui_image_median0.04180266221364339
get_ui_image_min0.039117956161499025
in-drivable-lane_max10.100000000000088
in-drivable-lane_mean9.587500000000055
in-drivable-lane_min8.90000000000004
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 3.053165193619612, "get_ui_image": 0.04106145222981771, "step_physics": 0.10341248114903769, "survival_time": 14.950000000000076, "driven_lanedir": 0.8467744952002938, "get_state_dump": 0.013828378518422444, "sim_render-ego": 0.007070086797078451, "get_robot_state": 0.016465946038564046, "get_duckie_state": 2.7950604756673176e-06, "in-drivable-lane": 10.100000000000088, "agent_compute-ego": 0.01933816194534302, "deviation-heading": 1.7683824709998914, "complete-iteration": 0.2387960410118103, "set_robot_commands": 0.0052981734275817875, "deviation-center-line": 0.3635317526868464, "driven_lanedir_consec": 0.8467744952002938, "sim_compute_sim_state": 0.027550713221232096, "sim_compute_performance-ego": 0.004600561459859212}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 3.0631313314409176, "get_ui_image": 0.039117956161499025, "step_physics": 0.09519768238067627, "survival_time": 14.950000000000076, "driven_lanedir": 0.9615080843689, "get_state_dump": 0.013791819413503012, "sim_render-ego": 0.006682376861572265, "get_robot_state": 0.016674263477325438, "get_duckie_state": 2.739429473876953e-06, "in-drivable-lane": 9.600000000000051, "agent_compute-ego": 0.01863622188568115, "deviation-heading": 1.9434220711101895, "complete-iteration": 0.2261216219266256, "set_robot_commands": 0.005021638075510661, "deviation-center-line": 0.369721157200539, "driven_lanedir_consec": 0.9615080843689, "sim_compute_sim_state": 0.02637982924779256, "sim_compute_performance-ego": 0.004454344908396403}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 3.063108597249219, "get_ui_image": 0.04254387219746907, "step_physics": 0.10745374123255412, "survival_time": 14.950000000000076, "driven_lanedir": 1.080500510287073, "get_state_dump": 0.013990996678670249, "sim_render-ego": 0.0071771240234375, "get_robot_state": 0.01671367645263672, "get_duckie_state": 3.0843416849772137e-06, "in-drivable-lane": 8.90000000000004, "agent_compute-ego": 0.019686365922292073, "deviation-heading": 2.082242930954546, "complete-iteration": 0.2457407228151957, "set_robot_commands": 0.005390435854593913, "deviation-center-line": 0.6550944862865037, "driven_lanedir_consec": 1.080500510287073, "sim_compute_sim_state": 0.02790862639745077, "sim_compute_performance-ego": 0.004700021743774414}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.937353902877996, "get_ui_image": 0.04410774117978192, "step_physics": 0.1090060327111221, "survival_time": 14.350000000000067, "driven_lanedir": 0.7986161416817149, "get_state_dump": 0.01422989077684356, "sim_render-ego": 0.007077546069846336, "get_robot_state": 0.017075302708855072, "get_duckie_state": 2.73392175548168e-06, "in-drivable-lane": 9.750000000000044, "agent_compute-ego": 0.01873386196973847, "deviation-heading": 1.8075147508200788, "complete-iteration": 0.25178424738840777, "set_robot_commands": 0.005387938396440565, "deviation-center-line": 0.3719843832487569, "driven_lanedir_consec": 0.7986161416817149, "sim_compute_sim_state": 0.03127070835658482, "sim_compute_performance-ego": 0.0047218483914897}}
set_robot_commands_max0.005390435854593913
set_robot_commands_mean0.0052745464385317315
set_robot_commands_median0.005343055912011176
set_robot_commands_min0.005021638075510661
sim_compute_performance-ego_max0.0047218483914897
sim_compute_performance-ego_mean0.004619194125879932
sim_compute_performance-ego_median0.0046502916018168125
sim_compute_performance-ego_min0.004454344908396403
sim_compute_sim_state_max0.03127070835658482
sim_compute_sim_state_mean0.02827746930576506
sim_compute_sim_state_median0.027729669809341433
sim_compute_sim_state_min0.02637982924779256
sim_render-ego_max0.0071771240234375
sim_render-ego_mean0.007001783437983638
sim_render-ego_median0.007073816433462393
sim_render-ego_min0.006682376861572265
simulation-passed1
step_physics_max0.1090060327111221
step_physics_mean0.10376748436834754
step_physics_median0.1054331111907959
step_physics_min0.09519768238067627
survival_time_max14.950000000000076
survival_time_mean14.800000000000075
survival_time_min14.350000000000067
No reset possible