Duckietown Challenges Home Challenges Submissions

Evaluator 2187

ID2187
evaluatormont03-153f60ff88da-1
ownerI don't have one 😀
machineaido-ms-7b09
processmont03-153f60ff88da-1
versiond-c:6.1.20;d-c-r:6.0.49
first heard
last heard
statusinactive
# evaluating
# success1 38000
# timeout
# failed
# error2 38007
# aborted
# host-error5 38108
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
381269369Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simhost-errornomont03-153f60ff88da-10:00:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
381219369Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simhost-errornomont03-153f60ff88da-10:00:01
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
381179367Jean-Sébastien Grondin 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simhost-errornomont03-153f60ff88da-10:00:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
381109371Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simhost-errornomont03-153f60ff88da-10:00:01
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
381089370Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simhost-errornomont03-153f60ff88da-10:00:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.49-py3.8.egg/duckietown_challenges_runner/runner.py", line 654, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/media/sdb/duckietown-tmp-evals/production/aido5-LF-sim-validation'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
380099973Bea Baselines 🐤template-pytorchaido5-LFVI-sim-validationLFVIv-simerrornomont03-153f60ff88da-11:00:34
Waited 3605 seconds [...]
Waited 3605 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
380079976Bea Baselines 🐤template-tensorflowaido5-LFP-sim-validationLFP-simerrornomont03-153f60ff88da-10:00:54
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 208, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 141, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 305, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego" aborted with the following error:

error in ego |Unexpected error:
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             ||     call_if_fun_exists(node, "init", context=context_data)
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 32, in init
             ||     raise Exception(msg)
             || Exception: Could not find gpu device.
             ||
             || The above exception was the direct cause of the following exception:
             ||
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
             ||     raise Exception(msg) from e
             || Exception: Exception while calling the node's init() function.
             ||
             || | Traceback (most recent call last):
             || |   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             || |     call_if_fun_exists(node, "init", context=context_data)
             || |   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             || |     f(**kwargs)
             || |   File "solution.py", line 32, in init
             || |     raise Exception(msg)
             || | Exception: Could not find gpu device.
             || |
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 655, in go
    wrap(cie)
  File "experiment_manager.py", line 640, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 323, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
3800010808Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnomont03-153f60ff88da-10:10:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5460927423134307
survival_time_median14.950000000000076
deviation-center-line_median0.8442475638328444
in-drivable-lane_median6.475000000000092


other stats
agent_compute-ego_max0.01915273666381836
agent_compute-ego_mean0.018763840794563297
agent_compute-ego_median0.018850847880045575
agent_compute-ego_min0.018200930754343668
complete-iteration_max0.2598147741953532
complete-iteration_mean0.24226650834083557
complete-iteration_median0.2416749926408132
complete-iteration_min0.22590127388636272
deviation-center-line_max1.2118232739254846
deviation-center-line_mean0.9176786032207256
deviation-center-line_min0.7703960112917295
deviation-heading_max2.4263681128917907
deviation-heading_mean2.059416968055291
deviation-heading_median2.0031675683177763
deviation-heading_min1.8049646226938216
driven_any_max1.021050783961801
driven_any_mean1.0193021083807854
driven_any_median1.0210475968647734
driven_any_min1.0140624558317937
driven_lanedir_consec_max0.6311297615342181
driven_lanedir_consec_mean0.5553977400164748
driven_lanedir_consec_min0.4982757139048195
driven_lanedir_max0.6311297615342181
driven_lanedir_mean0.5553977400164748
driven_lanedir_median0.5460927423134307
driven_lanedir_min0.4982757139048195
get_duckie_state_max2.8506914774576824e-06
get_duckie_state_mean2.7708212534586587e-06
get_duckie_state_median2.768834431966146e-06
get_duckie_state_min2.6949246724446613e-06
get_robot_state_max0.01721007823944092
get_robot_state_mean0.016778456966082253
get_robot_state_median0.01675150791803996
get_robot_state_min0.016400733788808186
get_state_dump_max0.014170968532562256
get_state_dump_mean0.013922423919041952
get_state_dump_median0.013870363235473632
get_state_dump_min0.013778000672658284
get_ui_image_max0.04384433746337891
get_ui_image_mean0.042799320022265115
get_ui_image_median0.04314765135447184
get_ui_image_min0.04105763991673787
in-drivable-lane_max7.100000000000097
in-drivable-lane_mean6.300000000000082
in-drivable-lane_min5.150000000000047
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.0140624558317937, "get_ui_image": 0.04260603109995524, "step_physics": 0.10441447416941324, "survival_time": 14.950000000000076, "driven_lanedir": 0.4982757139048195, "get_state_dump": 0.013889633019765216, "sim_render-ego": 0.0066071613629659015, "get_robot_state": 0.016508015791575115, "get_duckie_state": 2.6949246724446613e-06, "in-drivable-lane": 7.100000000000097, "agent_compute-ego": 0.01875911235809326, "deviation-heading": 1.8049646226938216, "complete-iteration": 0.2381471808751424, "set_robot_commands": 0.00527084747950236, "deviation-center-line": 0.7703960112917295, "driven_lanedir_consec": 0.4982757139048195, "sim_compute_sim_state": 0.02539307196935018, "sim_compute_performance-ego": 0.004534870783487956}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 1.0210503119081469, "get_ui_image": 0.04105763991673787, "step_physics": 0.09677472035090127, "survival_time": 14.950000000000076, "driven_lanedir": 0.5533296947655155, "get_state_dump": 0.013778000672658284, "sim_render-ego": 0.006540697415669759, "get_robot_state": 0.016400733788808186, "get_duckie_state": 2.7871131896972655e-06, "in-drivable-lane": 6.400000000000091, "agent_compute-ego": 0.018200930754343668, "deviation-heading": 1.981438012256454, "complete-iteration": 0.22590127388636272, "set_robot_commands": 0.00495206356048584, "deviation-center-line": 0.8034853325687019, "driven_lanedir_consec": 0.5533296947655155, "sim_compute_sim_state": 0.02361580848693848, "sim_compute_performance-ego": 0.004415493806203206}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 1.0210448818214004, "get_ui_image": 0.043689271608988445, "step_physics": 0.10863844315210978, "survival_time": 14.950000000000076, "driven_lanedir": 0.6311297615342181, "get_state_dump": 0.014170968532562256, "sim_render-ego": 0.006838011741638184, "get_robot_state": 0.016995000044504802, "get_duckie_state": 2.750555674235026e-06, "in-drivable-lane": 5.150000000000047, "agent_compute-ego": 0.01915273666381836, "deviation-heading": 2.4263681128917907, "complete-iteration": 0.245202804406484, "set_robot_commands": 0.00523996114730835, "deviation-center-line": 1.2118232739254846, "driven_lanedir_consec": 0.6311297615342181, "sim_compute_sim_state": 0.02548868576685587, "sim_compute_performance-ego": 0.0048189711570739745}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 1.021050783961801, "get_ui_image": 0.04384433746337891, "step_physics": 0.11962850411732992, "survival_time": 14.950000000000076, "driven_lanedir": 0.538855789861346, "get_state_dump": 0.013851093451182048, "sim_render-ego": 0.006956323782602946, "get_robot_state": 0.01721007823944092, "get_duckie_state": 2.8506914774576824e-06, "in-drivable-lane": 6.550000000000093, "agent_compute-ego": 0.018942583401997885, "deviation-heading": 2.0248971243790987, "complete-iteration": 0.2598147741953532, "set_robot_commands": 0.005049480597178141, "deviation-center-line": 0.885009795096987, "driven_lanedir_consec": 0.538855789861346, "sim_compute_sim_state": 0.02940179983774821, "sim_compute_performance-ego": 0.004757342338562012}}
set_robot_commands_max0.00527084747950236
set_robot_commands_mean0.005128088196118673
set_robot_commands_median0.005144720872243246
set_robot_commands_min0.00495206356048584
sim_compute_performance-ego_max0.0048189711570739745
sim_compute_performance-ego_mean0.004631669521331787
sim_compute_performance-ego_median0.004646106561024984
sim_compute_performance-ego_min0.004415493806203206
sim_compute_sim_state_max0.02940179983774821
sim_compute_sim_state_mean0.025974841515223186
sim_compute_sim_state_median0.025440878868103027
sim_compute_sim_state_min0.02361580848693848
sim_render-ego_max0.006956323782602946
sim_render-ego_mean0.006735548575719197
sim_render-ego_median0.006722586552302043
sim_render-ego_min0.006540697415669759
simulation-passed1
step_physics_max0.11962850411732992
step_physics_mean0.10736403544743855
step_physics_median0.10652645866076152
step_physics_min0.09677472035090127
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible