Duckietown Challenges Home Challenges Submissions

Evaluator 4890

ID4890
evaluatornogpu-prod-06
ownerI don't have one 😀
machinenogpu-prod_f6d601ffe2c5
processnogpu-prod-06_f6d601ffe2c5
version6.2.5
first heard
last heard
statusinactive
# evaluating
# success21 53947
# timeout1 57568
# failed4 54346
# error
# aborted2 57322
# host-error44 53915
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5756810891Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-06----No reset possible
5738910941Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:45:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.0130324288272143
agent_compute-ego0_mean0.01231136677366411
agent_compute-ego0_median0.012181477880199982
agent_compute-ego0_min0.011850082507042168
complete-iteration_max0.3403157004706568
complete-iteration_mean0.28275576633378724
complete-iteration_median0.281531765994978
complete-iteration_min0.22764383287453632
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max1.3767134438545678e-06
get_duckie_state_mean1.3351241912968847e-06
get_duckie_state_median1.3325434739544033e-06
get_duckie_state_min1.298696373424165e-06
get_robot_state_max0.0037438287822332706
get_robot_state_mean0.0036583295174184983
get_robot_state_median0.003658938765228043
get_robot_state_min0.0035716117569846374
get_state_dump_max0.004760517069541048
get_state_dump_mean0.0046418131042975964
get_state_dump_median0.004658304781441287
get_state_dump_min0.004490125784766763
get_ui_image_max0.034918574270459635
get_ui_image_mean0.02959938425505588
get_ui_image_median0.029207353190915174
get_ui_image_min0.02506425636793354
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.026981507014672425, "step_physics": 0.19950111382807623, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004590716588308571, "get_robot_state": 0.0035716117569846374, "sim_render-ego0": 0.0037588739673065008, "get_duckie_state": 1.3177539784147976e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.011850082507042168, "complete-iteration": 0.2625168856732752, "set_robot_commands": 0.002143733606648187, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008142256518386981, "sim_compute_performance-ego0": 0.0018969664069436968}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.034918574270459635, "step_physics": 0.26410773195493825, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.004725892974574004, "get_robot_state": 0.003697692901268291, "sim_render-ego0": 0.003979021266140013, "get_duckie_state": 1.347332969494009e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.0130324288272143, "complete-iteration": 0.3403157004706568, "set_robot_commands": 0.002263822722296036, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.011501838126647086, "sim_compute_performance-ego0": 0.002003864086637092}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.031433199367157924, "step_physics": 0.2317957282562637, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004760517069541048, "get_robot_state": 0.0037438287822332706, "sim_render-ego0": 0.003919113089301802, "get_duckie_state": 1.3767134438545678e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.011878256396786756, "complete-iteration": 0.30054664631668077, "set_robot_commands": 0.002236640225044397, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008685921153656946, "sim_compute_performance-ego0": 0.002002852048405402}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.02506425636793354, "step_physics": 0.16804489426370664, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004490125784766763, "get_robot_state": 0.0036201846291877946, "sim_render-ego0": 0.0037992913756739783, "get_duckie_state": 1.298696373424165e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.012484699363613207, "complete-iteration": 0.22764383287453632, "set_robot_commands": 0.0021969926645118527, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.005890434925800358, "sim_compute_performance-ego0": 0.0019749179668569444}}
set_robot_commands_max0.002263822722296036
set_robot_commands_mean0.002210297304625118
set_robot_commands_median0.002216816444778125
set_robot_commands_min0.002143733606648187
sim_compute_performance-ego0_max0.002003864086637092
sim_compute_performance-ego0_mean0.001969650127210784
sim_compute_performance-ego0_median0.001988885007631173
sim_compute_performance-ego0_min0.0018969664069436968
sim_compute_sim_state_max0.011501838126647086
sim_compute_sim_state_mean0.008555112681122842
sim_compute_sim_state_median0.008414088836021964
sim_compute_sim_state_min0.005890434925800358
sim_render-ego0_max0.003979021266140013
sim_render-ego0_mean0.003864074924605574
sim_render-ego0_median0.0038592022324878904
sim_render-ego0_min0.0037588739673065008
simulation-passed1
step_physics_max0.26410773195493825
step_physics_mean0.2158623670757462
step_physics_median0.21564842104216997
step_physics_min0.16804489426370664
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5737211010Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-060:03:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5733111312Moustafa Elarabitemplate-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-060:03:50
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5732611312Moustafa Elarabitemplate-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-060:02:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5732211012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-060:00:55
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5731211313Moustafa Elarabitemplate-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-060:01:06
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5713611005Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:39:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.864481713646221
survival_time_median59.99999999999873
deviation-center-line_median2.6095385872143853
in-drivable-lane_median2.0499999999999474


other stats
agent_compute-ego0_max0.03364773098376272
agent_compute-ego0_mean0.01927788211940032
agent_compute-ego0_median0.015430895513936343
agent_compute-ego0_min0.012602006465965862
complete-iteration_max0.22899174908614972
complete-iteration_mean0.19763667388919193
complete-iteration_median0.19319080978904937
complete-iteration_min0.17517332689251927
deviation-center-line_max3.600463357960853
deviation-center-line_mean2.7391142334552656
deviation-center-line_min2.13691640143144
deviation-heading_max19.81829997473284
deviation-heading_mean15.126183547365692
deviation-heading_median14.764256111364618
deviation-heading_min11.157921992000684
driven_any_max7.78868005820792
driven_any_mean7.469029175010416
driven_any_median7.434721011012584
driven_any_min7.217994619808575
driven_lanedir_consec_max7.348353926741447
driven_lanedir_consec_mean5.330039352911675
driven_lanedir_consec_min4.242840057612814
driven_lanedir_max7.348353926741447
driven_lanedir_mean6.722546998296327
driven_lanedir_median6.836330999395647
driven_lanedir_min5.869172067652565
get_duckie_state_max1.5432689708039524e-06
get_duckie_state_mean1.3687231459288075e-06
get_duckie_state_median1.3241065134116751e-06
get_duckie_state_min1.283410586087928e-06
get_robot_state_max0.003868197521301829
get_robot_state_mean0.0037346125443114726
get_robot_state_median0.0037547986175892854
get_robot_state_min0.0035606554207654917
get_state_dump_max0.004745526873598885
get_state_dump_mean0.0046406506896515265
get_state_dump_median0.0046718913649242185
get_state_dump_min0.004473293155158787
get_ui_image_max0.03716632150591263
get_ui_image_mean0.030897610987552893
get_ui_image_median0.03078982911439462
get_ui_image_min0.0248444642155097
in-drivable-lane_max5.399999999999962
in-drivable-lane_mean2.5874999999999524
in-drivable-lane_min0.8499999999999517
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.78868005820792, "get_ui_image": 0.028462533053510097, "step_physics": 0.10866261282927984, "survival_time": 59.99999999999873, "driven_lanedir": 7.348353926741447, "get_state_dump": 0.004600935633434642, "get_robot_state": 0.003717037362917377, "sim_render-ego0": 0.0039159529016575744, "get_duckie_state": 1.3334367991883392e-06, "in-drivable-lane": 1.8000000000000256, "deviation-heading": 11.157921992000684, "agent_compute-ego0": 0.012602006465965862, "complete-iteration": 0.17602484946842495, "set_robot_commands": 0.002221240687628372, "deviation-center-line": 2.13691640143144, "driven_lanedir_consec": 7.348353926741447, "sim_compute_sim_state": 0.009732054433258844, "sim_compute_performance-ego0": 0.0020252316321659643}, "LF-norm-zigzag-000-ego0": {"driven_any": 7.390735654062513, "get_ui_image": 0.03716632150591263, "step_physics": 0.14531010553104295, "survival_time": 59.99999999999873, "driven_lanedir": 6.622347627302603, "get_state_dump": 0.004745526873598885, "get_robot_state": 0.003868197521301829, "sim_render-ego0": 0.004061062667490937, "get_duckie_state": 1.5432689708039524e-06, "in-drivable-lane": 2.2999999999998693, "deviation-heading": 16.304126552435797, "agent_compute-ego0": 0.017581796765228194, "complete-iteration": 0.22899174908614972, "set_robot_commands": 0.002315747946327076, "deviation-center-line": 2.8270091022588093, "driven_lanedir_consec": 4.961817446388439, "sim_compute_sim_state": 0.011735588585109535, "sim_compute_performance-ego0": 0.0021151039225175716}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.217994619808575, "get_ui_image": 0.03311712517527914, "step_physics": 0.13493684924313865, "survival_time": 59.99999999999873, "driven_lanedir": 5.869172067652565, "get_state_dump": 0.004742847096413796, "get_robot_state": 0.0037925598722611934, "sim_render-ego0": 0.004028859682424579, "get_duckie_state": 1.3147762276350112e-06, "in-drivable-lane": 5.399999999999962, "deviation-heading": 19.81829997473284, "agent_compute-ego0": 0.013279994262644493, "complete-iteration": 0.2103567701096737, "set_robot_commands": 0.0022921889747409994, "deviation-center-line": 3.600463357960853, "driven_lanedir_consec": 4.767145980904003, "sim_compute_sim_state": 0.012002250733323935, "sim_compute_performance-ego0": 0.002078944697765188}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.478706367962656, "get_ui_image": 0.0248444642155097, "step_physics": 0.09452667919225637, "survival_time": 59.99999999999873, "driven_lanedir": 7.0503143714886924, "get_state_dump": 0.004473293155158787, "get_robot_state": 0.0035606554207654917, "sim_render-ego0": 0.003777796183895013, "get_duckie_state": 1.283410586087928e-06, "in-drivable-lane": 0.8499999999999517, "deviation-heading": 13.22438567029344, "agent_compute-ego0": 0.03364773098376272, "complete-iteration": 0.17517332689251927, "set_robot_commands": 0.0021948814392089844, "deviation-center-line": 2.392068072169961, "driven_lanedir_consec": 4.242840057612814, "sim_compute_sim_state": 0.0060922990333627, "sim_compute_performance-ego0": 0.0019721681132701713}}
set_robot_commands_max0.002315747946327076
set_robot_commands_mean0.002256014761976358
set_robot_commands_median0.0022567148311846857
set_robot_commands_min0.0021948814392089844
sim_compute_performance-ego0_max0.0021151039225175716
sim_compute_performance-ego0_mean0.0020478620914297237
sim_compute_performance-ego0_median0.0020520881649655764
sim_compute_performance-ego0_min0.0019721681132701713
sim_compute_sim_state_max0.012002250733323935
sim_compute_sim_state_mean0.009890548196263754
sim_compute_sim_state_median0.010733821509184187
sim_compute_sim_state_min0.0060922990333627
sim_render-ego0_max0.004061062667490937
sim_render-ego0_mean0.003945917858867026
sim_render-ego0_median0.003972406292041077
sim_render-ego0_min0.003777796183895013
simulation-passed1
step_physics_max0.14531010553104295
step_physics_mean0.12085906169892946
step_physics_median0.12179973103620924
step_physics_min0.09452667919225637
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5703011037Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:18:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.26108391691152
survival_time_median26.17500000000024
deviation-center-line_median1.2964247973540162
in-drivable-lane_median5.175000000000006


other stats
agent_compute-ego0_max0.01337911668031112
agent_compute-ego0_mean0.012584152986681476
agent_compute-ego0_median0.012380051787530587
agent_compute-ego0_min0.012197391691353608
complete-iteration_max0.2085850072943646
complete-iteration_mean0.18427803860854636
complete-iteration_median0.18380141898070915
complete-iteration_min0.16092430917840256
deviation-center-line_max2.7143396205026793
deviation-center-line_mean1.4595705069665212
deviation-center-line_min0.5310928126553732
deviation-heading_max11.574273319602336
deviation-heading_mean7.054287057509079
deviation-heading_median6.629033230802484
deviation-heading_min3.384808448829008
driven_any_max9.71853768456339
driven_any_mean4.711273646921494
driven_any_median3.8290413537583534
driven_any_min1.4684741956058818
driven_lanedir_consec_max8.882008004295642
driven_lanedir_consec_mean3.580167799458793
driven_lanedir_consec_min0.916495359716488
driven_lanedir_max8.882008004295642
driven_lanedir_mean3.606101983285421
driven_lanedir_median2.312952284564777
driven_lanedir_min0.916495359716488
get_duckie_state_max1.469384069028108e-06
get_duckie_state_mean1.3430504495348052e-06
get_duckie_state_median1.3188919948842882e-06
get_duckie_state_min1.265033739342537e-06
get_robot_state_max0.004025046203447425
get_robot_state_mean0.0037183797638681714
get_robot_state_median0.003626566327268059
get_robot_state_min0.003595340197489142
get_state_dump_max0.00517400192177814
get_state_dump_mean0.004741617420996911
get_state_dump_median0.004622124612268618
get_state_dump_min0.004548218537672267
get_ui_image_max0.033942852934745894
get_ui_image_mean0.02997787127513826
get_ui_image_median0.03033253379610901
get_ui_image_min0.025303564573589125
in-drivable-lane_max10.950000000000149
in-drivable-lane_mean6.075000000000039
in-drivable-lane_min2.9999999999999973
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 9.71853768456339, "get_ui_image": 0.027409436601484743, "step_physics": 0.10405892853335874, "survival_time": 59.99999999999873, "driven_lanedir": 8.882008004295642, "get_state_dump": 0.0046077906142464285, "get_robot_state": 0.0036445865027612697, "sim_render-ego0": 0.003835329306711265, "get_duckie_state": 1.337208616842735e-06, "in-drivable-lane": 3.549999999999915, "deviation-heading": 11.574273319602336, "agent_compute-ego0": 0.012351619512413464, "complete-iteration": 0.16920608445865526, "set_robot_commands": 0.0021646221313349513, "deviation-center-line": 2.7143396205026793, "driven_lanedir_consec": 8.882008004295642, "sim_compute_sim_state": 0.009077810228714638, "sim_compute_performance-ego0": 0.0019729506661750992}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.4684741956058818, "get_ui_image": 0.033942852934745894, "step_physics": 0.12573904751642653, "survival_time": 10.90000000000002, "driven_lanedir": 0.916495359716488, "get_state_dump": 0.004636458610290806, "get_robot_state": 0.003595340197489142, "sim_render-ego0": 0.003778851739892132, "get_duckie_state": 1.265033739342537e-06, "in-drivable-lane": 2.9999999999999973, "deviation-heading": 3.384808448829008, "agent_compute-ego0": 0.01240848406264771, "complete-iteration": 0.19839675350276303, "set_robot_commands": 0.0021559571566647047, "deviation-center-line": 0.5310928126553732, "driven_lanedir_consec": 0.916495359716488, "sim_compute_sim_state": 0.010172256051677549, "sim_compute_performance-ego0": 0.0018860128916561875}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.33450474406205, "get_ui_image": 0.033255630990733274, "step_physics": 0.1338293951490651, "survival_time": 22.95000000000019, "driven_lanedir": 2.180136658636483, "get_state_dump": 0.00517400192177814, "get_robot_state": 0.004025046203447425, "sim_render-ego0": 0.004117647461269212, "get_duckie_state": 1.469384069028108e-06, "in-drivable-lane": 6.800000000000097, "deviation-heading": 5.946403229392855, "agent_compute-ego0": 0.01337911668031112, "complete-iteration": 0.2085850072943646, "set_robot_commands": 0.002337320991184401, "deviation-center-line": 1.1802662007210858, "driven_lanedir_consec": 2.143203709480649, "sim_compute_sim_state": 0.01020949819813604, "sim_compute_performance-ego0": 0.002160167175790538}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.323577963454657, "get_ui_image": 0.025303564573589125, "step_physics": 0.10108341304474654, "survival_time": 29.400000000000283, "driven_lanedir": 2.4457679104930716, "get_state_dump": 0.004548218537672267, "get_robot_state": 0.003608546151774849, "sim_render-ego0": 0.003754402057019478, "get_duckie_state": 1.300575372925841e-06, "in-drivable-lane": 10.950000000000149, "deviation-heading": 7.311663232212114, "agent_compute-ego0": 0.012197391691353608, "complete-iteration": 0.16092430917840256, "set_robot_commands": 0.0021594619103319983, "deviation-center-line": 1.4125833939869468, "driven_lanedir_consec": 2.3789641243423914, "sim_compute_sim_state": 0.006296031948666014, "sim_compute_performance-ego0": 0.0018862702445789589}}
set_robot_commands_max0.002337320991184401
set_robot_commands_mean0.0022043405473790136
set_robot_commands_median0.002162042020833475
set_robot_commands_min0.0021559571566647047
sim_compute_performance-ego0_max0.002160167175790538
sim_compute_performance-ego0_mean0.001976350244550196
sim_compute_performance-ego0_median0.001929610455377029
sim_compute_performance-ego0_min0.0018860128916561875
sim_compute_sim_state_max0.01020949819813604
sim_compute_sim_state_mean0.00893889910679856
sim_compute_sim_state_median0.009625033140196092
sim_compute_sim_state_min0.006296031948666014
sim_render-ego0_max0.004117647461269212
sim_render-ego0_mean0.003871557641223022
sim_render-ego0_median0.003807090523301699
sim_render-ego0_min0.003754402057019478
simulation-passed1
step_physics_max0.1338293951490651
step_physics_mean0.11617769606089924
step_physics_median0.11489898802489264
step_physics_min0.10108341304474654
survival_time_max59.99999999999873
survival_time_mean30.812499999999808
survival_time_min10.90000000000002
No reset possible
5686611069Francois Hebertsim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:33:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median10.21461708106266
survival_time_median59.99999999999873
deviation-center-line_median3.1531431009325015
in-drivable-lane_median0.19999999999998863


other stats
agent_compute-ego0_max0.01892631794391921
agent_compute-ego0_mean0.01346855075234279
agent_compute-ego0_median0.011702444928571842
agent_compute-ego0_min0.011542995208308262
complete-iteration_max0.1917562343794341
complete-iteration_mean0.16961806809078347
complete-iteration_median0.16716917021685496
complete-iteration_min0.1523776975499899
deviation-center-line_max3.549000634410673
deviation-center-line_mean3.2021837182236
deviation-center-line_min2.9534480366187243
deviation-heading_max9.44523834743225
deviation-heading_mean8.405443593685181
deviation-heading_median8.775350525530925
deviation-heading_min6.625834976246622
driven_any_max10.42262453232309
driven_any_mean10.420332672581036
driven_any_median10.422464149515145
driven_any_min10.413777858970764
driven_lanedir_consec_max10.296561602940448
driven_lanedir_consec_mean10.08410130714127
driven_lanedir_consec_min9.61060946349931
driven_lanedir_max10.296561602940448
driven_lanedir_mean10.08410130714127
driven_lanedir_median10.21461708106266
driven_lanedir_min9.61060946349931
get_duckie_state_max1.2188926525258898e-06
get_duckie_state_mean1.1691642145034577e-06
get_duckie_state_median1.1771048832495545e-06
get_duckie_state_min1.103554438988831e-06
get_robot_state_max0.003565863109845901
get_robot_state_mean0.003546745205401977
get_robot_state_median0.0035526713761163693
get_robot_state_min0.00351577495952927
get_state_dump_max0.004738992100254284
get_state_dump_mean0.004524572455416512
get_state_dump_median0.004480021085270637
get_state_dump_min0.00439925555087049
get_ui_image_max0.03336244260738732
get_ui_image_mean0.028504697508260075
get_ui_image_median0.028147668663806264
get_ui_image_min0.02436101009804045
in-drivable-lane_max3.4499999999999176
in-drivable-lane_mean0.9624999999999736
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.42262453232309, "get_ui_image": 0.026143462532862937, "step_physics": 0.09000599572898745, "survival_time": 59.99999999999873, "driven_lanedir": 10.296561602940448, "get_state_dump": 0.004428928440357624, "get_robot_state": 0.003539872904006488, "sim_render-ego0": 0.003755129941992716, "get_duckie_state": 1.2188926525258898e-06, "in-drivable-lane": 0.0, "deviation-heading": 6.625834976246622, "agent_compute-ego0": 0.01171669952081304, "complete-iteration": 0.1523776975499899, "set_robot_commands": 0.002160247616922726, "deviation-center-line": 2.9944261567992494, "driven_lanedir_consec": 10.296561602940448, "sim_compute_sim_state": 0.008652139563643862, "sim_compute_performance-ego0": 0.0018986266022617871}, "LF-norm-zigzag-000-ego0": {"driven_any": 10.422461588543724, "get_ui_image": 0.03336244260738732, "step_physics": 0.11812168354793552, "survival_time": 59.99999999999873, "driven_lanedir": 10.233286857205831, "get_state_dump": 0.004531113730183649, "get_robot_state": 0.00351577495952927, "sim_render-ego0": 0.00383368896306504, "get_duckie_state": 1.2077757162813542e-06, "in-drivable-lane": 0.0, "deviation-heading": 9.44523834743225, "agent_compute-ego0": 0.011542995208308262, "complete-iteration": 0.1917562343794341, "set_robot_commands": 0.0021256918117068986, "deviation-center-line": 3.311860045065753, "driven_lanedir_consec": 10.233286857205831, "sim_compute_sim_state": 0.012732650119994304, "sim_compute_performance-ego0": 0.0019145049620032013}, "LF-norm-techtrack-000-ego0": {"driven_any": 10.413777858970764, "get_ui_image": 0.030151874794749595, "step_physics": 0.10939376280766344, "survival_time": 59.99999999999873, "driven_lanedir": 9.61060946349931, "get_state_dump": 0.00439925555087049, "get_robot_state": 0.0035654698482262505, "sim_render-ego0": 0.003778722264387526, "get_duckie_state": 1.1464340502177548e-06, "in-drivable-lane": 3.4499999999999176, "deviation-heading": 8.841587479391483, "agent_compute-ego0": 0.011688190336330643, "complete-iteration": 0.17965621674289114, "set_robot_commands": 0.0021702780711660772, "deviation-center-line": 3.549000634410673, "driven_lanedir_consec": 9.61060946349931, "sim_compute_sim_state": 0.012513097577249875, "sim_compute_performance-ego0": 0.0019186821507970857}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.422466710486566, "get_ui_image": 0.02436101009804045, "step_physics": 0.08929182508406691, "survival_time": 59.99999999999873, "driven_lanedir": 10.195947304919487, "get_state_dump": 0.004738992100254284, "get_robot_state": 0.003565863109845901, "sim_render-ego0": 0.003741690161623228, "get_duckie_state": 1.103554438988831e-06, "in-drivable-lane": 0.39999999999997726, "deviation-heading": 8.709113571670368, "agent_compute-ego0": 0.01892631794391921, "complete-iteration": 0.15468212369081877, "set_robot_commands": 0.002179115241413609, "deviation-center-line": 2.9534480366187243, "driven_lanedir_consec": 10.195947304919487, "sim_compute_sim_state": 0.005898530040553567, "sim_compute_performance-ego0": 0.0019051448987187395}}
set_robot_commands_max0.002179115241413609
set_robot_commands_mean0.0021588331853023275
set_robot_commands_median0.0021652628440444015
set_robot_commands_min0.0021256918117068986
sim_compute_performance-ego0_max0.0019186821507970857
sim_compute_performance-ego0_mean0.0019092396534452032
sim_compute_performance-ego0_median0.0019098249303609704
sim_compute_performance-ego0_min0.0018986266022617871
sim_compute_sim_state_max0.012732650119994304
sim_compute_sim_state_mean0.009949104325360402
sim_compute_sim_state_median0.010582618570446869
sim_compute_sim_state_min0.005898530040553567
sim_render-ego0_max0.00383368896306504
sim_render-ego0_mean0.003777307832767127
sim_render-ego0_median0.003766926103190121
sim_render-ego0_min0.003741690161623228
simulation-passed1
step_physics_max0.11812168354793552
step_physics_mean0.10170331679216332
step_physics_median0.09969987926832544
step_physics_min0.08929182508406691
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5685011427Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-060:00:51
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5669811095Olga (Ge Ya) Xu 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:25:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.217212063559323
survival_time_median58.37499999999882
deviation-center-line_median3.31043831573664
in-drivable-lane_median13.749999999999671


other stats
agent_compute-ego0_max0.011936284877635817
agent_compute-ego0_mean0.011327395833465438
agent_compute-ego0_median0.011212547038713052
agent_compute-ego0_min0.010948204378799831
complete-iteration_max0.1818995266048997
complete-iteration_mean0.151897789771267
complete-iteration_median0.14558543283476816
complete-iteration_min0.13452076681063208
deviation-center-line_max4.00148943610362
deviation-center-line_mean2.726915062070111
deviation-center-line_min0.2852941807035451
deviation-heading_max14.308514126568278
deviation-heading_mean10.002790956441968
deviation-heading_median11.874252155472568
deviation-heading_min1.9541453882544555
driven_any_max10.264863457607046
driven_any_mean7.093282175853277
driven_any_median8.354141190137064
driven_any_min1.399982865531936
driven_lanedir_consec_max8.495348869976075
driven_lanedir_consec_mean4.898119871632934
driven_lanedir_consec_min0.6627064894370176
driven_lanedir_max8.495348869976075
driven_lanedir_mean4.898119871632934
driven_lanedir_median5.217212063559323
driven_lanedir_min0.6627064894370176
get_duckie_state_max1.111516246089229e-06
get_duckie_state_mean1.046716269860332e-06
get_duckie_state_median1.0484299635571475e-06
get_duckie_state_min9.78488906237803e-07
get_robot_state_max0.003387367283856427
get_robot_state_mean0.0033027722709278134
get_robot_state_median0.003306903950644172
get_robot_state_min0.003209913898566482
get_state_dump_max0.004290991359286838
get_state_dump_mean0.004132905462277948
get_state_dump_median0.004143770334287818
get_state_dump_min0.003953089821249321
get_ui_image_max0.03277860305927418
get_ui_image_mean0.026973737816754932
get_ui_image_median0.025816630364258426
get_ui_image_min0.023483087479228705
in-drivable-lane_max23.099999999999213
in-drivable-lane_mean14.137499999999646
in-drivable-lane_min5.950000000000027
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.264863457607046, "get_ui_image": 0.024521079885274744, "step_physics": 0.08152686150048198, "survival_time": 59.99999999999873, "driven_lanedir": 8.495348869976075, "get_state_dump": 0.004034448722121519, "get_robot_state": 0.0032627927969139285, "sim_render-ego0": 0.0033590180192958506, "get_duckie_state": 1.0418157394879267e-06, "in-drivable-lane": 13.099999999999582, "deviation-heading": 13.663403660518162, "agent_compute-ego0": 0.011105193385077356, "complete-iteration": 0.13928655105069912, "set_robot_commands": 0.0019000184029762592, "deviation-center-line": 3.7039067362214024, "driven_lanedir_consec": 8.495348869976075, "sim_compute_sim_state": 0.007781027159424844, "sim_compute_performance-ego0": 0.0017238653867469045}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.399982865531936, "get_ui_image": 0.03277860305927418, "step_physics": 0.11319464997008996, "survival_time": 10.750000000000018, "driven_lanedir": 0.6627064894370176, "get_state_dump": 0.004290991359286838, "get_robot_state": 0.003387367283856427, "sim_render-ego0": 0.003546298653991134, "get_duckie_state": 1.111516246089229e-06, "in-drivable-lane": 5.950000000000027, "deviation-heading": 1.9541453882544555, "agent_compute-ego0": 0.011936284877635817, "complete-iteration": 0.1818995266048997, "set_robot_commands": 0.0019840929243299696, "deviation-center-line": 0.2852941807035451, "driven_lanedir_consec": 0.6627064894370176, "sim_compute_sim_state": 0.00891691004788434, "sim_compute_performance-ego0": 0.0017879936430189344}, "LF-norm-techtrack-000-ego0": {"driven_any": 8.179372918376085, "get_ui_image": 0.0271121808432421, "step_physics": 0.08834536744593383, "survival_time": 59.99999999999873, "driven_lanedir": 6.152070666378175, "get_state_dump": 0.003953089821249321, "get_robot_state": 0.003209913898566482, "sim_render-ego0": 0.003288698037597758, "get_duckie_state": 9.78488906237803e-07, "in-drivable-lane": 14.399999999999764, "deviation-heading": 14.308514126568278, "agent_compute-ego0": 0.010948204378799831, "complete-iteration": 0.15188431461883722, "set_robot_commands": 0.002035927117416801, "deviation-center-line": 4.00148943610362, "driven_lanedir_consec": 6.152070666378175, "sim_compute_sim_state": 0.011259092875662494, "sim_compute_performance-ego0": 0.001660835335991166}, "LF-norm-small_loop-000-ego0": {"driven_any": 8.528909461898042, "get_ui_image": 0.023483087479228705, "step_physics": 0.07930270005279863, "survival_time": 56.74999999999891, "driven_lanedir": 4.282353460740471, "get_state_dump": 0.004253091946454115, "get_robot_state": 0.0033510151043744155, "sim_render-ego0": 0.0034322383957849424, "get_duckie_state": 1.0550441876263683e-06, "in-drivable-lane": 23.099999999999213, "deviation-heading": 10.085100650426975, "agent_compute-ego0": 0.011319900692348749, "complete-iteration": 0.13452076681063208, "set_robot_commands": 0.0019243384750796035, "deviation-center-line": 2.916969895251877, "driven_lanedir_consec": 4.282353460740471, "sim_compute_sim_state": 0.0056493899352113965, "sim_compute_performance-ego0": 0.001730851304363197}}
set_robot_commands_max0.002035927117416801
set_robot_commands_mean0.0019610942299506584
set_robot_commands_median0.0019542156997047868
set_robot_commands_min0.0019000184029762592
sim_compute_performance-ego0_max0.0017879936430189344
sim_compute_performance-ego0_mean0.0017258864175300505
sim_compute_performance-ego0_median0.0017273583455550507
sim_compute_performance-ego0_min0.001660835335991166
sim_compute_sim_state_max0.011259092875662494
sim_compute_sim_state_mean0.00840160500454577
sim_compute_sim_state_median0.008348968603654593
sim_compute_sim_state_min0.0056493899352113965
sim_render-ego0_max0.003546298653991134
sim_render-ego0_mean0.003406563276667421
sim_render-ego0_median0.0033956282075403967
sim_render-ego0_min0.003288698037597758
simulation-passed1
step_physics_max0.11319464997008996
step_physics_mean0.09059239474232608
step_physics_median0.0849361144732079
step_physics_min0.07930270005279863
survival_time_max59.99999999999873
survival_time_mean46.8749999999991
survival_time_min10.750000000000018
No reset possible
5636411420Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LFVI-sim-validationLFVIv-simsuccessyesnogpu-prod-060:34:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median7.701855859250158
survival_time_median33.64999999999935
deviation-center-line_median1.603711367784741
in-drivable-lane_median0.6749999999999976


other stats
agent_compute-ego0_max0.05118834566907818
agent_compute-ego0_mean0.04869418706771138
agent_compute-ego0_median0.04869418706771138
agent_compute-ego0_min0.04620002846634457
agent_compute-npc0_max0.041986441235061886
agent_compute-npc0_mean0.040158893841013454
agent_compute-npc0_median0.040158893841013454
agent_compute-npc0_min0.038331346446965016
agent_compute-npc1_max0.04353606512306334
agent_compute-npc1_mean0.0405753164366143
agent_compute-npc1_median0.0405753164366143
agent_compute-npc1_min0.03761456775016525
agent_compute-npc2_max0.04388107328391095
agent_compute-npc2_mean0.04095392932393366
agent_compute-npc2_median0.04095392932393366
agent_compute-npc2_min0.03802678536395637
complete-iteration_max1.3123853644546195
complete-iteration_mean1.1909147349713978
complete-iteration_median1.1909147349713978
complete-iteration_min1.069444105488176
deviation-center-line_max2.8305007562607853
deviation-center-line_mean1.603711367784741
deviation-center-line_min0.37692197930869664
deviation-heading_max10.829028225443134
deviation-heading_mean6.408976580456913
deviation-heading_median6.408976580456913
deviation-heading_min1.9889249354706924
driven_any_max14.590269734375006
driven_any_mean8.060581173787787
driven_any_median8.060581173787787
driven_any_min1.530892613200568
driven_lanedir_consec_max14.2852209595819
driven_lanedir_consec_mean7.701855859250158
driven_lanedir_consec_min1.118490758918418
driven_lanedir_max14.285221064869503
driven_lanedir_mean7.702168807761216
driven_lanedir_median7.702168807761216
driven_lanedir_min1.1191165506529286
get_duckie_state_max1.6835271095742983e-06
get_duckie_state_mean1.6530021255602844e-06
get_duckie_state_median1.6530021255602844e-06
get_duckie_state_min1.62247714154627e-06
get_robot_state_max0.014777929604458971
get_robot_state_mean0.014634343729258628
get_robot_state_median0.014634343729258628
get_robot_state_min0.014490757854058284
get_state_dump_max0.010002055135713952
get_state_dump_mean0.009871468902983994
get_state_dump_median0.009871468902983994
get_state_dump_min0.009740882670254036
get_ui_image_max0.05414496798093627
get_ui_image_mean0.05286055881659036
get_ui_image_median0.05286055881659036
get_ui_image_min0.05157614965224445
in-drivable-lane_max1.3499999999999952
in-drivable-lane_mean0.6749999999999976
in-drivable-lane_min0.0
per-episodes
details{"LFVI-norm-4way-000-ego0": {"driven_any": 14.590269734375006, "get_ui_image": 0.05157614965224445, "step_physics": 0.7428344981457967, "survival_time": 59.99999999999873, "driven_lanedir": 14.285221064869503, "get_state_dump": 0.009740882670254036, "get_robot_state": 0.014490757854058284, "sim_render-ego0": 0.003997029908789286, "sim_render-npc0": 0.003969436084896599, "sim_render-npc1": 0.0039455166069494495, "sim_render-npc2": 0.003991461514831086, "get_duckie_state": 1.62247714154627e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.829028225443134, "agent_compute-ego0": 0.04620002846634457, "agent_compute-npc0": 0.041986441235061886, "agent_compute-npc1": 0.04353606512306334, "agent_compute-npc2": 0.04388107328391095, "complete-iteration": 1.069444105488176, "set_robot_commands": 0.002365317372457868, "deviation-center-line": 2.8305007562607853, "driven_lanedir_consec": 14.2852209595819, "sim_compute_sim_state": 0.041612495292136314, "sim_compute_performance-ego0": 0.002159331859299583, "sim_compute_performance-npc0": 0.0020265875013543606, "sim_compute_performance-npc1": 0.002013457804099408, "sim_compute_performance-npc2": 0.002049704971758154}, "LFVI-norm-udem1-000-ego0": {"driven_any": 1.530892613200568, "get_ui_image": 0.05414496798093627, "step_physics": 0.9909517748826216, "survival_time": 7.299999999999982, "driven_lanedir": 1.1191165506529286, "get_state_dump": 0.010002055135713952, "get_robot_state": 0.014777929604458971, "sim_render-ego0": 0.004173343684397588, "sim_render-npc0": 0.004233566271204527, "sim_render-npc1": 0.004000684841960466, "sim_render-npc2": 0.00413723543387692, "get_duckie_state": 1.6835271095742983e-06, "in-drivable-lane": 1.3499999999999952, "deviation-heading": 1.9889249354706924, "agent_compute-ego0": 0.05118834566907818, "agent_compute-npc0": 0.038331346446965016, "agent_compute-npc1": 0.03761456775016525, "agent_compute-npc2": 0.03802678536395637, "complete-iteration": 1.3123853644546195, "set_robot_commands": 0.002444974419211044, "deviation-center-line": 0.37692197930869664, "driven_lanedir_consec": 1.118490758918418, "sim_compute_sim_state": 0.04213133474596504, "sim_compute_performance-ego0": 0.0023213292465728967, "sim_compute_performance-npc0": 0.002200193145648152, "sim_compute_performance-npc1": 0.0020625526402272333, "sim_compute_performance-npc2": 0.0021560759771437872}}
set_robot_commands_max0.002444974419211044
set_robot_commands_mean0.002405145895834456
set_robot_commands_median0.002405145895834456
set_robot_commands_min0.002365317372457868
sim_compute_performance-ego0_max0.0023213292465728967
sim_compute_performance-ego0_mean0.00224033055293624
sim_compute_performance-ego0_median0.00224033055293624
sim_compute_performance-ego0_min0.002159331859299583
sim_compute_performance-npc0_max0.002200193145648152
sim_compute_performance-npc0_mean0.0021133903235012563
sim_compute_performance-npc0_median0.0021133903235012563
sim_compute_performance-npc0_min0.0020265875013543606
sim_compute_performance-npc1_max0.0020625526402272333
sim_compute_performance-npc1_mean0.002038005222163321
sim_compute_performance-npc1_median0.002038005222163321
sim_compute_performance-npc1_min0.002013457804099408
sim_compute_performance-npc2_max0.0021560759771437872
sim_compute_performance-npc2_mean0.0021028904744509703
sim_compute_performance-npc2_median0.0021028904744509703
sim_compute_performance-npc2_min0.002049704971758154
sim_compute_sim_state_max0.04213133474596504
sim_compute_sim_state_mean0.04187191501905067
sim_compute_sim_state_median0.04187191501905067
sim_compute_sim_state_min0.041612495292136314
sim_render-ego0_max0.004173343684397588
sim_render-ego0_mean0.004085186796593436
sim_render-ego0_median0.004085186796593436
sim_render-ego0_min0.003997029908789286
sim_render-npc0_max0.004233566271204527
sim_render-npc0_mean0.004101501178050563
sim_render-npc0_median0.004101501178050563
sim_render-npc0_min0.003969436084896599
sim_render-npc1_max0.004000684841960466
sim_render-npc1_mean0.003973100724454958
sim_render-npc1_median0.003973100724454958
sim_render-npc1_min0.0039455166069494495
sim_render-npc2_max0.00413723543387692
sim_render-npc2_mean0.004064348474354003
sim_render-npc2_median0.004064348474354003
sim_render-npc2_min0.003991461514831086
simulation-passed1
step_physics_max0.9909517748826216
step_physics_mean0.8668931365142092
step_physics_median0.8668931365142092
step_physics_min0.7428344981457967
survival_time_max59.99999999999873
survival_time_mean33.64999999999935
survival_time_min7.299999999999982
No reset possible
5633011429Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-testingLFt-simhost-erroryesnogpu-prod-060:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5623811135Bhavya Patwa 🇨🇦real-exercise-1aido5-LF-sim-validationLFv-simfailedyesnogpu-prod-060:05:50
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5622311137Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:51
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5617711140Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-060:01:31
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 331, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego0" aborted:

error in ego0 |Exception while handling a message on topic "get_commands".
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 331, in loop
              ||     handle_message_node(parsed, receiver0, context0)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 486, in handle_message_node
              ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 68, in on_received_get_commands
              ||     velocity, omega = self.compute_action(self.current_image)
              ||   File "solution.py", line 62, in compute_action
              ||     observation = self.preprocessor.preprocess(observation)
              ||   File "/workspace/wrappers.py", line 19, in preprocess
              ||     obs = cv2.cvtColor(obs, cv2.COLOR_BGR2RGB)
              || cv2.error: OpenCV(4.4.0) /tmp/pip-req-build-99ib2vsi/opencv/modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = cv::impl::<unnamed>::NONE; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
              || > Invalid number of channels in input image:
              || >     'VScn::contains(scn)'
              || > where
              || >     'scn' is 120
              ||
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5616311278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:39
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5614111278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5610811151Dishank Bansal 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simfailedyesnogpu-prod-060:03:04
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5607811282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:35
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5606511282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5604211284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5602611290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:37
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5601911290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5600111290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5597911281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:02:20
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5595311289Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:43
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5593711301Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:35
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5590911283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:46
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5586411286Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5583711302Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:49
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5581411280Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5573211291Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:02:10
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5570511450Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-testingLFt-simhost-erroryesnogpu-prod-060:00:36
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5565711308Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:02:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5560411299Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5558811311Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:36
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5555011285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:01:27
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5552011287Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:31
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5547811303Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:33
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5543111309Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:02:11
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5540811296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:55
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5535511305Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:02:11
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5532411306Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5529711310Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:33
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5525811307Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:55
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5524211424Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5521911424Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5500511413Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LFP-sim-validationLFP-simsuccessnonogpu-prod-060:14:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median8.025
in-drivable-lane_median0.5250000000000075
driven_lanedir_consec_median1.5425060846496597
deviation-center-line_median0.3402907324174555


other stats
agent_compute-ego0_max0.051712997143085186
agent_compute-ego0_mean0.04800970943189167
agent_compute-ego0_median0.047288759820238
agent_compute-ego0_min0.0457483209440055
complete-iteration_max0.32686875416682315
complete-iteration_mean0.27159396010815645
complete-iteration_median0.2697128892792109
complete-iteration_min0.2200813077073808
deviation-center-line_max3.9835052701742866
deviation-center-line_mean1.2333248413974425
deviation-center-line_min0.2692126305805727
deviation-heading_max15.44160972654172
deviation-heading_mean4.8278744100420585
deviation-heading_median1.5094336864352862
deviation-heading_min0.851020540755943
driven_any_max14.5898100863319
driven_any_mean4.633620267778668
driven_any_median1.7082199312507167
driven_any_min0.5282311222813394
driven_lanedir_consec_max11.088425839240465
driven_lanedir_consec_mean3.666387674925479
driven_lanedir_consec_min0.49211269116213174
driven_lanedir_max11.088425839240465
driven_lanedir_mean3.666387674925479
driven_lanedir_median1.5425060846496597
driven_lanedir_min0.49211269116213174
get_duckie_state_max0.026059958797234755
get_duckie_state_mean0.018622034989791755
get_duckie_state_median0.02174334207107391
get_duckie_state_min0.004941497019784437
get_robot_state_max0.004119859944771569
get_robot_state_mean0.003916430641514952
get_robot_state_median0.0038848109069875257
get_robot_state_min0.0037762408073131856
get_state_dump_max0.009145099383134108
get_state_dump_mean0.00800738349859055
get_state_dump_median0.008443608969749017
get_state_dump_min0.005997216671730061
get_ui_image_max0.03888417023878831
get_ui_image_mean0.03319550938862686
get_ui_image_median0.03306224391512249
get_ui_image_min0.02777337948547414
in-drivable-lane_max11.099999999999756
in-drivable-lane_mean3.037499999999943
in-drivable-lane_min0.0
per-episodes
details{"LFP-norm-loop-000-ego0": {"driven_any": 2.276806063496427, "get_ui_image": 0.0304536372423172, "step_physics": 0.11625494406773494, "survival_time": 10.350000000000012, "driven_lanedir": 1.9665277864594513, "get_state_dump": 0.009145099383134108, "get_robot_state": 0.0037762408073131856, "sim_render-ego0": 0.00396895523254688, "get_duckie_state": 0.026059958797234755, "in-drivable-lane": 1.050000000000015, "deviation-heading": 1.747008755262681, "agent_compute-ego0": 0.04704454655830677, "complete-iteration": 0.2503119833194293, "set_robot_commands": 0.0036573604895518376, "deviation-center-line": 0.3724996356547582, "driven_lanedir_consec": 1.9665277864594513, "sim_compute_sim_state": 0.007761038266695463, "sim_compute_performance-ego0": 0.0020781996158453133}, "LFP-norm-zigzag-000-ego0": {"driven_any": 0.5282311222813394, "get_ui_image": 0.03888417023878831, "step_physics": 0.18370141249436597, "survival_time": 3.1999999999999966, "driven_lanedir": 0.49211269116213174, "get_state_dump": 0.008301107700054463, "get_robot_state": 0.003783402076134315, "sim_render-ego0": 0.003994974723229042, "get_duckie_state": 0.02148960553682767, "in-drivable-lane": 0.0, "deviation-heading": 1.2718586176078914, "agent_compute-ego0": 0.051712997143085186, "complete-iteration": 0.32686875416682315, "set_robot_commands": 0.002399411568274865, "deviation-center-line": 0.2692126305805727, "driven_lanedir_consec": 0.49211269116213174, "sim_compute_sim_state": 0.010409960379967324, "sim_compute_performance-ego0": 0.002093227093036358}, "LFP-norm-techtrack-000-ego0": {"driven_any": 1.1396337990050067, "get_ui_image": 0.03567085058792778, "step_physics": 0.14884069069572117, "survival_time": 5.699999999999988, "driven_lanedir": 1.118484382839868, "get_state_dump": 0.008586110239443572, "get_robot_state": 0.003986219737840736, "sim_render-ego0": 0.004202231116916822, "get_duckie_state": 0.02199707860532014, "in-drivable-lane": 0.0, "deviation-heading": 0.851020540755943, "agent_compute-ego0": 0.04753297308216924, "complete-iteration": 0.28911379523899244, "set_robot_commands": 0.002446299013884171, "deviation-center-line": 0.3080818291801528, "driven_lanedir_consec": 1.118484382839868, "sim_compute_sim_state": 0.01358689017917799, "sim_compute_performance-ego0": 0.002152084267657736}, "LFP-norm-small_loop-000-ego0": {"driven_any": 14.5898100863319, "get_ui_image": 0.02777337948547414, "step_physics": 0.11575420710764558, "survival_time": 59.99999999999873, "driven_lanedir": 11.088425839240465, "get_state_dump": 0.005997216671730061, "get_robot_state": 0.004119859944771569, "sim_render-ego0": 0.004185639055047206, "get_duckie_state": 0.004941497019784437, "in-drivable-lane": 11.099999999999756, "deviation-heading": 15.44160972654172, "agent_compute-ego0": 0.0457483209440055, "complete-iteration": 0.2200813077073808, "set_robot_commands": 0.0025354724045498584, "deviation-center-line": 3.9835052701742866, "driven_lanedir_consec": 11.088425839240465, "sim_compute_sim_state": 0.006639027972701785, "sim_compute_performance-ego0": 0.002284145077301203}}
set_robot_commands_max0.0036573604895518376
set_robot_commands_mean0.0027596358690651825
set_robot_commands_median0.002490885709217015
set_robot_commands_min0.002399411568274865
sim_compute_performance-ego0_max0.002284145077301203
sim_compute_performance-ego0_mean0.0021519140134601527
sim_compute_performance-ego0_median0.002122655680347047
sim_compute_performance-ego0_min0.0020781996158453133
sim_compute_sim_state_max0.01358689017917799
sim_compute_sim_state_mean0.00959922919963564
sim_compute_sim_state_median0.009085499323331392
sim_compute_sim_state_min0.006639027972701785
sim_render-ego0_max0.004202231116916822
sim_render-ego0_mean0.0040879500319349876
sim_render-ego0_median0.004090306889138124
sim_render-ego0_min0.00396895523254688
simulation-passed1
step_physics_max0.18370141249436597
step_physics_mean0.14113781359136693
step_physics_median0.13254781738172805
step_physics_min0.11575420710764558
survival_time_max59.99999999999873
survival_time_mean19.81249999999968
survival_time_min3.1999999999999966
No reset possible
5481711360Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:22:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.7686538311627356
survival_time_median46.524999999999494
deviation-center-line_median0.8621187581074969
in-drivable-lane_median13.450000000000111


other stats
agent_compute-ego0_max0.012886242203842118
agent_compute-ego0_mean0.0120705989995071
agent_compute-ego0_median0.011979730258449369
agent_compute-ego0_min0.011436693277287543
complete-iteration_max0.21299368069495683
complete-iteration_mean0.1818740106562239
complete-iteration_median0.1804522494191116
complete-iteration_min0.15359786309171578
deviation-center-line_max4.22087868113322
deviation-center-line_mean1.5479463856004347
deviation-center-line_min0.2466693450535255
deviation-heading_max11.923205613936371
deviation-heading_mean4.757432690090893
deviation-heading_median2.8869699497265535
deviation-heading_min1.332585246974093
driven_any_max11.039442009336137
driven_any_mean6.645349810449424
driven_any_median7.442542653910984
driven_any_min0.6568719246395952
driven_lanedir_consec_max9.83401734781271
driven_lanedir_consec_mean3.917206763446971
driven_lanedir_consec_min0.2975020436497058
driven_lanedir_max9.83401734781271
driven_lanedir_mean3.917206763446971
driven_lanedir_median2.7686538311627356
driven_lanedir_min0.2975020436497058
get_duckie_state_max1.4708481526446848e-06
get_duckie_state_mean1.406714538409306e-06
get_duckie_state_median1.399177081064302e-06
get_duckie_state_min1.357655838863935e-06
get_robot_state_max0.0040815514019969
get_robot_state_mean0.003795737850542854
get_robot_state_median0.0037243556180290823
get_robot_state_min0.003652688764116349
get_state_dump_max0.005129851963584877
get_state_dump_mean0.004960319401926597
get_state_dump_median0.0049457290886477335
get_state_dump_min0.004819967466826046
get_ui_image_max0.034177048706714015
get_ui_image_mean0.029955045128171515
get_ui_image_median0.03034075270265715
get_ui_image_min0.024961626400657737
in-drivable-lane_max41.34999999999917
in-drivable-lane_mean17.34999999999985
in-drivable-lane_min1.149999999999996
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.039442009336137, "get_ui_image": 0.027222965678803432, "step_physics": 0.0943644979812025, "survival_time": 59.99999999999873, "driven_lanedir": 3.6984620783016937, "get_state_dump": 0.004955987549145752, "get_robot_state": 0.003723273765633843, "sim_render-ego0": 0.003848701988429848, "get_duckie_state": 1.398748799624987e-06, "in-drivable-lane": 41.34999999999917, "deviation-heading": 3.4444645675466714, "agent_compute-ego0": 0.011905086328346068, "complete-iteration": 0.1591754878788169, "set_robot_commands": 0.0022591760414625386, "deviation-center-line": 0.942552546192519, "driven_lanedir_consec": 3.6984620783016937, "sim_compute_sim_state": 0.008743482267330528, "sim_compute_performance-ego0": 0.002058130021297763}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6568719246395952, "get_ui_image": 0.034177048706714015, "step_physics": 0.13904085277039327, "survival_time": 8.04999999999998, "driven_lanedir": 0.2975020436497058, "get_state_dump": 0.004935470628149715, "get_robot_state": 0.0037254374704243226, "sim_render-ego0": 0.00387088163399402, "get_duckie_state": 1.3996053625036169e-06, "in-drivable-lane": 4.549999999999984, "deviation-heading": 1.332585246974093, "agent_compute-ego0": 0.012054374188552668, "complete-iteration": 0.21299368069495683, "set_robot_commands": 0.0023134004922560703, "deviation-center-line": 0.2466693450535255, "driven_lanedir_consec": 0.2975020436497058, "sim_compute_sim_state": 0.010725979451779966, "sim_compute_performance-ego0": 0.0020564514913676696}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.674895745610957, "get_ui_image": 0.03345853972651087, "step_physics": 0.1261820948015887, "survival_time": 33.05000000000026, "driven_lanedir": 1.8388455840237776, "get_state_dump": 0.005129851963584877, "get_robot_state": 0.0040815514019969, "sim_render-ego0": 0.00418178624617009, "get_duckie_state": 1.4708481526446848e-06, "in-drivable-lane": 22.350000000000243, "deviation-heading": 2.3294753319064356, "agent_compute-ego0": 0.012886242203842118, "complete-iteration": 0.20172901095940624, "set_robot_commands": 0.0024922686401090592, "deviation-center-line": 0.7816849700224747, "driven_lanedir_consec": 1.8388455840237776, "sim_compute_sim_state": 0.010906393074196993, "sim_compute_performance-ego0": 0.002308298093674766}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.21018956221101, "get_ui_image": 0.024961626400657737, "step_physics": 0.09456580147755136, "survival_time": 59.99999999999873, "driven_lanedir": 9.83401734781271, "get_state_dump": 0.004819967466826046, "get_robot_state": 0.003652688764116349, "sim_render-ego0": 0.003779025399416909, "get_duckie_state": 1.357655838863935e-06, "in-drivable-lane": 1.149999999999996, "deviation-heading": 11.923205613936371, "agent_compute-ego0": 0.011436693277287543, "complete-iteration": 0.15359786309171578, "set_robot_commands": 0.002214167536942786, "deviation-center-line": 4.22087868113322, "driven_lanedir_consec": 9.83401734781271, "sim_compute_sim_state": 0.006086298468507994, "sim_compute_performance-ego0": 0.00199111435038164}}
set_robot_commands_max0.0024922686401090592
set_robot_commands_mean0.0023197531776926136
set_robot_commands_median0.0022862882668593047
set_robot_commands_min0.002214167536942786
sim_compute_performance-ego0_max0.002308298093674766
sim_compute_performance-ego0_mean0.0021034984891804597
sim_compute_performance-ego0_median0.0020572907563327163
sim_compute_performance-ego0_min0.00199111435038164
sim_compute_sim_state_max0.010906393074196993
sim_compute_sim_state_mean0.009115538315453867
sim_compute_sim_state_median0.009734730859555249
sim_compute_sim_state_min0.006086298468507994
sim_render-ego0_max0.00418178624617009
sim_render-ego0_mean0.003920098817002717
sim_render-ego0_median0.003859791811211934
sim_render-ego0_min0.003779025399416909
simulation-passed1
step_physics_max0.13904085277039327
step_physics_mean0.11353831175768396
step_physics_median0.11037394813957004
step_physics_min0.0943644979812025
survival_time_max59.99999999999873
survival_time_mean40.27499999999942
survival_time_min8.04999999999998
No reset possible
5477911365brian wongtemplate-randomaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:05:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5040495128145347
survival_time_median4.6499999999999915
deviation-center-line_median0.10620228737371736
in-drivable-lane_median2.674999999999992


other stats
agent_compute-ego0_max0.012033058537377251
agent_compute-ego0_mean0.011224470358317556
agent_compute-ego0_median0.01101609267372316
agent_compute-ego0_min0.010832637548446655
complete-iteration_max0.21439606210459833
complete-iteration_mean0.184523065175752
complete-iteration_median0.18238485333118087
complete-iteration_min0.15892649193604788
deviation-center-line_max0.16455763165774434
deviation-center-line_mean0.1092777488823554
deviation-center-line_min0.06014878912424249
deviation-heading_max1.819139548495243
deviation-heading_mean0.860079214882594
deviation-heading_median0.6517793617099584
deviation-heading_min0.31761858761521594
driven_any_max3.078535752374254
driven_any_mean1.8526223552312917
driven_any_median1.6219947547238958
driven_any_min1.0879641591031215
driven_lanedir_consec_max0.6643282900394009
driven_lanedir_consec_mean0.4884432809313369
driven_lanedir_consec_min0.28134580805687714
driven_lanedir_max0.6643282900394009
driven_lanedir_mean0.4924956463812869
driven_lanedir_median0.5121542437144349
driven_lanedir_min0.28134580805687714
get_duckie_state_max2.565591231636379e-06
get_duckie_state_mean2.3801521670524554e-06
get_duckie_state_median2.3930353202900304e-06
get_duckie_state_min2.168946795993381e-06
get_robot_state_max0.003808350666709568
get_robot_state_mean0.003762403232314452
get_robot_state_median0.0037723597553041247
get_robot_state_min0.003696542751939991
get_state_dump_max0.005031282487122909
get_state_dump_mean0.004815012010632542
get_state_dump_median0.004771295520994399
get_state_dump_min0.0046861745134184635
get_ui_image_max0.03599235804184624
get_ui_image_mean0.030331108388907575
get_ui_image_median0.029611402381153885
get_ui_image_min0.026109270751476288
in-drivable-lane_max6.549999999999982
in-drivable-lane_mean3.3499999999999894
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.078535752374254, "get_ui_image": 0.027869695349584653, "step_physics": 0.1027777421323559, "survival_time": 7.84999999999998, "driven_lanedir": 0.28134580805687714, "get_state_dump": 0.0046861745134184635, "get_robot_state": 0.003696542751939991, "sim_render-ego0": 0.003744626347022721, "get_duckie_state": 2.3298625704608387e-06, "in-drivable-lane": 6.549999999999982, "deviation-heading": 0.891006057417327, "agent_compute-ego0": 0.01089809212503554, "complete-iteration": 0.16754001001768473, "set_robot_commands": 0.002136751066280317, "deviation-center-line": 0.13094886067496936, "driven_lanedir_consec": 0.28134580805687714, "sim_compute_sim_state": 0.009656184836278989, "sim_compute_performance-ego0": 0.001983959463578236}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5806581102909054, "get_ui_image": 0.03599235804184624, "step_physics": 0.1408755468285602, "survival_time": 4.549999999999992, "driven_lanedir": 0.3891269085945981, "get_state_dump": 0.005031282487122909, "get_robot_state": 0.003808350666709568, "sim_render-ego0": 0.003844761330148448, "get_duckie_state": 2.565591231636379e-06, "in-drivable-lane": 2.449999999999993, "deviation-heading": 1.819139548495243, "agent_compute-ego0": 0.011134093222410782, "complete-iteration": 0.21439606210459833, "set_robot_commands": 0.002272416716036589, "deviation-center-line": 0.16455763165774434, "driven_lanedir_consec": 0.37291744679479777, "sim_compute_sim_state": 0.009281383908313252, "sim_compute_performance-ego0": 0.002057946246603261}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0879641591031215, "get_ui_image": 0.03135310941272312, "step_physics": 0.1295608811908298, "survival_time": 3.5499999999999954, "driven_lanedir": 0.6643282900394009, "get_state_dump": 0.004742589261796739, "get_robot_state": 0.003795408540301853, "sim_render-ego0": 0.0038661791218651664, "get_duckie_state": 2.168946795993381e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.41255266600258983, "agent_compute-ego0": 0.012033058537377251, "complete-iteration": 0.19722969664467704, "set_robot_commands": 0.00219230850537618, "deviation-center-line": 0.08145571407246535, "driven_lanedir_consec": 0.6643282900394009, "sim_compute_sim_state": 0.007606297731399536, "sim_compute_performance-ego0": 0.0019896096653408473}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6633313991568863, "get_ui_image": 0.026109270751476288, "step_physics": 0.10043163845936458, "survival_time": 4.749999999999991, "driven_lanedir": 0.6351815788342716, "get_state_dump": 0.004800001780192058, "get_robot_state": 0.003749310970306397, "sim_render-ego0": 0.003739096224308014, "get_duckie_state": 2.456208070119222e-06, "in-drivable-lane": 2.89999999999999, "deviation-heading": 0.31761858761521594, "agent_compute-ego0": 0.010832637548446655, "complete-iteration": 0.15892649193604788, "set_robot_commands": 0.00221403439839681, "deviation-center-line": 0.06014878912424249, "driven_lanedir_consec": 0.6351815788342716, "sim_compute_sim_state": 0.005021862685680389, "sim_compute_performance-ego0": 0.0019369199872016909}}
set_robot_commands_max0.002272416716036589
set_robot_commands_mean0.0022038776715224743
set_robot_commands_median0.002203171451886495
set_robot_commands_min0.002136751066280317
sim_compute_performance-ego0_max0.002057946246603261
sim_compute_performance-ego0_mean0.001992108840681009
sim_compute_performance-ego0_median0.0019867845644595417
sim_compute_performance-ego0_min0.0019369199872016909
sim_compute_sim_state_max0.009656184836278989
sim_compute_sim_state_mean0.007891432290418042
sim_compute_sim_state_median0.008443840819856396
sim_compute_sim_state_min0.005021862685680389
sim_render-ego0_max0.0038661791218651664
sim_render-ego0_mean0.003798665755836087
sim_render-ego0_median0.003794693838585584
sim_render-ego0_min0.003739096224308014
simulation-passed1
step_physics_max0.1408755468285602
step_physics_mean0.11841145215277764
step_physics_median0.11616931166159286
step_physics_min0.10043163845936458
survival_time_max7.84999999999998
survival_time_mean5.17499999999999
survival_time_min3.5499999999999954
No reset possible
5476611436Charlie Gauthier 🇨🇦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:51
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5475211436Charlie Gauthier 🇨🇦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:00:49
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5467111395Charlie Gauthier 🇨🇦Baseline solution using reinforcement learningaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:11:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4953415167970959
survival_time_median3.8749999999999942
deviation-center-line_median0.10019431734994318
in-drivable-lane_median2.5999999999999948


other stats
agent_compute-ego0_max0.01667423232409678
agent_compute-ego0_mean0.01611473274915348
agent_compute-ego0_median0.01612591311214416
agent_compute-ego0_min0.01553287244822881
complete-iteration_max0.26725945530868156
complete-iteration_mean0.22263533879517464
complete-iteration_median0.2157655048805829
complete-iteration_min0.1917508901108512
deviation-center-line_max1.3347053459531408
deviation-center-line_mean0.39649251993821305
deviation-center-line_min0.050876099099825205
deviation-heading_max7.092505358170257
deviation-heading_mean2.1832773338209277
deviation-heading_median0.6958540662065884
deviation-heading_min0.24889584470027704
driven_any_max32.21451956876099
driven_any_mean9.164620243328644
driven_any_median1.53081754802698
driven_any_min1.3823263084996291
driven_lanedir_consec_max9.151844330275928
driven_lanedir_consec_mean2.6196278682405016
driven_lanedir_consec_min0.3359841090918858
driven_lanedir_max10.09337842757754
driven_lanedir_mean2.85860808999487
driven_lanedir_median0.5025349116550267
driven_lanedir_min0.3359841090918858
get_duckie_state_max1.4014360381335749e-06
get_duckie_state_mean1.341851214077448e-06
get_duckie_state_median1.3450509336985411e-06
get_duckie_state_min1.275866950779136e-06
get_robot_state_max0.003958795128799066
get_robot_state_mean0.003781851425056325
get_robot_state_median0.003767939942068161
get_robot_state_min0.003632730687289909
get_state_dump_max0.00493578503771526
get_state_dump_mean0.004758805788164382
get_state_dump_median0.004752783448728797
get_state_dump_min0.004593871217484676
get_ui_image_max0.03762520813360447
get_ui_image_mean0.03154139638816928
get_ui_image_median0.0305827582372378
get_ui_image_min0.02737486094459705
in-drivable-lane_max39.1499999999992
in-drivable-lane_mean11.612499999999796
in-drivable-lane_min2.0999999999999934
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 32.21451956876099, "get_ui_image": 0.02737486094459705, "step_physics": 0.12354233640119694, "survival_time": 59.99999999999873, "driven_lanedir": 10.09337842757754, "get_state_dump": 0.004593871217484676, "get_robot_state": 0.003632730687289909, "sim_render-ego0": 0.0036992857994187583, "get_duckie_state": 1.275866950779136e-06, "in-drivable-lane": 39.1499999999992, "deviation-heading": 7.092505358170257, "agent_compute-ego0": 0.01667423232409678, "complete-iteration": 0.1917508901108512, "set_robot_commands": 0.0023409712820823343, "deviation-center-line": 1.3347053459531408, "driven_lanedir_consec": 9.151844330275928, "sim_compute_sim_state": 0.007857668906028424, "sim_compute_performance-ego0": 0.0019520115197250783}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.6253119880518745, "get_ui_image": 0.03762520813360447, "step_physics": 0.18549586505424687, "survival_time": 4.049999999999994, "driven_lanedir": 0.6188196930254084, "get_state_dump": 0.00493578503771526, "get_robot_state": 0.003958795128799066, "sim_render-ego0": 0.0039470748203556715, "get_duckie_state": 1.4014360381335749e-06, "in-drivable-lane": 2.0999999999999934, "deviation-heading": 1.1348402688492234, "agent_compute-ego0": 0.016661123531620678, "complete-iteration": 0.26725945530868156, "set_robot_commands": 0.0024794107530175186, "deviation-center-line": 0.14647640362993186, "driven_lanedir_consec": 0.6044329033095468, "sim_compute_sim_state": 0.009894539670246404, "sim_compute_performance-ego0": 0.0021701440578553734}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.436323108002086, "get_ui_image": 0.03228041648864746, "step_physics": 0.15720373789469402, "survival_time": 3.699999999999995, "driven_lanedir": 0.38625013028464505, "get_state_dump": 0.004835329055786133, "get_robot_state": 0.003852094014485677, "sim_render-ego0": 0.0037900129954020183, "get_duckie_state": 1.3510386149088542e-06, "in-drivable-lane": 2.5999999999999943, "deviation-heading": 0.25686786356395347, "agent_compute-ego0": 0.015590702692667643, "complete-iteration": 0.2297046724955241, "set_robot_commands": 0.0023685296376546225, "deviation-center-line": 0.050876099099825205, "driven_lanedir_consec": 0.38625013028464505, "sim_compute_sim_state": 0.007695169448852539, "sim_compute_performance-ego0": 0.002002925872802734}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.3823263084996291, "get_ui_image": 0.02888509998582814, "step_physics": 0.13572599136666075, "survival_time": 3.599999999999995, "driven_lanedir": 0.3359841090918858, "get_state_dump": 0.004670237841671461, "get_robot_state": 0.0036837858696506447, "sim_render-ego0": 0.003609396006963024, "get_duckie_state": 1.3390632524882278e-06, "in-drivable-lane": 2.599999999999995, "deviation-heading": 0.24889584470027704, "agent_compute-ego0": 0.01553287244822881, "complete-iteration": 0.20182633726564173, "set_robot_commands": 0.0024748338411932124, "deviation-center-line": 0.053912231069954485, "driven_lanedir_consec": 0.3359841090918858, "sim_compute_sim_state": 0.005266398599703018, "sim_compute_performance-ego0": 0.001892419710551223}}
set_robot_commands_max0.0024794107530175186
set_robot_commands_mean0.002415936378486922
set_robot_commands_median0.0024216817394239172
set_robot_commands_min0.0023409712820823343
sim_compute_performance-ego0_max0.0021701440578553734
sim_compute_performance-ego0_mean0.0020043752902336024
sim_compute_performance-ego0_median0.0019774686962639064
sim_compute_performance-ego0_min0.001892419710551223
sim_compute_sim_state_max0.009894539670246404
sim_compute_sim_state_mean0.007678444156207596
sim_compute_sim_state_median0.007776419177440482
sim_compute_sim_state_min0.005266398599703018
sim_render-ego0_max0.0039470748203556715
sim_render-ego0_mean0.003761442405534868
sim_render-ego0_median0.003744649397410389
sim_render-ego0_min0.003609396006963024
simulation-passed1
step_physics_max0.18549586505424687
step_physics_mean0.15049198267919964
step_physics_median0.14646486463067737
step_physics_min0.12354233640119694
survival_time_max59.99999999999873
survival_time_mean17.83749999999968
survival_time_min3.599999999999995
No reset possible
5456411445Yishu Malhotra 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:16:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.7724884155871918
survival_time_median22.85000000000019
deviation-center-line_median0.33002959627002537
in-drivable-lane_median18.600000000000176


other stats
agent_compute-ego0_max0.050086300389296345
agent_compute-ego0_mean0.03471220037466802
agent_compute-ego0_median0.037803172522381315
agent_compute-ego0_min0.013156156064613072
complete-iteration_max0.214928650707932
complete-iteration_mean0.19904486192938223
complete-iteration_median0.19697569135470472
complete-iteration_min0.1872994143001876
deviation-center-line_max0.9363462383037952
deviation-center-line_mean0.43698829402633904
deviation-center-line_min0.15154774526151024
deviation-heading_max5.758749968369324
deviation-heading_mean2.9259937355544774
deviation-heading_median2.703339847563921
deviation-heading_min0.5385452787207428
driven_any_max12.249161255972346
driven_any_mean5.660607320457167
driven_any_median4.531555317555973
driven_any_min1.330157390744377
driven_lanedir_consec_max3.461535356198567
driven_lanedir_consec_mean1.4250501759879888
driven_lanedir_consec_min0.6936885165790048
driven_lanedir_max3.661085732128009
driven_lanedir_mean1.4778857473613796
driven_lanedir_median0.7783843703692528
driven_lanedir_min0.6936885165790048
get_duckie_state_max1.56888624479533e-06
get_duckie_state_mean1.3452259326201485e-06
get_duckie_state_median1.2855280928826651e-06
get_duckie_state_min1.240961299919934e-06
get_robot_state_max0.004005779407415359
get_robot_state_mean0.003660937902180928
get_robot_state_median0.003616195425698227
get_robot_state_min0.003405581349911897
get_state_dump_max0.006249258602063219
get_state_dump_mean0.005394584706477056
get_state_dump_median0.0054790710235754405
get_state_dump_min0.004370938176694123
get_ui_image_max0.03405147158447953
get_ui_image_mean0.029288190253945863
get_ui_image_median0.029274548712255787
get_ui_image_min0.02455219200679234
in-drivable-lane_max38.44999999999946
in-drivable-lane_mean19.76249999999995
in-drivable-lane_min3.399999999999988
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.249161255972346, "get_ui_image": 0.027098273457010335, "step_physics": 0.09679789710840306, "survival_time": 55.39999999999899, "driven_lanedir": 3.661085732128009, "get_state_dump": 0.006249258602063219, "get_robot_state": 0.003741985833569622, "sim_render-ego0": 0.0038868753194164006, "get_duckie_state": 1.3157093815162871e-06, "in-drivable-lane": 38.44999999999946, "deviation-heading": 5.758749968369324, "agent_compute-ego0": 0.03565348344842414, "complete-iteration": 0.1872994143001876, "set_robot_commands": 0.002551811895895047, "deviation-center-line": 0.9363462383037952, "driven_lanedir_consec": 3.461535356198567, "sim_compute_sim_state": 0.009111285962103937, "sim_compute_performance-ego0": 0.0021208925651364547}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.330157390744377, "get_ui_image": 0.03145082396750124, "step_physics": 0.1192127666117982, "survival_time": 7.99999999999998, "driven_lanedir": 0.7653589709187589, "get_state_dump": 0.004370938176694123, "get_robot_state": 0.003405581349911897, "sim_render-ego0": 0.0035162786519305307, "get_duckie_state": 1.240961299919934e-06, "in-drivable-lane": 3.399999999999988, "deviation-heading": 1.3927384462559225, "agent_compute-ego0": 0.0399528615963385, "complete-iteration": 0.214928650707932, "set_robot_commands": 0.0020026153659228212, "deviation-center-line": 0.2758653129249678, "driven_lanedir_consec": 0.7653589709187589, "sim_compute_sim_state": 0.009082789746870906, "sim_compute_performance-ego0": 0.001853112108218744}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.966521494163616, "get_ui_image": 0.03405147158447953, "step_physics": 0.12461106915182622, "survival_time": 31.050000000000303, "driven_lanedir": 0.7914097698197466, "get_state_dump": 0.0051917113675181885, "get_robot_state": 0.004005779407415359, "sim_render-ego0": 0.004125420303590045, "get_duckie_state": 1.56888624479533e-06, "in-drivable-lane": 25.55000000000027, "deviation-heading": 4.0139412488719195, "agent_compute-ego0": 0.013156156064613072, "complete-iteration": 0.20338975079956545, "set_robot_commands": 0.002335105294966621, "deviation-center-line": 0.384193879615083, "driven_lanedir_consec": 0.7796178602556247, "sim_compute_sim_state": 0.013604709956423646, "sim_compute_performance-ego0": 0.0022080136265402056}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.0965891409483293, "get_ui_image": 0.02455219200679234, "step_physics": 0.09351331441580844, "survival_time": 14.650000000000071, "driven_lanedir": 0.6936885165790048, "get_state_dump": 0.005766430679632693, "get_robot_state": 0.0034904050178268327, "sim_render-ego0": 0.00360166783235511, "get_duckie_state": 1.2553468042490434e-06, "in-drivable-lane": 11.650000000000077, "deviation-heading": 0.5385452787207428, "agent_compute-ego0": 0.050086300389296345, "complete-iteration": 0.190561631909844, "set_robot_commands": 0.002194279716128395, "deviation-center-line": 0.15154774526151024, "driven_lanedir_consec": 0.6936885165790048, "sim_compute_sim_state": 0.00536812973671219, "sim_compute_performance-ego0": 0.0019055021052457849}}
set_robot_commands_max0.002551811895895047
set_robot_commands_mean0.002270953068228221
set_robot_commands_median0.002264692505547508
set_robot_commands_min0.0020026153659228212
sim_compute_performance-ego0_max0.0022080136265402056
sim_compute_performance-ego0_mean0.002021880101285297
sim_compute_performance-ego0_median0.0020131973351911196
sim_compute_performance-ego0_min0.001853112108218744
sim_compute_sim_state_max0.013604709956423646
sim_compute_sim_state_mean0.00929172885052767
sim_compute_sim_state_median0.00909703785448742
sim_compute_sim_state_min0.00536812973671219
sim_render-ego0_max0.004125420303590045
sim_render-ego0_mean0.0037825605268230218
sim_render-ego0_median0.0037442715758857553
sim_render-ego0_min0.0035162786519305307
simulation-passed1
step_physics_max0.12461106915182622
step_physics_mean0.10853376182195898
step_physics_median0.10800533186010064
step_physics_min0.09351331441580844
survival_time_max55.39999999999899
survival_time_mean27.27499999999984
survival_time_min7.99999999999998
No reset possible
5452211466Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:17:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.196412243228877
survival_time_median19.225000000000136
deviation-center-line_median0.3724895589750083
in-drivable-lane_median14.075000000000118


other stats
agent_compute-ego0_max0.012727564121662429
agent_compute-ego0_mean0.012458570561198949
agent_compute-ego0_median0.0125013708878129
agent_compute-ego0_min0.012103976347507575
complete-iteration_max0.21970284249069943
complete-iteration_mean0.1954112631557551
complete-iteration_median0.197308808349177
complete-iteration_min0.167324593433967
deviation-center-line_max0.8636606485653999
deviation-center-line_mean0.43816108934054554
deviation-center-line_min0.1440045908467656
deviation-heading_max6.836488078602638
deviation-heading_mean2.4904311000059445
deviation-heading_median1.2443962402727151
deviation-heading_min0.6364438408757102
driven_any_max11.493818771442896
driven_any_mean5.253634525014082
driven_any_median3.90913475526179
driven_any_min1.7024498180898548
driven_lanedir_consec_max2.1303351033894367
driven_lanedir_consec_mean1.3047060738422744
driven_lanedir_consec_min0.6956647055219061
driven_lanedir_max2.1303351033894367
driven_lanedir_mean1.3254754407717697
driven_lanedir_median1.2379509770878685
driven_lanedir_min0.6956647055219061
get_duckie_state_max1.3036605639335437e-06
get_duckie_state_mean1.2846090159897533e-06
get_duckie_state_median1.2890662981009748e-06
get_duckie_state_min1.2566429038235194e-06
get_robot_state_max0.0037536730277945257
get_robot_state_mean0.003667619990946744
get_robot_state_median0.003664648274011041
get_robot_state_min0.0035875103879703706
get_state_dump_max0.0046614403530124025
get_state_dump_mean0.004599979500236188
get_state_dump_median0.004596812002186699
get_state_dump_min0.004544853643558952
get_ui_image_max0.0367534987600967
get_ui_image_mean0.03083950385430179
get_ui_image_median0.030625122952725382
get_ui_image_min0.025354270751659688
in-drivable-lane_max49.34999999999885
in-drivable-lane_mean19.98749999999978
in-drivable-lane_min2.450000000000035
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.5329041577112927, "get_ui_image": 0.028234847247860836, "step_physics": 0.11917326856388276, "survival_time": 11.400000000000029, "driven_lanedir": 2.1303351033894367, "get_state_dump": 0.004544853643558952, "get_robot_state": 0.0035875103879703706, "sim_render-ego0": 0.003782175513854714, "get_duckie_state": 1.2566429038235194e-06, "in-drivable-lane": 2.450000000000035, "deviation-heading": 1.675367084998529, "agent_compute-ego0": 0.012392140892395286, "complete-iteration": 0.1855805732277283, "set_robot_commands": 0.002230181964724345, "deviation-center-line": 0.5195134338257585, "driven_lanedir_consec": 2.1303351033894367, "sim_compute_sim_state": 0.009546615150818135, "sim_compute_performance-ego0": 0.0020058977551855894}, "LF-norm-zigzag-000-ego0": {"driven_any": 5.285365352812287, "get_ui_image": 0.0367534987600967, "step_physics": 0.14280273069754737, "survival_time": 27.05000000000025, "driven_lanedir": 0.7443584622791026, "get_state_dump": 0.004632934872954534, "get_robot_state": 0.003701753282019133, "sim_render-ego0": 0.003950740138543048, "get_duckie_state": 1.285788757774663e-06, "in-drivable-lane": 23.300000000000253, "deviation-heading": 0.8134253955469012, "agent_compute-ego0": 0.012610600883230513, "complete-iteration": 0.21970284249069943, "set_robot_commands": 0.0022372675997744625, "deviation-center-line": 0.22546568412425816, "driven_lanedir_consec": 0.7443584622791026, "sim_compute_sim_state": 0.010882123370012237, "sim_compute_performance-ego0": 0.0020464685130383254}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.493818771442896, "get_ui_image": 0.03301539865758993, "step_physics": 0.13528046024332038, "survival_time": 59.99999999999873, "driven_lanedir": 1.731543491896634, "get_state_dump": 0.0046614403530124025, "get_robot_state": 0.0037536730277945257, "sim_render-ego0": 0.0038863085985779266, "get_duckie_state": 1.2923438384272871e-06, "in-drivable-lane": 49.34999999999885, "deviation-heading": 6.836488078602638, "agent_compute-ego0": 0.012727564121662429, "complete-iteration": 0.20903704347062568, "set_robot_commands": 0.0022851239632408786, "deviation-center-line": 0.8636606485653999, "driven_lanedir_consec": 1.6484660241786515, "sim_compute_sim_state": 0.011284624706398535, "sim_compute_performance-ego0": 0.0020569383254356926}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.7024498180898548, "get_ui_image": 0.025354270751659688, "step_physics": 0.10889944663414589, "survival_time": 7.7499999999999805, "driven_lanedir": 0.6956647055219061, "get_state_dump": 0.004560689131418864, "get_robot_state": 0.003627543266002949, "sim_render-ego0": 0.003641223296141013, "get_duckie_state": 1.3036605639335437e-06, "in-drivable-lane": 4.849999999999983, "deviation-heading": 0.6364438408757102, "agent_compute-ego0": 0.012103976347507575, "complete-iteration": 0.167324593433967, "set_robot_commands": 0.002122628383147411, "deviation-center-line": 0.1440045908467656, "driven_lanedir_consec": 0.6956647055219061, "sim_compute_sim_state": 0.0048811099468133385, "sim_compute_performance-ego0": 0.0020551513402889935}}
set_robot_commands_max0.0022851239632408786
set_robot_commands_mean0.0022188004777217744
set_robot_commands_median0.0022337247822494036
set_robot_commands_min0.002122628383147411
sim_compute_performance-ego0_max0.0020569383254356926
sim_compute_performance-ego0_mean0.0020411139834871504
sim_compute_performance-ego0_median0.0020508099266636597
sim_compute_performance-ego0_min0.0020058977551855894
sim_compute_sim_state_max0.011284624706398535
sim_compute_sim_state_mean0.009148618293510562
sim_compute_sim_state_median0.010214369260415189
sim_compute_sim_state_min0.0048811099468133385
sim_render-ego0_max0.003950740138543048
sim_render-ego0_mean0.003815111886779176
sim_render-ego0_median0.003834242056216321
sim_render-ego0_min0.003641223296141013
simulation-passed1
step_physics_max0.14280273069754737
step_physics_mean0.1265389765347241
step_physics_median0.12722686440360156
step_physics_min0.10889944663414589
survival_time_max59.99999999999873
survival_time_mean26.54999999999975
survival_time_min7.7499999999999805
No reset possible
5447611520Olga (Ge Ya) Xu 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:16:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.6634955539927105
survival_time_median24.649999999999956
deviation-center-line_median1.0383308289401314
in-drivable-lane_median7.57499999999976


other stats
agent_compute-ego0_max0.013590381259009953
agent_compute-ego0_mean0.012450413547734948
agent_compute-ego0_median0.01218060106868704
agent_compute-ego0_min0.011850070794555766
complete-iteration_max0.21657865874621332
complete-iteration_mean0.18383152565280556
complete-iteration_median0.18005811175836822
complete-iteration_min0.1586312203482724
deviation-center-line_max3.9799819628134983
deviation-center-line_mean1.556286948963371
deviation-center-line_min0.16850417515972346
deviation-heading_max13.431307676478635
deviation-heading_mean5.35287797616083
deviation-heading_median3.638079283410697
deviation-heading_min0.7040456613432883
driven_any_max10.038017694588282
driven_any_mean5.1279144112081525
driven_any_median4.5993433790842655
driven_any_min1.2749531920757968
driven_lanedir_consec_max8.388318105998872
driven_lanedir_consec_mean3.5896248888710325
driven_lanedir_consec_min0.6431903414998374
driven_lanedir_max8.388318105998872
driven_lanedir_mean3.5896248888710325
driven_lanedir_median2.6634955539927105
driven_lanedir_min0.6431903414998374
get_duckie_state_max2.1782051138326423e-06
get_duckie_state_mean2.0571322855173274e-06
get_duckie_state_median2.0628427492332093e-06
get_duckie_state_min1.924638529770247e-06
get_robot_state_max0.004078454711810261
get_robot_state_mean0.003811277049205313
get_robot_state_median0.003765603323888732
get_robot_state_min0.0036354468372335265
get_state_dump_max0.005047457558768136
get_state_dump_mean0.004786225627402573
get_state_dump_median0.004746837807250432
get_state_dump_min0.004603769336341295
get_ui_image_max0.03557355215053747
get_ui_image_mean0.031143801953960673
get_ui_image_median0.03108632132142493
get_ui_image_min0.02682901302245535
in-drivable-lane_max17.649999999999846
in-drivable-lane_mean9.349999999999838
in-drivable-lane_min4.599999999999984
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.524511506882414, "get_ui_image": 0.027783883133614032, "step_physics": 0.09426741320971616, "survival_time": 39.24999999999991, "driven_lanedir": 4.661315323777423, "get_state_dump": 0.004603769336341295, "get_robot_state": 0.0036354468372335265, "sim_render-ego0": 0.0038396508335763863, "get_duckie_state": 1.924638529770247e-06, "in-drivable-lane": 17.649999999999846, "deviation-heading": 6.257779229707023, "agent_compute-ego0": 0.012020938572386142, "complete-iteration": 0.16013553731010768, "set_robot_commands": 0.002177762621231662, "deviation-center-line": 1.8921822273764308, "driven_lanedir_consec": 4.661315323777423, "sim_compute_sim_state": 0.009755073612882892, "sim_compute_performance-ego0": 0.0019588085227946897}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.6741752512861172, "get_ui_image": 0.03557355215053747, "step_physics": 0.12575204891733605, "survival_time": 10.050000000000008, "driven_lanedir": 0.6431903414998374, "get_state_dump": 0.004689167041589718, "get_robot_state": 0.003680329511661341, "sim_render-ego0": 0.003771532880197657, "get_duckie_state": 2.0902935821231047e-06, "in-drivable-lane": 6.500000000000012, "deviation-heading": 1.01837933711437, "agent_compute-ego0": 0.012340263564987936, "complete-iteration": 0.1999806862066288, "set_robot_commands": 0.0022262972180205997, "deviation-center-line": 0.18447943050383217, "driven_lanedir_consec": 0.6431903414998374, "sim_compute_sim_state": 0.009872619468386813, "sim_compute_performance-ego0": 0.001982016138511129}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.2749531920757968, "get_ui_image": 0.034388759509235825, "step_physics": 0.14110507283891952, "survival_time": 7.299999999999982, "driven_lanedir": 0.6656757842079988, "get_state_dump": 0.005047457558768136, "get_robot_state": 0.004078454711810261, "sim_render-ego0": 0.004183579464347995, "get_duckie_state": 2.1782051138326423e-06, "in-drivable-lane": 4.599999999999984, "deviation-heading": 0.7040456613432883, "agent_compute-ego0": 0.013590381259009953, "complete-iteration": 0.21657865874621332, "set_robot_commands": 0.0023959850778385084, "deviation-center-line": 0.16850417515972346, "driven_lanedir_consec": 0.6656757842079988, "sim_compute_sim_state": 0.009486678506241363, "sim_compute_performance-ego0": 0.0022006359230093407}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.038017694588282, "get_ui_image": 0.02682901302245535, "step_physics": 0.09659794089597629, "survival_time": 59.99999999999873, "driven_lanedir": 8.388318105998872, "get_state_dump": 0.004804508572911145, "get_robot_state": 0.003850877136116123, "sim_render-ego0": 0.0038882242849129223, "get_duckie_state": 2.0353919163433142e-06, "in-drivable-lane": 8.649999999999508, "deviation-heading": 13.431307676478635, "agent_compute-ego0": 0.011850070794555766, "complete-iteration": 0.1586312203482724, "set_robot_commands": 0.002335664930192755, "deviation-center-line": 3.9799819628134983, "driven_lanedir_consec": 8.388318105998872, "sim_compute_sim_state": 0.006319900634981611, "sim_compute_performance-ego0": 0.0020598362724945805}}
set_robot_commands_max0.0023959850778385084
set_robot_commands_mean0.0022839274618208813
set_robot_commands_median0.0022809810741066775
set_robot_commands_min0.002177762621231662
sim_compute_performance-ego0_max0.0022006359230093407
sim_compute_performance-ego0_mean0.0020503242142024354
sim_compute_performance-ego0_median0.0020209262055028545
sim_compute_performance-ego0_min0.0019588085227946897
sim_compute_sim_state_max0.009872619468386813
sim_compute_sim_state_mean0.008858568055623168
sim_compute_sim_state_median0.009620876059562126
sim_compute_sim_state_min0.006319900634981611
sim_render-ego0_max0.004183579464347995
sim_render-ego0_mean0.003920746865758741
sim_render-ego0_median0.0038639375592446545
sim_render-ego0_min0.003771532880197657
simulation-passed1
step_physics_max0.14110507283891952
step_physics_mean0.114430618965487
step_physics_median0.11117499490665617
step_physics_min0.09426741320971616
survival_time_max59.99999999999873
survival_time_mean29.149999999999658
survival_time_min7.299999999999982
No reset possible
5437611560Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:38:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.4540036958145
survival_time_median59.99999999999873
deviation-center-line_median3.787393040390469
in-drivable-lane_median6.074999999999765


other stats
agent_compute-ego0_max0.012417950499166957
agent_compute-ego0_mean0.012158793743207264
agent_compute-ego0_median0.01217778769500383
agent_compute-ego0_min0.01186164908365445
complete-iteration_max0.2463485503772415
complete-iteration_mean0.21262724704358155
complete-iteration_median0.2129349190045128
complete-iteration_min0.1782905997880591
deviation-center-line_max4.13585014843881
deviation-center-line_mean3.56840721003942
deviation-center-line_min2.5629926109379335
deviation-heading_max19.61169560075959
deviation-heading_mean17.407508087774975
deviation-heading_median17.99100814588045
deviation-heading_min14.036320458579414
driven_any_max11.946717839451614
driven_any_mean11.50890848374462
driven_any_median11.566497880470852
driven_any_min10.955920334585151
driven_lanedir_consec_max10.934222476114703
driven_lanedir_consec_mean8.543772404454144
driven_lanedir_consec_min6.33285975007287
driven_lanedir_max10.934222476114703
driven_lanedir_mean9.210975986985568
driven_lanedir_median9.78017950478906
driven_lanedir_min6.349322462249449
get_duckie_state_max1.3524398995887122e-06
get_duckie_state_mean1.2733222364687526e-06
get_duckie_state_median1.2697129325009107e-06
get_duckie_state_min1.2014231812844764e-06
get_robot_state_max0.0038105034207617104
get_robot_state_mean0.003748431446818147
get_robot_state_median0.00376946294833778
get_robot_state_min0.0036442964698353178
get_state_dump_max0.004859374412261168
get_state_dump_mean0.004688026502832606
get_state_dump_median0.004649266215982683
get_state_dump_min0.004594199167103891
get_ui_image_max0.03568118855319948
get_ui_image_mean0.03066707847193202
get_ui_image_median0.030430995726632708
get_ui_image_min0.026125133881263193
in-drivable-lane_max23.44999999999868
in-drivable-lane_mean9.024999999999554
in-drivable-lane_min0.5000000000000071
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.955920334585151, "get_ui_image": 0.02913989388172012, "step_physics": 0.13955767428945037, "survival_time": 59.49999999999876, "driven_lanedir": 6.349322462249449, "get_state_dump": 0.004859374412261168, "get_robot_state": 0.0038105034207617104, "sim_render-ego0": 0.003901380936104545, "get_duckie_state": 1.3524398995887122e-06, "in-drivable-lane": 23.44999999999868, "deviation-heading": 14.036320458579414, "agent_compute-ego0": 0.012333522415481428, "complete-iteration": 0.2074646857683645, "set_robot_commands": 0.002311477132648305, "deviation-center-line": 2.5629926109379335, "driven_lanedir_consec": 6.33285975007287, "sim_compute_sim_state": 0.009388133520643617, "sim_compute_performance-ego0": 0.002075474168151292}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.946717839451614, "get_ui_image": 0.03568118855319948, "step_physics": 0.16758962237368416, "survival_time": 59.99999999999873, "driven_lanedir": 10.934222476114703, "get_state_dump": 0.004639985658644041, "get_robot_state": 0.0037747958022093, "sim_render-ego0": 0.0038755241778371335, "get_duckie_state": 1.2943290056138114e-06, "in-drivable-lane": 0.5000000000000071, "deviation-heading": 19.61169560075959, "agent_compute-ego0": 0.012417950499166957, "complete-iteration": 0.2463485503772415, "set_robot_commands": 0.0023034113233631397, "deviation-center-line": 4.13585014843881, "driven_lanedir_consec": 10.934222476114703, "sim_compute_sim_state": 0.013892208308998889, "sim_compute_performance-ego0": 0.002086236018324573}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.703579858292503, "get_ui_image": 0.031722097571545295, "step_physics": 0.1453220943527158, "survival_time": 59.99999999999873, "driven_lanedir": 9.962196586065398, "get_state_dump": 0.004658546773321325, "get_robot_state": 0.003764130094466261, "sim_render-ego0": 0.003824356295087753, "get_duckie_state": 1.2014231812844764e-06, "in-drivable-lane": 5.699999999999896, "deviation-heading": 19.59498521602345, "agent_compute-ego0": 0.012022052974526232, "complete-iteration": 0.2184051522406611, "set_robot_commands": 0.0022552305216793217, "deviation-center-line": 4.026517634370256, "driven_lanedir_consec": 9.962196586065398, "sim_compute_sim_state": 0.01271764364568121, "sim_compute_performance-ego0": 0.0020345236042159285}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.4294159026492, "get_ui_image": 0.026125133881263193, "step_physics": 0.11802689796879726, "survival_time": 59.99999999999873, "driven_lanedir": 9.598162423512726, "get_state_dump": 0.004594199167103891, "get_robot_state": 0.0036442964698353178, "sim_render-ego0": 0.0037595977592627073, "get_duckie_state": 1.24509685938801e-06, "in-drivable-lane": 6.449999999999633, "deviation-heading": 16.387031075737454, "agent_compute-ego0": 0.01186164908365445, "complete-iteration": 0.1782905997880591, "set_robot_commands": 0.002199880884251527, "deviation-center-line": 3.5482684464106815, "driven_lanedir_consec": 6.945810805563603, "sim_compute_sim_state": 0.006036233147614008, "sim_compute_performance-ego0": 0.0019606577168892665}}
set_robot_commands_max0.002311477132648305
set_robot_commands_mean0.0022674999654855735
set_robot_commands_median0.002279320922521231
set_robot_commands_min0.002199880884251527
sim_compute_performance-ego0_max0.002086236018324573
sim_compute_performance-ego0_mean0.002039222876895265
sim_compute_performance-ego0_median0.00205499888618361
sim_compute_performance-ego0_min0.0019606577168892665
sim_compute_sim_state_max0.013892208308998889
sim_compute_sim_state_mean0.010508554655734432
sim_compute_sim_state_median0.011052888583162414
sim_compute_sim_state_min0.006036233147614008
sim_render-ego0_max0.003901380936104545
sim_render-ego0_mean0.0038402147920730345
sim_render-ego0_median0.003849940236462443
sim_render-ego0_min0.0037595977592627073
simulation-passed1
step_physics_max0.16758962237368416
step_physics_mean0.1426240722461619
step_physics_median0.1424398843210831
step_physics_min0.11802689796879726
survival_time_max59.99999999999873
survival_time_mean59.874999999998735
survival_time_min59.49999999999876
No reset possible
5434611571Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simfailedyesnogpu-prod-060:04:36
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 308 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5431211585Yishu Malhotra 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:06:52
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5429411591Yishu Malhotra 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:06:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4082843531515713
survival_time_median6.374999999999986
deviation-center-line_median0.1581610107276048
in-drivable-lane_median4.399999999999986


other stats
agent_compute-ego0_max0.013251248826371862
agent_compute-ego0_mean0.012569403294376887
agent_compute-ego0_median0.012371212247650162
agent_compute-ego0_min0.012283939855835369
complete-iteration_max0.2205948771500006
complete-iteration_mean0.20928709778855617
complete-iteration_median0.2143994688079182
complete-iteration_min0.1877545763883876
deviation-center-line_max0.30274892612520354
deviation-center-line_mean0.17343529587935816
deviation-center-line_min0.07467023593701948
deviation-heading_max2.392147425735463
deviation-heading_mean1.2623488709114072
deviation-heading_median1.1274537380844656
deviation-heading_min0.4023405817412355
driven_any_max2.838252599229776
driven_any_mean1.6110285955621193
driven_any_median1.2414299076552469
driven_any_min1.123001967708208
driven_lanedir_consec_max0.5655628247989355
driven_lanedir_consec_mean0.4001895063232127
driven_lanedir_consec_min0.2186264941907727
driven_lanedir_max0.5655628247989355
driven_lanedir_mean0.4001895063232127
driven_lanedir_median0.4082843531515713
driven_lanedir_min0.2186264941907727
get_duckie_state_max1.466020624688331e-06
get_duckie_state_mean1.331839274789002e-06
get_duckie_state_median1.3150501150760724e-06
get_duckie_state_min1.2312362443155318e-06
get_robot_state_max0.003832699848567925
get_robot_state_mean0.003740815225948419
get_robot_state_median0.00373951563118904
get_robot_state_min0.003651529792847672
get_state_dump_max0.005034284388765376
get_state_dump_mean0.004787013676919629
get_state_dump_median0.004771621756781346
get_state_dump_min0.004570526805350451
get_ui_image_max0.03504283641412006
get_ui_image_mean0.031143357680037027
get_ui_image_median0.03141209868333877
get_ui_image_min0.02670639693935052
in-drivable-lane_max11.150000000000084
in-drivable-lane_mean5.675000000000011
in-drivable-lane_min2.7499999999999902
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 1.1954385978347135, "get_ui_image": 0.02917139209918122, "step_physics": 0.1481706473364759, "survival_time": 6.649999999999984, "driven_lanedir": 0.2186264941907727, "get_state_dump": 0.004774047367608369, "get_robot_state": 0.003704297008799083, "sim_render-ego0": 0.003777519980473305, "get_duckie_state": 1.2312362443155318e-06, "in-drivable-lane": 4.599999999999987, "deviation-heading": 1.605355647217083, "agent_compute-ego0": 0.012308364483847548, "complete-iteration": 0.2143679003217327, "set_robot_commands": 0.0021711338811845923, "deviation-center-line": 0.21247740857406025, "driven_lanedir_consec": 0.2186264941907727, "sim_compute_sim_state": 0.008211762157838736, "sim_compute_performance-ego0": 0.001990519352813265}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.2874212174757802, "get_ui_image": 0.03504283641412006, "step_physics": 0.1468732201956152, "survival_time": 6.099999999999986, "driven_lanedir": 0.3430341022313601, "get_state_dump": 0.004570526805350451, "get_robot_state": 0.003651529792847672, "sim_render-ego0": 0.0036849374693583665, "get_duckie_state": 1.2851342922303736e-06, "in-drivable-lane": 4.199999999999985, "deviation-heading": 0.6495518289518485, "agent_compute-ego0": 0.012434060011452776, "complete-iteration": 0.2205948771500006, "set_robot_commands": 0.0022480429672613374, "deviation-center-line": 0.07467023593701948, "driven_lanedir_consec": 0.3430341022313601, "sim_compute_sim_state": 0.009998660746628677, "sim_compute_performance-ego0": 0.002001892260419644}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.123001967708208, "get_ui_image": 0.03365280526749631, "step_physics": 0.14176103916574032, "survival_time": 4.6499999999999915, "driven_lanedir": 0.4735346040717825, "get_state_dump": 0.005034284388765376, "get_robot_state": 0.0037747342535789977, "sim_render-ego0": 0.003983862856601147, "get_duckie_state": 1.466020624688331e-06, "in-drivable-lane": 2.7499999999999902, "deviation-heading": 0.4023405817412355, "agent_compute-ego0": 0.013251248826371862, "complete-iteration": 0.21443103729410373, "set_robot_commands": 0.0023701317766879467, "deviation-center-line": 0.10384461288114936, "driven_lanedir_consec": 0.4735346040717825, "sim_compute_sim_state": 0.008349791486212548, "sim_compute_performance-ego0": 0.002152714323490224}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.838252599229776, "get_ui_image": 0.02670639693935052, "step_physics": 0.12641479485850793, "survival_time": 15.000000000000078, "driven_lanedir": 0.5655628247989355, "get_state_dump": 0.004769196145954322, "get_robot_state": 0.003832699848567925, "sim_render-ego0": 0.0037903088667860063, "get_duckie_state": 1.3449659379217711e-06, "in-drivable-lane": 11.150000000000084, "deviation-heading": 2.392147425735463, "agent_compute-ego0": 0.012283939855835369, "complete-iteration": 0.1877545763883876, "set_robot_commands": 0.0021663892308738937, "deviation-center-line": 0.30274892612520354, "driven_lanedir_consec": 0.5655628247989355, "sim_compute_sim_state": 0.005739315799700461, "sim_compute_performance-ego0": 0.0019615734138362035}}
set_robot_commands_max0.0023701317766879467
set_robot_commands_mean0.0022389244640019425
set_robot_commands_median0.002209588424222965
set_robot_commands_min0.0021663892308738937
sim_compute_performance-ego0_max0.002152714323490224
sim_compute_performance-ego0_mean0.0020266748376398342
sim_compute_performance-ego0_median0.001996205806616454
sim_compute_performance-ego0_min0.0019615734138362035
sim_compute_sim_state_max0.009998660746628677
sim_compute_sim_state_mean0.008074882547595105
sim_compute_sim_state_median0.008280776822025642
sim_compute_sim_state_min0.005739315799700461
sim_render-ego0_max0.003983862856601147
sim_render-ego0_mean0.0038091572933047066
sim_render-ego0_median0.003783914423629656
sim_render-ego0_min0.0036849374693583665
simulation-passed1
step_physics_max0.1481706473364759
step_physics_mean0.14080492538908485
step_physics_median0.14431712968067775
step_physics_min0.12641479485850793
survival_time_max15.000000000000078
survival_time_mean8.10000000000001
survival_time_min4.6499999999999915
No reset possible
5426911606Jean-Sébastien Grondin 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:09:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.2116466260577274
survival_time_median13.175000000000052
deviation-center-line_median0.39977734756272887
in-drivable-lane_median7.050000000000046


other stats
agent_compute-ego0_max0.013365530263522506
agent_compute-ego0_mean0.012844721679657011
agent_compute-ego0_median0.012743852425584196
agent_compute-ego0_min0.012525651603937147
complete-iteration_max0.262661277799678
complete-iteration_mean0.23333610397972343
complete-iteration_median0.23262615111275312
complete-iteration_min0.20543083589370936
deviation-center-line_max0.6712528457284261
deviation-center-line_mean0.3845553184135045
deviation-center-line_min0.06741373280013406
deviation-heading_max5.406003192887891
deviation-heading_mean3.0025990149473714
deviation-heading_median2.9815386225629235
deviation-heading_min0.6413156217757467
driven_any_max6.326359751953097
driven_any_mean4.182403797939591
driven_any_median4.230422362309948
driven_any_min1.942410715185367
driven_lanedir_consec_max2.2558119561917422
driven_lanedir_consec_mean1.2724974610513513
driven_lanedir_consec_min0.4108846358982081
driven_lanedir_max2.2558119561917422
driven_lanedir_mean1.3193360765351936
driven_lanedir_median1.305323857025412
driven_lanedir_min0.4108846358982081
get_duckie_state_max1.3717233318172089e-06
get_duckie_state_mean1.3406247884895243e-06
get_duckie_state_median1.3401849737650231e-06
get_duckie_state_min1.3104058746108434e-06
get_robot_state_max0.0038915445029987063
get_robot_state_mean0.003806421896567888
get_robot_state_median0.003798506906121211
get_robot_state_min0.003737129271030426
get_state_dump_max0.004768766768991131
get_state_dump_mean0.0047274371195229845
get_state_dump_median0.004726687434447718
get_state_dump_min0.004687606840205372
get_ui_image_max0.03643213895926798
get_ui_image_mean0.031298895240885985
get_ui_image_median0.030849400888106743
get_ui_image_min0.02706464022806246
in-drivable-lane_max11.950000000000124
in-drivable-lane_mean7.75000000000005
in-drivable-lane_min4.949999999999983
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 6.326359751953097, "get_ui_image": 0.028685947880148888, "step_physics": 0.14885664048294223, "survival_time": 19.150000000000137, "driven_lanedir": 1.58928685556112, "get_state_dump": 0.004695565129319827, "get_robot_state": 0.003737129271030426, "sim_render-ego0": 0.003834715733925502, "get_duckie_state": 1.3373792171478271e-06, "in-drivable-lane": 11.950000000000124, "deviation-heading": 5.406003192887891, "agent_compute-ego0": 0.012525651603937147, "complete-iteration": 0.2160274895528952, "set_robot_commands": 0.0022302549332380295, "deviation-center-line": 0.6712528457284261, "driven_lanedir_consec": 1.4019323936257513, "sim_compute_sim_state": 0.00933188945055008, "sim_compute_performance-ego0": 0.0020433993389209113}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.942410715185367, "get_ui_image": 0.03643213895926798, "step_physics": 0.18632054687442637, "survival_time": 6.5999999999999845, "driven_lanedir": 0.4108846358982081, "get_state_dump": 0.004687606840205372, "get_robot_state": 0.003834588187081473, "sim_render-ego0": 0.003842332309350035, "get_duckie_state": 1.3104058746108434e-06, "in-drivable-lane": 4.949999999999983, "deviation-heading": 0.6413156217757467, "agent_compute-ego0": 0.012558196720324063, "complete-iteration": 0.262661277799678, "set_robot_commands": 0.0022443273013695738, "deviation-center-line": 0.06741373280013406, "driven_lanedir_consec": 0.4108846358982081, "sim_compute_sim_state": 0.010621787910174606, "sim_compute_performance-ego0": 0.002036266757133312}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.741127081175603, "get_ui_image": 0.0330128538960646, "step_physics": 0.1746287537023488, "survival_time": 11.800000000000033, "driven_lanedir": 1.0213608584897038, "get_state_dump": 0.004757809739575607, "get_robot_state": 0.0038915445029987063, "sim_render-ego0": 0.004001622461568454, "get_duckie_state": 1.342990730382219e-06, "in-drivable-lane": 8.150000000000038, "deviation-heading": 1.0509583664409392, "agent_compute-ego0": 0.013365530263522506, "complete-iteration": 0.24922481267261104, "set_robot_commands": 0.002398544222996708, "deviation-center-line": 0.18183737056349275, "driven_lanedir_consec": 1.0213608584897038, "sim_compute_sim_state": 0.01091070014213208, "sim_compute_performance-ego0": 0.0021686382937531933}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.7197176434442945, "get_ui_image": 0.02706464022806246, "step_physics": 0.14268979715974364, "survival_time": 14.550000000000072, "driven_lanedir": 2.2558119561917422, "get_state_dump": 0.004768766768991131, "get_robot_state": 0.003762425625160949, "sim_render-ego0": 0.003860510375401745, "get_duckie_state": 1.3717233318172089e-06, "in-drivable-lane": 5.950000000000054, "deviation-heading": 4.9121188786849075, "agent_compute-ego0": 0.012929508130844326, "complete-iteration": 0.20543083589370936, "set_robot_commands": 0.002287356820825028, "deviation-center-line": 0.6177173245619649, "driven_lanedir_consec": 2.2558119561917422, "sim_compute_sim_state": 0.005957186222076416, "sim_compute_performance-ego0": 0.0020241484250107856}}
set_robot_commands_max0.002398544222996708
set_robot_commands_mean0.0022901208196073348
set_robot_commands_median0.002265842061097301
set_robot_commands_min0.0022302549332380295
sim_compute_performance-ego0_max0.0021686382937531933
sim_compute_performance-ego0_mean0.0020681132037045505
sim_compute_performance-ego0_median0.0020398330480271114
sim_compute_performance-ego0_min0.0020241484250107856
sim_compute_sim_state_max0.01091070014213208
sim_compute_sim_state_mean0.009205390931233296
sim_compute_sim_state_median0.009976838680362345
sim_compute_sim_state_min0.005957186222076416
sim_render-ego0_max0.004001622461568454
sim_render-ego0_mean0.0038847952200614343
sim_render-ego0_median0.00385142134237589
sim_render-ego0_min0.003834715733925502
simulation-passed1
step_physics_max0.18632054687442637
step_physics_mean0.16312393455486526
step_physics_median0.16174269709264552
step_physics_min0.14268979715974364
survival_time_max19.150000000000137
survival_time_mean13.025000000000055
survival_time_min6.5999999999999845
No reset possible
5426111624Anthony Courchesne 🇨🇦TNetR1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:02:04
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5420211633Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:17:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.2689365585346293
survival_time_median23.6500000000002
deviation-center-line_median0.5778801482485184
in-drivable-lane_median14.125000000000163


other stats
agent_compute-ego0_max0.012859702649699078
agent_compute-ego0_mean0.012767348460280945
agent_compute-ego0_median0.012767483515488875
agent_compute-ego0_min0.012674724160446953
complete-iteration_max0.22905663268206872
complete-iteration_mean0.19784357173068529
complete-iteration_median0.19550989134623795
complete-iteration_min0.1712978715481965
deviation-center-line_max2.2690040630833437
deviation-center-line_mean0.9172301153629532
deviation-center-line_min0.2441561018714324
deviation-heading_max8.858632537386994
deviation-heading_mean3.662910371883634
deviation-heading_median2.271655435777902
deviation-heading_min1.2496980785917384
driven_any_max13.749295558050688
driven_any_mean6.382882193242287
driven_any_median5.207001048637358
driven_any_min1.3682311176437425
driven_lanedir_consec_max8.241948200271883
driven_lanedir_consec_mean3.3452720913046385
driven_lanedir_consec_min0.6012670478774123
driven_lanedir_max8.768538837678156
driven_lanedir_mean3.4769197506562066
driven_lanedir_median2.2689365585346293
driven_lanedir_min0.6012670478774123
get_duckie_state_max2.3139666204583154e-06
get_duckie_state_mean2.1665887316605947e-06
get_duckie_state_median2.1688210642903007e-06
get_duckie_state_min2.014746177603462e-06
get_robot_state_max0.003846559000451995
get_robot_state_mean0.003782182778220268
get_robot_state_median0.003765219277460104
get_robot_state_min0.003751733557508868
get_state_dump_max0.004955340738165869
get_state_dump_mean0.00484838016863724
get_state_dump_median0.004835477135658505
get_state_dump_min0.004767225665066082
get_ui_image_max0.03902531813268792
get_ui_image_mean0.03232872553739271
get_ui_image_median0.03169776716735903
get_ui_image_min0.026894049682164852
in-drivable-lane_max24.250000000000057
in-drivable-lane_mean14.000000000000092
in-drivable-lane_min3.4999999999999876
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.985589941832444, "get_ui_image": 0.02977475159847898, "step_physics": 0.12051702085123882, "survival_time": 22.05000000000018, "driven_lanedir": 3.798722213528078, "get_state_dump": 0.004863680757548475, "get_robot_state": 0.0037639459333808174, "sim_render-ego0": 0.0038753186955171473, "get_duckie_state": 2.145227803364059e-06, "in-drivable-lane": 6.850000000000097, "deviation-heading": 2.9430010176211203, "agent_compute-ego0": 0.012859702649699078, "complete-iteration": 0.19113750802985144, "set_robot_commands": 0.0023308199455295752, "deviation-center-line": 0.8484220271501554, "driven_lanedir_consec": 3.798722213528078, "sim_compute_sim_state": 0.010962813688079696, "sim_compute_performance-ego0": 0.002104792659638694}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3682311176437425, "get_ui_image": 0.03902531813268792, "step_physics": 0.14953734123543516, "survival_time": 7.249999999999982, "driven_lanedir": 0.6012670478774123, "get_state_dump": 0.004955340738165869, "get_robot_state": 0.0037664926215393905, "sim_render-ego0": 0.003961164657383749, "get_duckie_state": 2.3139666204583154e-06, "in-drivable-lane": 3.4999999999999876, "deviation-heading": 1.6003098539346854, "agent_compute-ego0": 0.0128326318035387, "complete-iteration": 0.22905663268206872, "set_robot_commands": 0.00230124878556761, "deviation-center-line": 0.3073382693468815, "driven_lanedir_consec": 0.6012670478774123, "sim_compute_sim_state": 0.01050248864578874, "sim_compute_performance-ego0": 0.002085914350535772}, "LF-norm-techtrack-000-ego0": {"driven_any": 13.749295558050688, "get_ui_image": 0.033620782736239085, "step_physics": 0.12406170199455369, "survival_time": 59.99999999999873, "driven_lanedir": 8.768538837678156, "get_state_dump": 0.0048072735137685355, "get_robot_state": 0.003846559000451995, "sim_render-ego0": 0.003950469996113265, "get_duckie_state": 2.014746177603462e-06, "in-drivable-lane": 24.250000000000057, "deviation-heading": 8.858632537386994, "agent_compute-ego0": 0.012702335227439049, "complete-iteration": 0.19988227466262448, "set_robot_commands": 0.0022833524000436243, "deviation-center-line": 2.2690040630833437, "driven_lanedir_consec": 8.241948200271883, "sim_compute_sim_state": 0.012444410594079415, "sim_compute_performance-ego0": 0.0020801564438158427}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.428412155442272, "get_ui_image": 0.026894049682164852, "step_physics": 0.1084339232312832, "survival_time": 25.250000000000224, "driven_lanedir": 0.7391509035411803, "get_state_dump": 0.004767225665066082, "get_robot_state": 0.003751733557508868, "sim_render-ego0": 0.003846822048835604, "get_duckie_state": 2.192414325216542e-06, "in-drivable-lane": 21.40000000000023, "deviation-heading": 1.2496980785917384, "agent_compute-ego0": 0.012674724160446953, "complete-iteration": 0.1712978715481965, "set_robot_commands": 0.0022665050189956846, "deviation-center-line": 0.2441561018714324, "driven_lanedir_consec": 0.7391509035411803, "sim_compute_sim_state": 0.006604207834236236, "sim_compute_performance-ego0": 0.001974487021977722}}
set_robot_commands_max0.0023308199455295752
set_robot_commands_mean0.0022954815375341234
set_robot_commands_median0.0022923005928056172
set_robot_commands_min0.0022665050189956846
sim_compute_performance-ego0_max0.002104792659638694
sim_compute_performance-ego0_mean0.0020613376189920077
sim_compute_performance-ego0_median0.002083035397175807
sim_compute_performance-ego0_min0.001974487021977722
sim_compute_sim_state_max0.012444410594079415
sim_compute_sim_state_mean0.010128480190546022
sim_compute_sim_state_median0.01073265116693422
sim_compute_sim_state_min0.006604207834236236
sim_render-ego0_max0.003961164657383749
sim_render-ego0_mean0.003908443849462441
sim_render-ego0_median0.003912894345815206
sim_render-ego0_min0.003846822048835604
simulation-passed1
step_physics_max0.14953734123543516
step_physics_mean0.12563749682812772
step_physics_median0.12228936142289626
step_physics_min0.1084339232312832
survival_time_max59.99999999999873
survival_time_mean28.63749999999978
survival_time_min7.249999999999982
No reset possible
5411711661Dishank Bansal 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:36:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median9.897040621882477
survival_time_median59.99999999999873
deviation-center-line_median2.7877897814050687
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.01284730980338701
agent_compute-ego0_mean0.01270676041324371
agent_compute-ego0_median0.012699470035638738
agent_compute-ego0_min0.012580791778310353
complete-iteration_max0.2179109825877524
complete-iteration_mean0.19397910791670256
complete-iteration_median0.19142750627690808
complete-iteration_min0.17515043652524162
deviation-center-line_max3.598819126901866
deviation-center-line_mean2.878822618386825
deviation-center-line_min2.3408917838352954
deviation-heading_max12.201796542924674
deviation-heading_mean11.23380003679772
deviation-heading_median11.130352412249303
deviation-heading_min10.472698779767594
driven_any_max11.925482318288406
driven_any_mean10.45522826583226
driven_any_median10.174212380066356
driven_any_min9.547005984907928
driven_lanedir_consec_max11.637915072116355
driven_lanedir_consec_mean10.122421145028028
driven_lanedir_consec_min9.0576882642308
driven_lanedir_max11.637915072116355
driven_lanedir_mean10.122421145028028
driven_lanedir_median9.897040621882477
driven_lanedir_min9.0576882642308
get_duckie_state_max1.4386506600741243e-06
get_duckie_state_mean1.3713534924509525e-06
get_duckie_state_median1.3711549757322995e-06
get_duckie_state_min1.304453358265085e-06
get_robot_state_max0.0038926307605168502
get_robot_state_mean0.0038540358845141408
get_robot_state_median0.003872272275468889
get_robot_state_min0.0037789682266019366
get_state_dump_max0.004880655218818404
get_state_dump_mean0.004793205279096973
get_state_dump_median0.00478043514524868
get_state_dump_min0.0047312956070721295
get_ui_image_max0.036545187110805594
get_ui_image_mean0.031845617800529157
get_ui_image_median0.0315875228894541
get_ui_image_min0.02766223831240283
in-drivable-lane_max2.500000000000025
in-drivable-lane_mean0.6250000000000062
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.925482318288406, "get_ui_image": 0.029942925426981827, "step_physics": 0.11274089582953029, "survival_time": 59.99999999999873, "driven_lanedir": 11.637915072116355, "get_state_dump": 0.004880655218818404, "get_robot_state": 0.003881116592318291, "sim_render-ego0": 0.003942355625238347, "get_duckie_state": 1.4386506600741243e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.78584439201945, "agent_compute-ego0": 0.012814931925091515, "complete-iteration": 0.1825843242483274, "set_robot_commands": 0.0023751985421287926, "deviation-center-line": 2.81756854210401, "driven_lanedir_consec": 11.637915072116355, "sim_compute_sim_state": 0.00977828381559831, "sim_compute_performance-ego0": 0.0021374485673356512}, "LF-norm-zigzag-000-ego0": {"driven_any": 10.068325386847564, "get_ui_image": 0.036545187110805594, "step_physics": 0.13852592967729782, "survival_time": 59.99999999999873, "driven_lanedir": 9.764384975698997, "get_state_dump": 0.0047312956070721295, "get_robot_state": 0.0037789682266019366, "sim_render-ego0": 0.0038884017588593977, "get_duckie_state": 1.3308560818458575e-06, "in-drivable-lane": 0.0, "deviation-heading": 12.201796542924674, "agent_compute-ego0": 0.012584008146185958, "complete-iteration": 0.2179109825877524, "set_robot_commands": 0.002296636940339126, "deviation-center-line": 2.7580110207061272, "driven_lanedir_consec": 9.764384975698997, "sim_compute_sim_state": 0.013381855573185676, "sim_compute_performance-ego0": 0.002091666244646592}, "LF-norm-techtrack-000-ego0": {"driven_any": 10.280099373285148, "get_ui_image": 0.033232120351926374, "step_physics": 0.12386132934309858, "survival_time": 59.99999999999873, "driven_lanedir": 10.029696268065951, "get_state_dump": 0.004764979923893073, "get_robot_state": 0.0038634279586194854, "sim_render-ego0": 0.003984768325145001, "get_duckie_state": 1.304453358265085e-06, "in-drivable-lane": 0.0, "deviation-heading": 11.47486043247916, "agent_compute-ego0": 0.012580791778310353, "complete-iteration": 0.20027068830548872, "set_robot_commands": 0.0023911626610926646, "deviation-center-line": 3.598819126901866, "driven_lanedir_consec": 10.029696268065951, "sim_compute_sim_state": 0.013359416632926236, "sim_compute_performance-ego0": 0.0021444393335829964}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.547005984907928, "get_ui_image": 0.02766223831240283, "step_physics": 0.11101134333582742, "survival_time": 59.99999999999873, "driven_lanedir": 9.0576882642308, "get_state_dump": 0.0047958903666042866, "get_robot_state": 0.0038926307605168502, "sim_render-ego0": 0.003969178410195788, "get_duckie_state": 1.411453869618742e-06, "in-drivable-lane": 2.500000000000025, "deviation-heading": 10.472698779767594, "agent_compute-ego0": 0.01284730980338701, "complete-iteration": 0.17515043652524162, "set_robot_commands": 0.002406391474130648, "deviation-center-line": 2.3408917838352954, "driven_lanedir_consec": 9.0576882642308, "sim_compute_sim_state": 0.0063713259939150845, "sim_compute_performance-ego0": 0.002105133221806535}}
set_robot_commands_max0.002406391474130648
set_robot_commands_mean0.002367347404422807
set_robot_commands_median0.0023831806016107286
set_robot_commands_min0.002296636940339126
sim_compute_performance-ego0_max0.0021444393335829964
sim_compute_performance-ego0_mean0.0021196718418429435
sim_compute_performance-ego0_median0.002121290894571093
sim_compute_performance-ego0_min0.002091666244646592
sim_compute_sim_state_max0.013381855573185676
sim_compute_sim_state_mean0.010722720503906328
sim_compute_sim_state_median0.011568850224262274
sim_compute_sim_state_min0.0063713259939150845
sim_render-ego0_max0.003984768325145001
sim_render-ego0_mean0.003946176029859633
sim_render-ego0_median0.003955767017717068
sim_render-ego0_min0.0038884017588593977
simulation-passed1
step_physics_max0.13852592967729782
step_physics_mean0.12153487454643852
step_physics_median0.11830111258631444
step_physics_min0.11101134333582742
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5408111674Dishank Bansal 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:10:46
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5406811705brian wongbaseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:02:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5402611686Philippe Reddy 🇨🇦real-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:13:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9089232161277172
survival_time_median5.724999999999987
deviation-center-line_median0.24356764963180777
in-drivable-lane_median2.1499999999999924


other stats
agent_compute-ego0_max0.013999797964608795
agent_compute-ego0_mean0.013039770715706889
agent_compute-ego0_median0.013015421688463886
agent_compute-ego0_min0.012128441521290985
complete-iteration_max0.29172456666324914
complete-iteration_mean0.25054994141925274
complete-iteration_median0.2532698501185594
complete-iteration_min0.20393549877664316
deviation-center-line_max3.669479998459467
deviation-center-line_mean1.077970229744082
deviation-center-line_min0.15526562125324556
deviation-heading_max16.86648596615549
deviation-heading_mean5.259234903652473
deviation-heading_median1.5756590132811228
deviation-heading_min1.019135621892154
driven_any_max19.80060868912293
driven_any_mean6.002208679042204
driven_any_median1.5465889320750064
driven_any_min1.1150481628958702
driven_lanedir_consec_max14.98191219920051
driven_lanedir_consec_mean4.373231324876671
driven_lanedir_consec_min0.6931666680507382
driven_lanedir_max14.98191219920051
driven_lanedir_mean4.387842044177235
driven_lanedir_median0.9344898706957768
driven_lanedir_min0.7004762361168748
get_duckie_state_max2.3692970371166933e-06
get_duckie_state_mean2.204577754099332e-06
get_duckie_state_median2.2279709062045735e-06
get_duckie_state_min1.993072166871489e-06
get_robot_state_max0.004047609387984582
get_robot_state_mean0.0038293556534643902
get_robot_state_median0.0038494876524509257
get_robot_state_min0.0035708379209711313
get_state_dump_max0.005107051582746608
get_state_dump_mean0.004885149657608327
get_state_dump_median0.004911944955052615
get_state_dump_min0.004609657137581472
get_ui_image_max0.035846029774526535
get_ui_image_mean0.031916918010980344
get_ui_image_median0.032567004052693145
get_ui_image_min0.02668763416400854
in-drivable-lane_max10.549999999999851
in-drivable-lane_mean4.049999999999958
in-drivable-lane_min1.3499999999999952
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 19.80060868912293, "get_ui_image": 0.03060666786244668, "step_physics": 0.17153274467048996, "survival_time": 59.99999999999873, "driven_lanedir": 14.98191219920051, "get_state_dump": 0.005072162709168649, "get_robot_state": 0.004047609387984582, "sim_render-ego0": 0.004085895917894045, "get_duckie_state": 2.3692970371166933e-06, "in-drivable-lane": 10.549999999999851, "deviation-heading": 16.86648596615549, "agent_compute-ego0": 0.013511919161362216, "complete-iteration": 0.2436437444027814, "set_robot_commands": 0.0024418805064408606, "deviation-center-line": 3.669479998459467, "driven_lanedir_consec": 14.98191219920051, "sim_compute_sim_state": 0.010027472323720204, "sim_compute_performance-ego0": 0.0022151444377152747}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.1150481628958702, "get_ui_image": 0.035846029774526535, "step_physics": 0.218113502759612, "survival_time": 4.399999999999992, "driven_lanedir": 0.8331279945627079, "get_state_dump": 0.004609657137581472, "get_robot_state": 0.0035708379209711313, "sim_render-ego0": 0.003636194079109792, "get_duckie_state": 1.993072166871489e-06, "in-drivable-lane": 1.3499999999999952, "deviation-heading": 1.019135621892154, "agent_compute-ego0": 0.012128441521290985, "complete-iteration": 0.29172456666324914, "set_robot_commands": 0.0021268089165848293, "deviation-center-line": 0.15526562125324556, "driven_lanedir_consec": 0.8331279945627079, "sim_compute_sim_state": 0.00965249136592565, "sim_compute_performance-ego0": 0.0019481048155366703}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.1649946782777252, "get_ui_image": 0.03452734024293961, "step_physics": 0.18798811717699931, "survival_time": 4.599999999999992, "driven_lanedir": 0.7004762361168748, "get_state_dump": 0.005107051582746608, "get_robot_state": 0.003927417980727329, "sim_render-ego0": 0.004027166674214024, "get_duckie_state": 2.3585493846606183e-06, "in-drivable-lane": 1.5999999999999943, "deviation-heading": 1.2387025195193433, "agent_compute-ego0": 0.013999797964608795, "complete-iteration": 0.26289595583433745, "set_robot_commands": 0.002397114230740455, "deviation-center-line": 0.1945806557492016, "driven_lanedir_consec": 0.6931666680507382, "sim_compute_sim_state": 0.008668240680489488, "sim_compute_performance-ego0": 0.0021468695773873277}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.928183185872287, "get_ui_image": 0.02668763416400854, "step_physics": 0.1422638772190481, "survival_time": 6.849999999999984, "driven_lanedir": 1.0358517468288455, "get_state_dump": 0.00475172720093658, "get_robot_state": 0.0037715573241745215, "sim_render-ego0": 0.003978865734045056, "get_duckie_state": 2.0973924277485283e-06, "in-drivable-lane": 2.6999999999999904, "deviation-heading": 1.9126155070429025, "agent_compute-ego0": 0.012518924215565556, "complete-iteration": 0.20393549877664316, "set_robot_commands": 0.0022779582203298373, "deviation-center-line": 0.29255464351441396, "driven_lanedir_consec": 0.9847184376927268, "sim_compute_sim_state": 0.005562060121176899, "sim_compute_performance-ego0": 0.0020307008770928865}}
set_robot_commands_max0.0024418805064408606
set_robot_commands_mean0.0023109404685239957
set_robot_commands_median0.0023375362255351464
set_robot_commands_min0.0021268089165848293
sim_compute_performance-ego0_max0.0022151444377152747
sim_compute_performance-ego0_mean0.0020852049269330397
sim_compute_performance-ego0_median0.002088785227240107
sim_compute_performance-ego0_min0.0019481048155366703
sim_compute_sim_state_max0.010027472323720204
sim_compute_sim_state_mean0.008477566122828059
sim_compute_sim_state_median0.00916036602320757
sim_compute_sim_state_min0.005562060121176899
sim_render-ego0_max0.004085895917894045
sim_render-ego0_mean0.003932030601315729
sim_render-ego0_median0.00400301620412954
sim_render-ego0_min0.003636194079109792
simulation-passed1
step_physics_max0.218113502759612
step_physics_mean0.17997456045653734
step_physics_median0.17976043092374464
step_physics_min0.1422638772190481
survival_time_max59.99999999999873
survival_time_mean18.96249999999967
survival_time_min4.399999999999992
No reset possible
5396211734Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:33:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.757245786970036
survival_time_median59.99999999999873
deviation-center-line_median1.931242637805171
in-drivable-lane_median32.82499999999906


other stats
agent_compute-ego0_max0.012997586122700864
agent_compute-ego0_mean0.012672204597368092
agent_compute-ego0_median0.01265224648157226
agent_compute-ego0_min0.012386739303626982
complete-iteration_max0.2275347544553694
complete-iteration_mean0.20477509269468985
complete-iteration_median0.20726394454803593
complete-iteration_min0.1770377272273182
deviation-center-line_max2.7238486397930455
deviation-center-line_mean1.7975320590201855
deviation-center-line_min0.6037943206773542
deviation-heading_max10.592616319606828
deviation-heading_mean7.505397801334739
deviation-heading_median7.740198639918717
deviation-heading_min3.948577605894699
driven_any_max11.911610443305657
driven_any_mean9.93916251235051
driven_any_median10.37676866198833
driven_any_min7.091502282119721
driven_lanedir_consec_max6.144051747897905
driven_lanedir_consec_mean4.3104583413066635
driven_lanedir_consec_min1.5832900433886772
driven_lanedir_max6.144051747897905
driven_lanedir_mean4.314252629539764
driven_lanedir_median4.757245786970036
driven_lanedir_min1.598467196321082
get_duckie_state_max2.100434101803202e-06
get_duckie_state_mean2.0675834293801436e-06
get_duckie_state_median2.077477460697628e-06
get_duckie_state_min2.0149446943221145e-06
get_robot_state_max0.004060180534219518
get_robot_state_mean0.003985299723528221
get_robot_state_median0.003987840272107787
get_robot_state_min0.003905337815677792
get_state_dump_max0.004961634952757976
get_state_dump_mean0.00488584828587263
get_state_dump_median0.004903002382259101
get_state_dump_min0.0047757534262143405
get_ui_image_max0.03773462504371815
get_ui_image_mean0.03326204963437668
get_ui_image_median0.03363126423770928
get_ui_image_min0.028051045018370007
in-drivable-lane_max49.899999999998805
in-drivable-lane_mean30.337499999999192
in-drivable-lane_min5.799999999999848
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.911610443305657, "get_ui_image": 0.03159091712830962, "step_physics": 0.12164582340643865, "survival_time": 59.99999999999873, "driven_lanedir": 4.629189276914046, "get_state_dump": 0.004961634952757976, "get_robot_state": 0.004020551559232256, "sim_render-ego0": 0.0040629876841117105, "get_duckie_state": 2.09157214772195e-06, "in-drivable-lane": 36.29999999999957, "deviation-heading": 5.2843066887809504, "agent_compute-ego0": 0.0127446627636734, "complete-iteration": 0.19356632927474532, "set_robot_commands": 0.002471381480449642, "deviation-center-line": 1.6305575464059787, "driven_lanedir_consec": 4.629189276914046, "sim_compute_sim_state": 0.009767136704812536, "sim_compute_performance-ego0": 0.002210195812952707}, "LF-norm-zigzag-000-ego0": {"driven_any": 10.664846638722482, "get_ui_image": 0.03773462504371815, "step_physics": 0.13978359562272732, "survival_time": 59.99999999999873, "driven_lanedir": 1.598467196321082, "get_state_dump": 0.0047757534262143405, "get_robot_state": 0.003905337815677792, "sim_render-ego0": 0.0039558343148846905, "get_duckie_state": 2.0149446943221145e-06, "in-drivable-lane": 49.899999999998805, "deviation-heading": 3.948577605894699, "agent_compute-ego0": 0.012559830199471125, "complete-iteration": 0.22096155982132656, "set_robot_commands": 0.0023559338842006053, "deviation-center-line": 0.6037943206773542, "driven_lanedir_consec": 1.5832900433886772, "sim_compute_sim_state": 0.013631855022103264, "sim_compute_performance-ego0": 0.002167909171162398}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.091502282119721, "get_ui_image": 0.03567161134710894, "step_physics": 0.1470126408926198, "survival_time": 42.54999999999972, "driven_lanedir": 6.144051747897905, "get_state_dump": 0.0049006757042217705, "get_robot_state": 0.004060180534219518, "sim_render-ego0": 0.004124155626610411, "get_duckie_state": 2.100434101803202e-06, "in-drivable-lane": 5.799999999999848, "deviation-heading": 10.592616319606828, "agent_compute-ego0": 0.012997586122700864, "complete-iteration": 0.2275347544553694, "set_robot_commands": 0.0024951257056473565, "deviation-center-line": 2.7238486397930455, "driven_lanedir_consec": 6.144051747897905, "sim_compute_sim_state": 0.013901254380812668, "sim_compute_performance-ego0": 0.002275713172876779}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.088690685254178, "get_ui_image": 0.028051045018370007, "step_physics": 0.1125825706072195, "survival_time": 59.99999999999873, "driven_lanedir": 4.885302297026026, "get_state_dump": 0.004905329060296433, "get_robot_state": 0.0039551289849833185, "sim_render-ego0": 0.003959217039770528, "get_duckie_state": 2.063382773673306e-06, "in-drivable-lane": 29.34999999999854, "deviation-heading": 10.196090591056482, "agent_compute-ego0": 0.012386739303626982, "complete-iteration": 0.1770377272273182, "set_robot_commands": 0.002400169364617925, "deviation-center-line": 2.2319277292043633, "driven_lanedir_consec": 4.885302297026026, "sim_compute_sim_state": 0.0065443376815884835, "sim_compute_performance-ego0": 0.002155454232234145}}
set_robot_commands_max0.0024951257056473565
set_robot_commands_mean0.0024306526087288817
set_robot_commands_median0.0024357754225337834
set_robot_commands_min0.0023559338842006053
sim_compute_performance-ego0_max0.002275713172876779
sim_compute_performance-ego0_mean0.002202318097306507
sim_compute_performance-ego0_median0.0021890524920575526
sim_compute_performance-ego0_min0.002155454232234145
sim_compute_sim_state_max0.013901254380812668
sim_compute_sim_state_mean0.010961145947329238
sim_compute_sim_state_median0.0116994958634579
sim_compute_sim_state_min0.0065443376815884835
sim_render-ego0_max0.004124155626610411
sim_render-ego0_mean0.004025548666344335
sim_render-ego0_median0.004011102361941119
sim_render-ego0_min0.0039558343148846905
simulation-passed1
step_physics_max0.1470126408926198
step_physics_mean0.13025615763225137
step_physics_median0.13071470951458297
step_physics_min0.1125825706072195
survival_time_max59.99999999999873
survival_time_mean55.63749999999897
survival_time_min42.54999999999972
No reset possible
5394711704brian wongtemplate-randomaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-060:04:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5040495128145347
survival_time_median4.6499999999999915
deviation-center-line_median0.10620228737371736
in-drivable-lane_median2.674999999999992


other stats
agent_compute-ego0_max0.013356043232811822
agent_compute-ego0_mean0.01198921393789203
agent_compute-ego0_median0.01158633831306852
agent_compute-ego0_min0.011428135892619255
complete-iteration_max0.22908314932947577
complete-iteration_mean0.20618433791202995
complete-iteration_median0.20600439872168289
complete-iteration_min0.18364540487527847
deviation-center-line_max0.16455763165774434
deviation-center-line_mean0.1092777488823554
deviation-center-line_min0.06014878912424249
deviation-heading_max1.819139548495243
deviation-heading_mean0.860079214882594
deviation-heading_median0.6517793617099584
deviation-heading_min0.31761858761521594
driven_any_max3.078535752374254
driven_any_mean1.8526223552312917
driven_any_median1.6219947547238958
driven_any_min1.0879641591031215
driven_lanedir_consec_max0.6643282900394009
driven_lanedir_consec_mean0.4884432809313369
driven_lanedir_consec_min0.28134580805687714
driven_lanedir_max0.6643282900394009
driven_lanedir_mean0.4924956463812869
driven_lanedir_median0.5121542437144349
driven_lanedir_min0.28134580805687714
get_duckie_state_max1.8243548236315763e-06
get_duckie_state_mean1.6274045596303236e-06
get_duckie_state_median1.5839142499914492e-06
get_duckie_state_min1.5174349149068196e-06
get_robot_state_max0.004309624433517456
get_robot_state_mean0.0041891593204276215
get_robot_state_median0.004149497923067802
get_robot_state_min0.004148017002057426
get_state_dump_max0.005535158846113417
get_state_dump_mean0.005413499017070829
get_state_dump_median0.005386453499145146
get_state_dump_min0.005345930223879607
get_ui_image_max0.03895096675209377
get_ui_image_mean0.03419139740639729
get_ui_image_median0.03306329665304739
get_ui_image_min0.03168802956740061
in-drivable-lane_max6.549999999999982
in-drivable-lane_mean3.3499999999999894
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.078535752374254, "get_ui_image": 0.03185684318783917, "step_physics": 0.11999539182155948, "survival_time": 7.84999999999998, "driven_lanedir": 0.28134580805687714, "get_state_dump": 0.005421818057193032, "get_robot_state": 0.004148017002057426, "sim_render-ego0": 0.004246281672127639, "get_duckie_state": 1.8243548236315763e-06, "in-drivable-lane": 6.549999999999982, "deviation-heading": 0.891006057417327, "agent_compute-ego0": 0.011634076697916924, "complete-iteration": 0.1931107693080661, "set_robot_commands": 0.0024577333957334106, "deviation-center-line": 0.13094886067496936, "driven_lanedir_consec": 0.28134580805687714, "sim_compute_sim_state": 0.010860168481174904, "sim_compute_performance-ego0": 0.0023903394047218034}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5806581102909054, "get_ui_image": 0.03895096675209377, "step_physics": 0.14987939855326776, "survival_time": 4.549999999999992, "driven_lanedir": 0.3891269085945981, "get_state_dump": 0.005345930223879607, "get_robot_state": 0.004150952981865924, "sim_render-ego0": 0.004072635070137356, "get_duckie_state": 1.6015508900518005e-06, "in-drivable-lane": 2.449999999999993, "deviation-heading": 1.819139548495243, "agent_compute-ego0": 0.011428135892619255, "complete-iteration": 0.22908314932947577, "set_robot_commands": 0.0024754249531289806, "deviation-center-line": 0.16455763165774434, "driven_lanedir_consec": 0.37291744679479777, "sim_compute_sim_state": 0.010380545388097347, "sim_compute_performance-ego0": 0.00229871013890142}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0879641591031215, "get_ui_image": 0.034269750118255615, "step_physics": 0.1439243886205885, "survival_time": 3.5499999999999954, "driven_lanedir": 0.6643282900394009, "get_state_dump": 0.005535158846113417, "get_robot_state": 0.00414804286426968, "sim_render-ego0": 0.004327939616309272, "get_duckie_state": 1.566277609931098e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.41255266600258983, "agent_compute-ego0": 0.013356043232811822, "complete-iteration": 0.21889802813529968, "set_robot_commands": 0.0024614897039201525, "deviation-center-line": 0.08145571407246535, "driven_lanedir_consec": 0.6643282900394009, "sim_compute_sim_state": 0.00851032469007704, "sim_compute_performance-ego0": 0.00226554274559021}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6633313991568863, "get_ui_image": 0.03168802956740061, "step_physics": 0.11614755541086196, "survival_time": 4.749999999999991, "driven_lanedir": 0.6351815788342716, "get_state_dump": 0.0053510889410972595, "get_robot_state": 0.004309624433517456, "sim_render-ego0": 0.004314104715983073, "get_duckie_state": 1.5174349149068196e-06, "in-drivable-lane": 2.89999999999999, "deviation-heading": 0.31761858761521594, "agent_compute-ego0": 0.011538599928220114, "complete-iteration": 0.18364540487527847, "set_robot_commands": 0.0025628159443537393, "deviation-center-line": 0.06014878912424249, "driven_lanedir_consec": 0.6351815788342716, "sim_compute_sim_state": 0.00535436471303304, "sim_compute_performance-ego0": 0.00227973610162735}}
set_robot_commands_max0.0025628159443537393
set_robot_commands_mean0.0024893659992840707
set_robot_commands_median0.0024684573285245663
set_robot_commands_min0.0024577333957334106
sim_compute_performance-ego0_max0.0023903394047218034
sim_compute_performance-ego0_mean0.002308582097710196
sim_compute_performance-ego0_median0.002289223120264385
sim_compute_performance-ego0_min0.00226554274559021
sim_compute_sim_state_max0.010860168481174904
sim_compute_sim_state_mean0.008776350818095585
sim_compute_sim_state_median0.009445435039087194
sim_compute_sim_state_min0.00535436471303304
sim_render-ego0_max0.004327939616309272
sim_render-ego0_mean0.004240240268639335
sim_render-ego0_median0.004280193194055357
sim_render-ego0_min0.004072635070137356
simulation-passed1
step_physics_max0.14987939855326776
step_physics_mean0.13248668360156943
step_physics_median0.131959890221074
step_physics_min0.11614755541086196
survival_time_max7.84999999999998
survival_time_mean5.17499999999999
survival_time_min3.5499999999999954
No reset possible
5391511315Anastasiya Nikolskaya 🇷🇺JetBrains Researchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-060:02:56
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible