Duckietown Challenges Home Challenges Submissions

Evaluator 4898

ID4898
evaluatornogpu-prod-08
ownerI don't have one 😀
machinenogpu-prod_87263d5c91f4
processnogpu-prod-08_87263d5c91f4
version6.2.5
first heard
last heard
statusinactive
# evaluating
# success18 53958
# timeout1 55680
# failed4 54188
# error
# aborted2 57385
# host-error38 53934
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5757711010Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-080:00:49
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5755611313Moustafa Elarabitemplate-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-080:03:33
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5739810940Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:42:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.012862631323732604
agent_compute-ego0_mean0.012203886943693265
agent_compute-ego0_median0.01212682612829661
agent_compute-ego0_min0.011699264194447234
complete-iteration_max0.3432449381317723
complete-iteration_mean0.29142712340763866
complete-iteration_median0.29595185418013825
complete-iteration_min0.2305598471385057
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max1.423960422893845e-06
get_duckie_state_mean1.2975052731122503e-06
get_duckie_state_median1.2681247987516913e-06
get_duckie_state_min1.2298110720517732e-06
get_robot_state_max0.00392588250940785
get_robot_state_mean0.003752373487725048
get_robot_state_median0.0037329707117898575
get_robot_state_min0.003617670017912624
get_state_dump_max0.004818460923448193
get_state_dump_mean0.004670470481510465
get_state_dump_median0.004628894728089649
get_state_dump_min0.004605631546414365
get_ui_image_max0.034856457595126414
get_ui_image_mean0.030319376700525976
get_ui_image_median0.030399745449634714
get_ui_image_min0.025621558307708053
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.028107070406708096, "step_physics": 0.20901547085732644, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004645208434995068, "get_robot_state": 0.003701644178036349, "sim_render-ego0": 0.003908043995586462, "get_duckie_state": 1.2792417349962272e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.011699264194447234, "complete-iteration": 0.27392111074716025, "set_robot_commands": 0.0022511335336397728, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008486586149090236, "sim_compute_performance-ego0": 0.0020210600018402023}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.034856457595126414, "step_physics": 0.26713190309014745, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.004612581021184231, "get_robot_state": 0.0037642972455433662, "sim_render-ego0": 0.004047648297261438, "get_duckie_state": 1.2570078625071554e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.012862631323732604, "complete-iteration": 0.3432449381317723, "set_robot_commands": 0.0022616616692967854, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.011540664026481127, "sim_compute_performance-ego0": 0.002079623426426261}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.03269242049256133, "step_physics": 0.24680338592751636, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004818460923448193, "get_robot_state": 0.00392588250940785, "sim_render-ego0": 0.00408606743633896, "get_duckie_state": 1.423960422893845e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.011845742534538989, "complete-iteration": 0.3179825976131163, "set_robot_commands": 0.002370847651206087, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.009169096950686642, "sim_compute_performance-ego0": 0.0021770433224210335}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.025621558307708053, "step_physics": 0.17046374047824883, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004605631546414365, "get_robot_state": 0.003617670017912624, "sim_render-ego0": 0.003783597437964192, "get_duckie_state": 1.2298110720517732e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.012407909722054232, "complete-iteration": 0.2305598471385057, "set_robot_commands": 0.0021417067509507458, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.005921663590811571, "sim_compute_performance-ego0": 0.0019153750607810549}}
set_robot_commands_max0.002370847651206087
set_robot_commands_mean0.0022563374012733476
set_robot_commands_median0.002256397601468279
set_robot_commands_min0.0021417067509507458
sim_compute_performance-ego0_max0.0021770433224210335
sim_compute_performance-ego0_mean0.002048275452867138
sim_compute_performance-ego0_median0.0020503417141332315
sim_compute_performance-ego0_min0.0019153750607810549
sim_compute_sim_state_max0.011540664026481127
sim_compute_sim_state_mean0.008779502679267396
sim_compute_sim_state_median0.00882784154988844
sim_compute_sim_state_min0.005921663590811571
sim_render-ego0_max0.00408606743633896
sim_render-ego0_mean0.003956339291787763
sim_render-ego0_median0.0039778461464239495
sim_render-ego0_min0.003783597437964192
simulation-passed1
step_physics_max0.26713190309014745
step_physics_mean0.22335362508830975
step_physics_median0.2279094283924214
step_physics_min0.17046374047824883
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5738511012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-080:00:57
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5737111426Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFP-sim-testingLFP-simhost-erroryesnogpu-prod-080:00:48
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5727510971Himanshu Arora 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:20:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.9718893027639313
survival_time_median23.87500000000021
deviation-center-line_median0.9178116325719918
in-drivable-lane_median9.2500000000001


other stats
agent_compute-ego0_max0.012949728151840732
agent_compute-ego0_mean0.012624206284459595
agent_compute-ego0_median0.01261363657200394
agent_compute-ego0_min0.012319823841989775
complete-iteration_max0.20958893540976703
complete-iteration_mean0.1860899186823884
complete-iteration_median0.18975302721536752
complete-iteration_min0.15526468488905165
deviation-center-line_max3.30156073668644
deviation-center-line_mean1.3527100454944958
deviation-center-line_min0.27365618014755916
deviation-heading_max13.241781919695828
deviation-heading_mean6.08724122897049
deviation-heading_median4.281378552978705
deviation-heading_min2.5444258902287245
driven_any_max11.939654093893743
driven_any_mean5.548646145044145
driven_any_median4.426212926905213
driven_any_min1.402504632472414
driven_lanedir_consec_max7.298712670406824
driven_lanedir_consec_mean3.475150078092932
driven_lanedir_consec_min0.6581090364370406
driven_lanedir_max9.46556575695822
driven_lanedir_mean4.029170144090239
driven_lanedir_median2.990888460606695
driven_lanedir_min0.6693378981893461
get_duckie_state_max1.3580528723012398e-06
get_duckie_state_mean1.3197056825342077e-06
get_duckie_state_median1.3154773896443096e-06
get_duckie_state_min1.2898150785469715e-06
get_robot_state_max0.003970416161936586
get_robot_state_mean0.0037503936589872782
get_robot_state_median0.003718087998565269
get_robot_state_min0.0035949824768819927
get_state_dump_max0.005088315419015241
get_state_dump_mean0.004752770880201816
get_state_dump_median0.00469775152379188
get_state_dump_min0.004527265054208261
get_ui_image_max0.03674836320001722
get_ui_image_mean0.03138289507005119
get_ui_image_median0.03146625242868501
get_ui_image_min0.025850712222817505
in-drivable-lane_max14.499999999999734
in-drivable-lane_mean9.3875
in-drivable-lane_min4.550000000000065
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.514601277167822, "get_ui_image": 0.029698107121647267, "step_physics": 0.11370153617167818, "survival_time": 27.550000000000257, "driven_lanedir": 3.597414357669834, "get_state_dump": 0.004739126001579174, "get_robot_state": 0.0037494461605514303, "sim_render-ego0": 0.003897256609322368, "get_duckie_state": 1.3190767039423403e-06, "in-drivable-lane": 12.800000000000182, "deviation-heading": 2.747477700054773, "agent_compute-ego0": 0.01253616464310798, "complete-iteration": 0.18387602852738424, "set_robot_commands": 0.0021610588267229605, "deviation-center-line": 0.7954079326898623, "driven_lanedir_consec": 3.597414357669834, "sim_compute_sim_state": 0.011299182107483131, "sim_compute_performance-ego0": 0.0020062085511027903}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.402504632472414, "get_ui_image": 0.03674836320001722, "step_physics": 0.1337177073897947, "survival_time": 10.300000000000011, "driven_lanedir": 0.6693378981893461, "get_state_dump": 0.004656377046004585, "get_robot_state": 0.0036867298365791065, "sim_render-ego0": 0.003892707363994801, "get_duckie_state": 1.3118780753462786e-06, "in-drivable-lane": 5.70000000000002, "deviation-heading": 2.5444258902287245, "agent_compute-ego0": 0.0126911085008999, "complete-iteration": 0.20958893540976703, "set_robot_commands": 0.0021616825159045234, "deviation-center-line": 0.27365618014755916, "driven_lanedir_consec": 0.6581090364370406, "sim_compute_sim_state": 0.01001913420820006, "sim_compute_performance-ego0": 0.001928222352179928}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.939654093893743, "get_ui_image": 0.03323439773572275, "step_physics": 0.1191519609001852, "survival_time": 59.99999999999873, "driven_lanedir": 9.46556575695822, "get_state_dump": 0.005088315419015241, "get_robot_state": 0.003970416161936586, "sim_render-ego0": 0.0042779124051109145, "get_duckie_state": 1.3580528723012398e-06, "in-drivable-lane": 14.499999999999734, "deviation-heading": 13.241781919695828, "agent_compute-ego0": 0.012949728151840732, "complete-iteration": 0.1956300259033508, "set_robot_commands": 0.0023733537262623553, "deviation-center-line": 3.30156073668644, "driven_lanedir_consec": 7.298712670406824, "sim_compute_sim_state": 0.012274705202355176, "sim_compute_performance-ego0": 0.0022116017083542035}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.337824576642603, "get_ui_image": 0.025850712222817505, "step_physics": 0.09479959982412832, "survival_time": 20.200000000000152, "driven_lanedir": 2.384362563543557, "get_state_dump": 0.004527265054208261, "get_robot_state": 0.0035949824768819927, "sim_render-ego0": 0.00378519105322567, "get_duckie_state": 1.2898150785469715e-06, "in-drivable-lane": 4.550000000000065, "deviation-heading": 5.815279405902637, "agent_compute-ego0": 0.012319823841989775, "complete-iteration": 0.15526468488905165, "set_robot_commands": 0.002172745598687066, "deviation-center-line": 1.0402153324541217, "driven_lanedir_consec": 2.3463642478580287, "sim_compute_sim_state": 0.006161904629365897, "sim_compute_performance-ego0": 0.001969418702302156}}
set_robot_commands_max0.0023733537262623553
set_robot_commands_mean0.002217210166894226
set_robot_commands_median0.002167214057295794
set_robot_commands_min0.0021610588267229605
sim_compute_performance-ego0_max0.0022116017083542035
sim_compute_performance-ego0_mean0.0020288628284847693
sim_compute_performance-ego0_median0.001987813626702473
sim_compute_performance-ego0_min0.001928222352179928
sim_compute_sim_state_max0.012274705202355176
sim_compute_sim_state_mean0.009938731536851065
sim_compute_sim_state_median0.010659158157841596
sim_compute_sim_state_min0.006161904629365897
sim_render-ego0_max0.0042779124051109145
sim_render-ego0_mean0.003963266857913438
sim_render-ego0_median0.003894981986658584
sim_render-ego0_min0.00378519105322567
simulation-passed1
step_physics_max0.1337177073897947
step_physics_mean0.1153427010714466
step_physics_median0.11642674853593168
step_physics_min0.09479959982412832
survival_time_max59.99999999999873
survival_time_mean29.51249999999979
survival_time_min10.300000000000011
No reset possible
5719310994Mo Kleit 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:23:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.1277697640899405
survival_time_median48.94999999999936
deviation-center-line_median2.536891978106673
in-drivable-lane_median6.899999999999864


other stats
agent_compute-ego0_max0.014079959575946514
agent_compute-ego0_mean0.01313272599813678
agent_compute-ego0_median0.012863501016976631
agent_compute-ego0_min0.012723942382647334
complete-iteration_max0.22420351138481728
complete-iteration_mean0.18274766134032397
complete-iteration_median0.17164665744728314
complete-iteration_min0.16349381908191232
deviation-center-line_max3.678225659188895
deviation-center-line_mean2.250823202710445
deviation-center-line_min0.2512831954395374
deviation-heading_max10.05462820563008
deviation-heading_mean5.520864112590092
deviation-heading_median5.022132626540487
deviation-heading_min1.984562991649314
driven_any_max7.368708166334916
driven_any_mean4.6212740265044
driven_any_median5.369165900508057
driven_any_min0.3780561386665709
driven_lanedir_consec_max7.210217800104347
driven_lanedir_consec_mean3.914242708551812
driven_lanedir_consec_min0.19121350592301756
driven_lanedir_max7.210217800104347
driven_lanedir_mean3.914242708551812
driven_lanedir_median4.1277697640899405
driven_lanedir_min0.19121350592301756
get_duckie_state_max1.3498159555288462e-06
get_duckie_state_mean1.2734773834889917e-06
get_duckie_state_median1.2991275391826608e-06
get_duckie_state_min1.1458385000617974e-06
get_robot_state_max0.003937436984135554
get_robot_state_mean0.0038047718699595863
get_robot_state_median0.00379353894010914
get_robot_state_min0.003694572615484512
get_state_dump_max0.004963218248807467
get_state_dump_mean0.004762776072739434
get_state_dump_median0.004766474259538931
get_state_dump_min0.004554937523072407
get_ui_image_max0.03794208123133733
get_ui_image_mean0.031258061297821
get_ui_image_median0.030078324578037677
get_ui_image_min0.02693351480387133
in-drivable-lane_max22.049999999998786
in-drivable-lane_mean8.962499999999629
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.229655732993957, "get_ui_image": 0.02816135327335403, "step_physics": 0.0974967247255549, "survival_time": 37.899999999999984, "driven_lanedir": 4.419113809752394, "get_state_dump": 0.004746486704025972, "get_robot_state": 0.003773318606088912, "sim_render-ego0": 0.003898670550862791, "get_duckie_state": 1.3211970272742712e-06, "in-drivable-lane": 10.949999999999736, "deviation-heading": 5.101525857815931, "agent_compute-ego0": 0.012771739633029943, "complete-iteration": 0.16574943018525015, "set_robot_commands": 0.002209734068557679, "deviation-center-line": 1.8753910653516976, "driven_lanedir_consec": 4.419113809752394, "sim_compute_sim_state": 0.010612854373313692, "sim_compute_performance-ego0": 0.0019885541893276772}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.3780561386665709, "get_ui_image": 0.03794208123133733, "step_physics": 0.14297171372633713, "survival_time": 6.449999999999985, "driven_lanedir": 0.19121350592301756, "get_state_dump": 0.004963218248807467, "get_robot_state": 0.003937436984135554, "sim_render-ego0": 0.00426784662099985, "get_duckie_state": 1.3498159555288462e-06, "in-drivable-lane": 2.84999999999999, "deviation-heading": 1.984562991649314, "agent_compute-ego0": 0.014079959575946514, "complete-iteration": 0.22420351138481728, "set_robot_commands": 0.0024552308596097507, "deviation-center-line": 0.2512831954395374, "driven_lanedir_consec": 0.19121350592301756, "sim_compute_sim_state": 0.011379685768714318, "sim_compute_performance-ego0": 0.002112406950730544}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.368708166334916, "get_ui_image": 0.03199529588272133, "step_physics": 0.10336157959962664, "survival_time": 59.99999999999873, "driven_lanedir": 7.210217800104347, "get_state_dump": 0.004554937523072407, "get_robot_state": 0.003694572615484512, "sim_render-ego0": 0.003934425676395057, "get_duckie_state": 1.1458385000617974e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.05462820563008, "agent_compute-ego0": 0.012723942382647334, "complete-iteration": 0.17754388470931612, "set_robot_commands": 0.0022292677111470827, "deviation-center-line": 3.678225659188895, "driven_lanedir_consec": 7.210217800104347, "sim_compute_sim_state": 0.01294761712505458, "sim_compute_performance-ego0": 0.0020174352850743276}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.508676068022157, "get_ui_image": 0.02693351480387133, "step_physics": 0.10027298423074664, "survival_time": 59.99999999999873, "driven_lanedir": 3.836425718427487, "get_state_dump": 0.004786461815051889, "get_robot_state": 0.003813759274129367, "sim_render-ego0": 0.0040089077596164166, "get_duckie_state": 1.2770580510910503e-06, "in-drivable-lane": 22.049999999998786, "deviation-heading": 4.942739395265043, "agent_compute-ego0": 0.012955262400923324, "complete-iteration": 0.16349381908191232, "set_robot_commands": 0.00224127876669243, "deviation-center-line": 3.198392890861649, "driven_lanedir_consec": 3.836425718427487, "sim_compute_sim_state": 0.006362478103764746, "sim_compute_performance-ego0": 0.0020304073600546702}}
set_robot_commands_max0.0024552308596097507
set_robot_commands_mean0.0022838778515017356
set_robot_commands_median0.002235273238919756
set_robot_commands_min0.002209734068557679
sim_compute_performance-ego0_max0.002112406950730544
sim_compute_performance-ego0_mean0.0020372009462968047
sim_compute_performance-ego0_median0.002023921322564499
sim_compute_performance-ego0_min0.0019885541893276772
sim_compute_sim_state_max0.01294761712505458
sim_compute_sim_state_mean0.010325658842711834
sim_compute_sim_state_median0.010996270071014006
sim_compute_sim_state_min0.006362478103764746
sim_render-ego0_max0.00426784662099985
sim_render-ego0_mean0.004027462651968529
sim_render-ego0_median0.003971666718005737
sim_render-ego0_min0.003898670550862791
simulation-passed1
step_physics_max0.14297171372633713
step_physics_mean0.11102575057056634
step_physics_median0.10181728191518664
step_physics_min0.0974967247255549
survival_time_max59.99999999999873
survival_time_mean41.08749999999936
survival_time_min6.449999999999985
No reset possible
5718711312Moustafa Elarabitemplate-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-080:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5702511038Bhavya Patwa 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:25:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.790476280274176
survival_time_median44.749999999999595
deviation-center-line_median2.05180352975609
in-drivable-lane_median15.62499999999951


other stats
agent_compute-ego0_max0.012896027514543483
agent_compute-ego0_mean0.012326272422582631
agent_compute-ego0_median0.012220774941530049
agent_compute-ego0_min0.011967512292726946
complete-iteration_max0.20858421611151515
complete-iteration_mean0.18325947003530188
complete-iteration_median0.18651308049827373
complete-iteration_min0.1514275030331449
deviation-center-line_max3.5782412413831253
deviation-center-line_mean2.0886152352587386
deviation-center-line_min0.6726126401396495
deviation-heading_max10.666949346074798
deviation-heading_mean6.633426540326097
deviation-heading_median6.836941136048931
deviation-heading_min2.192874543131726
driven_any_max8.876319110394027
driven_any_mean6.362117034341232
driven_any_median6.778870016991728
driven_any_min3.014408992987444
driven_lanedir_consec_max6.6072686992005
driven_lanedir_consec_mean3.9067847748560895
driven_lanedir_consec_min1.438917839675505
driven_lanedir_max6.6072686992005
driven_lanedir_mean4.2012155689055035
driven_lanedir_median4.379337868373003
driven_lanedir_min1.438917839675505
get_duckie_state_max1.4388159419713397e-06
get_duckie_state_mean1.331471213836922e-06
get_duckie_state_median1.3165194008244897e-06
get_duckie_state_min1.254030111727369e-06
get_robot_state_max0.003736273952262112
get_robot_state_mean0.0036718653607193097
get_robot_state_median0.0036782688349570785
get_robot_state_min0.0035946498207009704
get_state_dump_max0.0048322223481677825
get_state_dump_mean0.004645906765888063
get_state_dump_median0.004675651543129542
get_state_dump_min0.004400101629125387
get_ui_image_max0.036241849087294345
get_ui_image_mean0.030725993968471095
get_ui_image_median0.03073814313680024
get_ui_image_min0.02518584051298956
in-drivable-lane_max25.299999999999905
in-drivable-lane_mean16.949999999999754
in-drivable-lane_min11.250000000000092
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.393060307867997, "get_ui_image": 0.028374735157141524, "step_physics": 0.10484269736857896, "survival_time": 44.44999999999961, "driven_lanedir": 4.052212095876813, "get_state_dump": 0.004691294070040242, "get_robot_state": 0.003718116845977441, "sim_render-ego0": 0.0039825452847427195, "get_duckie_state": 1.4388159419713397e-06, "in-drivable-lane": 25.299999999999905, "deviation-heading": 4.124375198859948, "agent_compute-ego0": 0.012414176812332667, "complete-iteration": 0.1735388305749786, "set_robot_commands": 0.0022504852059182156, "deviation-center-line": 1.2123534154087356, "driven_lanedir_consec": 4.052212095876813, "sim_compute_sim_state": 0.011085930834995228, "sim_compute_performance-ego0": 0.00209134535843067}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.164679726115459, "get_ui_image": 0.036241849087294345, "step_physics": 0.13255872721153986, "survival_time": 45.04999999999958, "driven_lanedir": 4.706463640869194, "get_state_dump": 0.004660009016218841, "get_robot_state": 0.003638420823936716, "sim_render-ego0": 0.003885145197950815, "get_duckie_state": 1.3160599837546335e-06, "in-drivable-lane": 11.449999999999733, "deviation-heading": 10.666949346074798, "agent_compute-ego0": 0.01202737307072743, "complete-iteration": 0.20858421611151515, "set_robot_commands": 0.0021936452045144632, "deviation-center-line": 2.891253644103444, "driven_lanedir_consec": 3.52874046467154, "sim_compute_sim_state": 0.01131321294873358, "sim_compute_performance-ego0": 0.0019790177334702993}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.014408992987444, "get_ui_image": 0.033101551116458953, "step_physics": 0.12601130853885065, "survival_time": 18.850000000000133, "driven_lanedir": 1.438917839675505, "get_state_dump": 0.0048322223481677825, "get_robot_state": 0.003736273952262112, "sim_render-ego0": 0.0040117596823071675, "get_duckie_state": 1.3169788178943452e-06, "in-drivable-lane": 11.250000000000092, "deviation-heading": 2.192874543131726, "agent_compute-ego0": 0.012896027514543483, "complete-iteration": 0.19948733042156885, "set_robot_commands": 0.0023055770409801018, "deviation-center-line": 0.6726126401396495, "driven_lanedir_consec": 1.438917839675505, "sim_compute_sim_state": 0.010496261889341649, "sim_compute_performance-ego0": 0.0020108771702599905}, "LF-norm-small_loop-000-ego0": {"driven_any": 8.876319110394027, "get_ui_image": 0.02518584051298956, "step_physics": 0.09236094298509634, "survival_time": 59.99999999999873, "driven_lanedir": 6.6072686992005, "get_state_dump": 0.004400101629125387, "get_robot_state": 0.0035946498207009704, "sim_render-ego0": 0.003792084027686584, "get_duckie_state": 1.254030111727369e-06, "in-drivable-lane": 19.799999999999287, "deviation-heading": 9.549507073237914, "agent_compute-ego0": 0.011967512292726946, "complete-iteration": 0.1514275030331449, "set_robot_commands": 0.002162385641982613, "deviation-center-line": 3.5782412413831253, "driven_lanedir_consec": 6.6072686992005, "sim_compute_sim_state": 0.005928787760293851, "sim_compute_performance-ego0": 0.0019558760446870853}}
set_robot_commands_max0.0023055770409801018
set_robot_commands_mean0.0022280232733488487
set_robot_commands_median0.002222065205216339
set_robot_commands_min0.002162385641982613
sim_compute_performance-ego0_max0.00209134535843067
sim_compute_performance-ego0_mean0.002009279076712011
sim_compute_performance-ego0_median0.0019949474518651446
sim_compute_performance-ego0_min0.0019558760446870853
sim_compute_sim_state_max0.01131321294873358
sim_compute_sim_state_mean0.009706048358341076
sim_compute_sim_state_median0.010791096362168438
sim_compute_sim_state_min0.005928787760293851
sim_render-ego0_max0.0040117596823071675
sim_render-ego0_mean0.003917883548171822
sim_render-ego0_median0.003933845241346767
sim_render-ego0_min0.003792084027686584
simulation-passed1
step_physics_max0.13255872721153986
step_physics_mean0.11394341902601646
step_physics_median0.1154270029537148
step_physics_min0.09236094298509634
survival_time_max59.99999999999873
survival_time_mean42.08749999999951
survival_time_min18.850000000000133
No reset possible
5692611056Bhavya Patwa 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-080:20:08
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 2270 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5691411427Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-080:01:01
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5667911096Bhavya Patwa 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:42:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.012604466683660122
agent_compute-ego0_mean0.011968945732323155
agent_compute-ego0_median0.011837205620828416
agent_compute-ego0_min0.01159690500397567
complete-iteration_max0.3115072571963295
complete-iteration_mean0.2832269908288039
complete-iteration_median0.2882279803810469
complete-iteration_min0.2449447453567924
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max1.579399013598694e-06
get_duckie_state_mean1.3692194377254389e-06
get_duckie_state_median1.346935936056704e-06
get_duckie_state_min1.2036068651896532e-06
get_robot_state_max0.003837642820550441
get_robot_state_mean0.003640494130235429
get_robot_state_median0.003641253109280017
get_robot_state_min0.0034418274818312417
get_state_dump_max0.004770462757145534
get_state_dump_mean0.004608274449118964
get_state_dump_median0.004644472732035743
get_state_dump_min0.004373689575258838
get_ui_image_max0.0324165156044432
get_ui_image_mean0.02937898037336351
get_ui_image_median0.029487499686502397
get_ui_image_min0.02612440651600605
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.027858097487742657, "step_physics": 0.21143863084016493, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.00457656869880365, "get_robot_state": 0.003680157522476285, "sim_render-ego0": 0.003865313271896527, "get_duckie_state": 1.3711549757323e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.01162175671643361, "complete-iteration": 0.27592688039577973, "set_robot_commands": 0.0021989575829930745, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008622128401668145, "sim_compute_performance-ego0": 0.001979195009560311}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.0324165156044432, "step_physics": 0.24068291022517505, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.004373689575258838, "get_robot_state": 0.0034418274818312417, "sim_render-ego0": 0.0037389609537751946, "get_duckie_state": 1.2036068651896532e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.012052654525223224, "complete-iteration": 0.3115072571963295, "set_robot_commands": 0.002025704300473076, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.010811406507976446, "sim_compute_performance-ego0": 0.00188135505218887}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.03111690188526214, "step_physics": 0.23279065335422233, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004712376765267835, "get_robot_state": 0.0036023486960837484, "sim_render-ego0": 0.00381269085714958, "get_duckie_state": 1.3227168963811082e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.01159690500397567, "complete-iteration": 0.3005290803663141, "set_robot_commands": 0.0021225441306159457, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.00874991956896627, "sim_compute_performance-ego0": 0.0019414869573690016}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.02612440651600605, "step_physics": 0.1828346675282017, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004770462757145534, "get_robot_state": 0.003837642820550441, "sim_render-ego0": 0.003945991061907029, "get_duckie_state": 1.579399013598694e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.012604466683660122, "complete-iteration": 0.2449447453567924, "set_robot_commands": 0.0022856087410678275, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.006345610733731005, "sim_compute_performance-ego0": 0.002102429424098489}}
set_robot_commands_max0.0022856087410678275
set_robot_commands_mean0.002158203688787481
set_robot_commands_median0.00216075085680451
set_robot_commands_min0.002025704300473076
sim_compute_performance-ego0_max0.002102429424098489
sim_compute_performance-ego0_mean0.0019761166108041676
sim_compute_performance-ego0_median0.001960340983464656
sim_compute_performance-ego0_min0.00188135505218887
sim_compute_sim_state_max0.010811406507976446
sim_compute_sim_state_mean0.008632266303085468
sim_compute_sim_state_median0.008686023985317207
sim_compute_sim_state_min0.006345610733731005
sim_render-ego0_max0.003945991061907029
sim_render-ego0_mean0.003840739036182083
sim_render-ego0_median0.0038390020645230538
sim_render-ego0_min0.0037389609537751946
simulation-passed1
step_physics_max0.24068291022517505
step_physics_mean0.216936715486941
step_physics_median0.2221146420971936
step_physics_min0.1828346675282017
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5662011098Philippe Reddy 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:07:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6686614624459106
survival_time_median7.124999999999984
deviation-center-line_median0.14935924738897116
in-drivable-lane_median4.874999999999986


other stats
agent_compute-ego0_max0.013465957641601562
agent_compute-ego0_mean0.012624226419589268
agent_compute-ego0_median0.012501366078669094
agent_compute-ego0_min0.012028215879417327
complete-iteration_max0.2174164027221932
complete-iteration_mean0.18492835734745833
complete-iteration_median0.1872249973475728
complete-iteration_min0.14784703197249446
deviation-center-line_max1.394593733362046
deviation-center-line_mean0.45106844153258696
deviation-center-line_min0.11096153799035945
deviation-heading_max4.953900075232931
deviation-heading_mean1.810497921508213
deviation-heading_median0.9406303546050492
deviation-heading_min0.40683090158982416
driven_any_max8.074637307154493
driven_any_mean3.0265895431368324
driven_any_median1.4817213211456424
driven_any_min1.0682782231015513
driven_lanedir_consec_max5.945704757629133
driven_lanedir_consec_mean1.9190326929568584
driven_lanedir_consec_min0.3931030893064811
driven_lanedir_max5.945704757629133
driven_lanedir_mean1.9190326929568584
driven_lanedir_median0.6686614624459106
driven_lanedir_min0.3931030893064811
get_duckie_state_max1.6307830810546877e-06
get_duckie_state_mean1.3492197772195903e-06
get_duckie_state_median1.3071091571664964e-06
get_duckie_state_min1.1518777134906814e-06
get_robot_state_max0.0040290760993957515
get_robot_state_mean0.0037104290547791833
get_robot_state_median0.003618468686027145
get_robot_state_min0.003575702747666692
get_state_dump_max0.005117356777191162
get_state_dump_mean0.004759826706920551
get_state_dump_median0.004684369873023474
get_state_dump_min0.004553210304444094
get_ui_image_max0.03453836756304276
get_ui_image_mean0.03055791926964524
get_ui_image_median0.03077604699790273
get_ui_image_min0.02614121551973274
in-drivable-lane_max8.450000000000104
in-drivable-lane_mean5.187500000000017
in-drivable-lane_min2.549999999999991
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.074637307154493, "get_ui_image": 0.028221732152696327, "step_physics": 0.103642029860585, "survival_time": 29.05000000000028, "driven_lanedir": 5.945704757629133, "get_state_dump": 0.004701658622505739, "get_robot_state": 0.0036428679305663225, "sim_render-ego0": 0.0038759749369932615, "get_duckie_state": 1.3965101995828634e-06, "in-drivable-lane": 8.450000000000104, "deviation-heading": 4.953900075232931, "agent_compute-ego0": 0.012449233392669573, "complete-iteration": 0.16972774041887, "set_robot_commands": 0.002199939845763531, "deviation-center-line": 1.394593733362046, "driven_lanedir_consec": 5.945704757629133, "sim_compute_sim_state": 0.008899567872797911, "sim_compute_performance-ego0": 0.0019988214846738835}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3011519868691592, "get_ui_image": 0.03453836756304276, "step_physics": 0.14416496221684227, "survival_time": 5.999999999999987, "driven_lanedir": 0.3931030893064811, "get_state_dump": 0.004667081123541209, "get_robot_state": 0.0035940694414879665, "sim_render-ego0": 0.003935677946106462, "get_duckie_state": 1.2177081147501292e-06, "in-drivable-lane": 3.8999999999999897, "deviation-heading": 1.452674109991012, "agent_compute-ego0": 0.012553498764668616, "complete-iteration": 0.2174164027221932, "set_robot_commands": 0.0022152120416814632, "deviation-center-line": 0.16921022183840148, "driven_lanedir_consec": 0.3931030893064811, "sim_compute_sim_state": 0.009671613204577736, "sim_compute_performance-ego0": 0.0019850632375922085}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0682782231015513, "get_ui_image": 0.033330361843109134, "step_physics": 0.1316469979286194, "survival_time": 4.94999999999999, "driven_lanedir": 0.6536742306847247, "get_state_dump": 0.005117356777191162, "get_robot_state": 0.0040290760993957515, "sim_render-ego0": 0.004159355163574218, "get_duckie_state": 1.6307830810546877e-06, "in-drivable-lane": 2.549999999999991, "deviation-heading": 0.40683090158982416, "agent_compute-ego0": 0.013465957641601562, "complete-iteration": 0.20472225427627563, "set_robot_commands": 0.0023818612098693848, "deviation-center-line": 0.11096153799035945, "driven_lanedir_consec": 0.6536742306847247, "sim_compute_sim_state": 0.008291704654693604, "sim_compute_performance-ego0": 0.0021931123733520508}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6622906554221255, "get_ui_image": 0.02614121551973274, "step_physics": 0.08883890042822044, "survival_time": 8.249999999999982, "driven_lanedir": 0.6836486942070965, "get_state_dump": 0.004553210304444094, "get_robot_state": 0.003575702747666692, "sim_render-ego0": 0.003780926566526114, "get_duckie_state": 1.1518777134906814e-06, "in-drivable-lane": 5.849999999999983, "deviation-heading": 0.4285865992190861, "agent_compute-ego0": 0.012028215879417327, "complete-iteration": 0.14784703197249446, "set_robot_commands": 0.0020993890532528064, "deviation-center-line": 0.12950827293954084, "driven_lanedir_consec": 0.6836486942070965, "sim_compute_sim_state": 0.004849561725754336, "sim_compute_performance-ego0": 0.0018923828400761247}}
set_robot_commands_max0.0023818612098693848
set_robot_commands_mean0.002224100537641796
set_robot_commands_median0.002207575943722497
set_robot_commands_min0.0020993890532528064
sim_compute_performance-ego0_max0.0021931123733520508
sim_compute_performance-ego0_mean0.0020173449839235667
sim_compute_performance-ego0_median0.0019919423611330462
sim_compute_performance-ego0_min0.0018923828400761247
sim_compute_sim_state_max0.009671613204577736
sim_compute_sim_state_mean0.007928111864455897
sim_compute_sim_state_median0.008595636263745758
sim_compute_sim_state_min0.004849561725754336
sim_render-ego0_max0.004159355163574218
sim_render-ego0_mean0.003937983653300014
sim_render-ego0_median0.0039058264415498616
sim_render-ego0_min0.003780926566526114
simulation-passed1
step_physics_max0.14416496221684227
step_physics_mean0.11707322260856678
step_physics_median0.1176445138946022
step_physics_min0.08883890042822044
survival_time_max29.05000000000028
survival_time_mean12.06250000000006
survival_time_min4.94999999999999
No reset possible
5653311109Daniil Lisusreal-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:08:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.48844402848486057
survival_time_median4.549999999999992
deviation-center-line_median0.1108373480286386
in-drivable-lane_median2.824999999999992


other stats
agent_compute-ego0_max0.01240293077520422
agent_compute-ego0_mean0.012083687844753416
agent_compute-ego0_median0.012236839317413695
agent_compute-ego0_min0.011458141968982054
complete-iteration_max0.1925520379859281
complete-iteration_mean0.16558549447889193
complete-iteration_median0.16250994918944975
complete-iteration_min0.14477004155074016
deviation-center-line_max0.1437210959637318
deviation-center-line_mean0.10586934320490124
deviation-center-line_min0.058081580798595926
deviation-heading_max1.1282098600551824
deviation-heading_mean0.6922977315969714
deviation-heading_median0.6712756467151071
deviation-heading_min0.298429772902489
driven_any_max3.023061004042608
driven_any_mean1.7599582113223078
driven_any_median1.4716982170444517
driven_any_min1.07337540715772
driven_lanedir_consec_max0.6438431345079845
driven_lanedir_consec_mean0.4738857200261674
driven_lanedir_consec_min0.27481168862696403
driven_lanedir_max0.6438431345079845
driven_lanedir_mean0.4738857200261674
driven_lanedir_median0.48844402848486057
driven_lanedir_min0.27481168862696403
get_duckie_state_max1.3127384415592056e-06
get_duckie_state_mean1.2243195012623083e-06
get_duckie_state_median1.2271028487542623e-06
get_duckie_state_min1.1303338659815042e-06
get_robot_state_max0.0034926196178758
get_robot_state_mean0.0034122237212111306
get_robot_state_median0.003392668096316425
get_robot_state_min0.0033709390743358715
get_state_dump_max0.004491851990481457
get_state_dump_mean0.004389588279379543
get_state_dump_median0.004443640592600783
get_state_dump_min0.004179219941835146
get_ui_image_max0.033338963267314864
get_ui_image_mean0.027846310018548027
get_ui_image_median0.027013066866156413
get_ui_image_min0.024020143074564414
in-drivable-lane_max6.8499999999999845
in-drivable-lane_mean3.5374999999999908
in-drivable-lane_min1.649999999999994
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.023061004042608, "get_ui_image": 0.02591254624975733, "step_physics": 0.0915846451219306, "survival_time": 8.249999999999982, "driven_lanedir": 0.27481168862696403, "get_state_dump": 0.004449412047144878, "get_robot_state": 0.0033924981772181497, "sim_render-ego0": 0.0036189211420266025, "get_duckie_state": 1.1303338659815042e-06, "in-drivable-lane": 6.8499999999999845, "deviation-heading": 1.0180204397553956, "agent_compute-ego0": 0.012280561837805322, "complete-iteration": 0.15414226773273515, "set_robot_commands": 0.0019380155816135637, "deviation-center-line": 0.1437210959637318, "driven_lanedir_consec": 0.27481168862696403, "sim_compute_sim_state": 0.00904747951461608, "sim_compute_performance-ego0": 0.0018374934253922429}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.2830189616286312, "get_ui_image": 0.033338963267314864, "step_physics": 0.12258351854531162, "survival_time": 4.099999999999993, "driven_lanedir": 0.36104707586777174, "get_state_dump": 0.004491851990481457, "get_robot_state": 0.0034926196178758, "sim_render-ego0": 0.003620607307158321, "get_duckie_state": 1.3127384415592056e-06, "in-drivable-lane": 2.4999999999999947, "deviation-heading": 1.1282098600551824, "agent_compute-ego0": 0.012193116797022072, "complete-iteration": 0.1925520379859281, "set_robot_commands": 0.002046323684324701, "deviation-center-line": 0.12058392075324084, "driven_lanedir_consec": 0.36104707586777174, "sim_compute_sim_state": 0.008799073207809264, "sim_compute_performance-ego0": 0.001901115279599845}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.07337540715772, "get_ui_image": 0.02811358748255549, "step_physics": 0.10849024154044488, "survival_time": 3.649999999999995, "driven_lanedir": 0.6438431345079845, "get_state_dump": 0.004179219941835146, "get_robot_state": 0.0033709390743358715, "sim_render-ego0": 0.003623176265407253, "get_duckie_state": 1.2597522220096074e-06, "in-drivable-lane": 1.649999999999994, "deviation-heading": 0.3245308536748185, "agent_compute-ego0": 0.01240293077520422, "complete-iteration": 0.17087763064616435, "set_robot_commands": 0.0019119977951049805, "deviation-center-line": 0.058081580798595926, "driven_lanedir_consec": 0.6438431345079845, "sim_compute_sim_state": 0.00693030292923386, "sim_compute_performance-ego0": 0.0017730964196694865}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6603774724602718, "get_ui_image": 0.024020143074564414, "step_physics": 0.08935988539516336, "survival_time": 4.99999999999999, "driven_lanedir": 0.6158409811019494, "get_state_dump": 0.004437869138056689, "get_robot_state": 0.0033928380154147008, "sim_render-ego0": 0.0035408086115771, "get_duckie_state": 1.194453475498917e-06, "in-drivable-lane": 3.1499999999999893, "deviation-heading": 0.298429772902489, "agent_compute-ego0": 0.011458141968982054, "complete-iteration": 0.14477004155074016, "set_robot_commands": 0.0019519635946443768, "deviation-center-line": 0.10109077530403636, "driven_lanedir_consec": 0.6158409811019494, "sim_compute_sim_state": 0.004752260623591961, "sim_compute_performance-ego0": 0.0017747760999320757}}
set_robot_commands_max0.002046323684324701
set_robot_commands_mean0.0019620751639219053
set_robot_commands_median0.0019449895881289705
set_robot_commands_min0.0019119977951049805
sim_compute_performance-ego0_max0.001901115279599845
sim_compute_performance-ego0_mean0.001821620306148412
sim_compute_performance-ego0_median0.0018061347626621592
sim_compute_performance-ego0_min0.0017730964196694865
sim_compute_sim_state_max0.00904747951461608
sim_compute_sim_state_mean0.007382279068812792
sim_compute_sim_state_median0.007864688068521562
sim_compute_sim_state_min0.004752260623591961
sim_render-ego0_max0.003623176265407253
sim_render-ego0_mean0.003600878331542319
sim_render-ego0_median0.003619764224592462
sim_render-ego0_min0.0035408086115771
simulation-passed1
step_physics_max0.12258351854531162
step_physics_mean0.1030045726507126
step_physics_median0.10003744333118772
step_physics_min0.08935988539516336
survival_time_max8.249999999999982
survival_time_mean5.24999999999999
survival_time_min3.649999999999995
No reset possible
5643111134Himanshu Arora 🇨🇦real-exercise-1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:06:15
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5637811425Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-080:02:19
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5629511138Mo Kleit 🇨🇦real-exercise-1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:04:18
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5625811133Mo Kleit 🇨🇦real-exercise-1aido5-LF-sim-validationLFv-simfailedyesnogpu-prod-080:02:41
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5622011278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:48
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5615811280Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:16
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5612711280Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:01:09
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5608111289Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:08
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5606011282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:22
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5603311283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:01:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5601811307Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:49
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5595811284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:31
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5592511296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:01:37
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5588311303Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5586311299Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5584811299Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5583211310Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:37
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5579811287Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:53
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5569211159Ayman Shams 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simfailedyesnogpu-prod-080:02:56
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5568011307Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-080:00:23
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5562411285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:39
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5560311311Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5556211450Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-testingLFt-simhost-erroryesnogpu-prod-080:00:32
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5552111286Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:33
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5548811290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5541311281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:03:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5536511291Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:37
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5534511305Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:44
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5514011413Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LFP-sim-validationLFP-simsuccessnonogpu-prod-080:12:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median8.025
in-drivable-lane_median0.5250000000000075
driven_lanedir_consec_median1.5425060846496597
deviation-center-line_median0.3402907324174555


other stats
agent_compute-ego0_max0.049546256432166466
agent_compute-ego0_mean0.04494713549358475
agent_compute-ego0_median0.044396806994210114
agent_compute-ego0_min0.04144867155375231
complete-iteration_max0.2986055667583759
complete-iteration_mean0.24263157856628953
complete-iteration_median0.24217770981250397
complete-iteration_min0.18756532788177413
deviation-center-line_max3.9835052701742866
deviation-center-line_mean1.2327409851460067
deviation-center-line_min0.26687720557482947
deviation-heading_max15.44160972654172
deviation-heading_mean4.822003085186283
deviation-heading_median1.4976910367237344
deviation-heading_min0.851020540755943
driven_any_max14.5898100863319
driven_any_mean4.63362024588694
driven_any_median1.7082199312507167
driven_any_min0.5282310347144248
driven_lanedir_consec_max11.088425839240465
driven_lanedir_consec_mean3.666707660339412
driven_lanedir_consec_min0.4933926328178654
driven_lanedir_max11.088425839240465
driven_lanedir_mean3.666707660339412
driven_lanedir_median1.5425060846496597
driven_lanedir_min0.4933926328178654
get_duckie_state_max0.025029385319122903
get_duckie_state_mean0.017736621000805784
get_duckie_state_median0.020870391979664067
get_duckie_state_min0.004176314724772101
get_robot_state_max0.0035704355973463792
get_robot_state_mean0.0035178357393001905
get_robot_state_median0.003529289708687709
get_robot_state_min0.0034423279424789645
get_state_dump_max0.00820276141166687
get_state_dump_mean0.007188714666381801
get_state_dump_median0.007766041947049042
get_state_dump_min0.005020013359762251
get_ui_image_max0.03549864475543683
get_ui_image_mean0.030336097437353594
get_ui_image_median0.03104837739647033
get_ui_image_min0.023748990201036897
in-drivable-lane_max11.099999999999756
in-drivable-lane_mean3.037499999999943
in-drivable-lane_min0.0
per-episodes
details{"LFP-norm-loop-000-ego0": {"driven_any": 2.276806063496427, "get_ui_image": 0.029535753222612236, "step_physics": 0.10708580223413612, "survival_time": 10.350000000000012, "driven_lanedir": 1.9665277864594513, "get_state_dump": 0.00820276141166687, "get_robot_state": 0.0035275048934496366, "sim_render-ego0": 0.0036281255575326774, "get_duckie_state": 0.025029385319122903, "in-drivable-lane": 1.050000000000015, "deviation-heading": 1.747008755262681, "agent_compute-ego0": 0.04420877993106842, "complete-iteration": 0.23261024745611045, "set_robot_commands": 0.002119761246901292, "deviation-center-line": 0.3724996356547582, "driven_lanedir_consec": 1.9665277864594513, "sim_compute_sim_state": 0.007262663199351384, "sim_compute_performance-ego0": 0.0019064350770070003}, "LFP-norm-zigzag-000-ego0": {"driven_any": 0.5282310347144248, "get_ui_image": 0.03549864475543683, "step_physics": 0.16400786913358248, "survival_time": 3.1999999999999966, "driven_lanedir": 0.4933926328178654, "get_state_dump": 0.007912613795353816, "get_robot_state": 0.0035704355973463792, "sim_render-ego0": 0.003772603548490084, "get_duckie_state": 0.020878175588754502, "in-drivable-lane": 0.0, "deviation-heading": 1.2483733181847878, "agent_compute-ego0": 0.049546256432166466, "complete-iteration": 0.2986055667583759, "set_robot_commands": 0.002207139822152945, "deviation-center-line": 0.26687720557482947, "driven_lanedir_consec": 0.4933926328178654, "sim_compute_sim_state": 0.009085996334369366, "sim_compute_performance-ego0": 0.002019302661602314}, "LFP-norm-techtrack-000-ego0": {"driven_any": 1.1396337990050067, "get_ui_image": 0.03256100157032842, "step_physics": 0.1227891673212466, "survival_time": 5.699999999999988, "driven_lanedir": 1.118484382839868, "get_state_dump": 0.007619470098744268, "get_robot_state": 0.003531074523925781, "sim_render-ego0": 0.0036808946858281674, "get_duckie_state": 0.020862608370573625, "in-drivable-lane": 0.0, "deviation-heading": 0.851020540755943, "agent_compute-ego0": 0.044584834057351815, "complete-iteration": 0.2517451721688975, "set_robot_commands": 0.002153046234794285, "deviation-center-line": 0.3080818291801528, "driven_lanedir_consec": 1.118484382839868, "sim_compute_sim_state": 0.01190293146216351, "sim_compute_performance-ego0": 0.0019542797752048657}, "LFP-norm-small_loop-000-ego0": {"driven_any": 14.5898100863319, "get_ui_image": 0.023748990201036897, "step_physics": 0.0963575405244724, "survival_time": 59.99999999999873, "driven_lanedir": 11.088425839240465, "get_state_dump": 0.005020013359762251, "get_robot_state": 0.0034423279424789645, "sim_render-ego0": 0.003539239437157268, "get_duckie_state": 0.004176314724772101, "in-drivable-lane": 11.099999999999756, "deviation-heading": 15.44160972654172, "agent_compute-ego0": 0.04144867155375231, "complete-iteration": 0.18756532788177413, "set_robot_commands": 0.0020648099103636984, "deviation-center-line": 3.9835052701742866, "driven_lanedir_consec": 11.088425839240465, "sim_compute_sim_state": 0.005809050019238017, "sim_compute_performance-ego0": 0.001864369167674094}}
set_robot_commands_max0.002207139822152945
set_robot_commands_mean0.0021361893035530552
set_robot_commands_median0.0021364037408477888
set_robot_commands_min0.0020648099103636984
sim_compute_performance-ego0_max0.002019302661602314
sim_compute_performance-ego0_mean0.0019360966703720688
sim_compute_performance-ego0_median0.001930357426105933
sim_compute_performance-ego0_min0.001864369167674094
sim_compute_sim_state_max0.01190293146216351
sim_compute_sim_state_mean0.00851516025378057
sim_compute_sim_state_median0.008174329766860375
sim_compute_sim_state_min0.005809050019238017
sim_render-ego0_max0.003772603548490084
sim_render-ego0_mean0.0036552158072520498
sim_render-ego0_median0.0036545101216804224
sim_render-ego0_min0.003539239437157268
simulation-passed1
step_physics_max0.16400786913358248
step_physics_mean0.12256009480335942
step_physics_median0.11493748477769136
step_physics_min0.0963575405244724
survival_time_max59.99999999999873
survival_time_mean19.81249999999968
survival_time_min3.1999999999999966
No reset possible
5512611300Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5511411300Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5507511302Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:38
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5503211309Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:03:12
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5499711301Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:30
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5497011308Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5494911424Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:51
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5491311317Marco Calì 🇮🇹template-randomaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:04:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5040495128145347
survival_time_median4.6499999999999915
deviation-center-line_median0.10620228737371736
in-drivable-lane_median2.674999999999992


other stats
agent_compute-ego0_max0.012187050448523628
agent_compute-ego0_mean0.01164764337802505
agent_compute-ego0_median0.011654904247194096
agent_compute-ego0_min0.011093714569188373
complete-iteration_max0.2034495965294216
complete-iteration_mean0.17657182238540875
complete-iteration_median0.17565283443354351
complete-iteration_min0.15153202414512634
deviation-center-line_max0.16455763165774434
deviation-center-line_mean0.1092777488823554
deviation-center-line_min0.06014878912424249
deviation-heading_max1.819139548495243
deviation-heading_mean0.860079214882594
deviation-heading_median0.6517793617099584
deviation-heading_min0.31761858761521594
driven_any_max3.078535752374254
driven_any_mean1.8526223552312917
driven_any_median1.6219947547238958
driven_any_min1.0879641591031215
driven_lanedir_consec_max0.6643282900394009
driven_lanedir_consec_mean0.4884432809313369
driven_lanedir_consec_min0.28134580805687714
driven_lanedir_max0.6643282900394009
driven_lanedir_mean0.4924956463812869
driven_lanedir_median0.5121542437144349
driven_lanedir_min0.28134580805687714
get_duckie_state_max1.6490618387858074e-06
get_duckie_state_mean1.4955041562060964e-06
get_duckie_state_median1.4987544737000396e-06
get_duckie_state_min1.3354458386384987e-06
get_robot_state_max0.003610756662156847
get_robot_state_mean0.003570002529264182
get_robot_state_median0.003565653672684794
get_robot_state_min0.003537946109530292
get_state_dump_max0.004709068271848891
get_state_dump_mean0.004528075442837078
get_state_dump_median0.0045391783778426014
get_state_dump_min0.0043248767438142195
get_ui_image_max0.034321137096570885
get_ui_image_mean0.02909646245579701
get_ui_image_median0.028423306685459764
get_ui_image_min0.025218099355697632
in-drivable-lane_max6.549999999999982
in-drivable-lane_mean3.3499999999999894
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.078535752374254, "get_ui_image": 0.026702696764016452, "step_physics": 0.09845997713789156, "survival_time": 7.84999999999998, "driven_lanedir": 0.28134580805687714, "get_state_dump": 0.004568587375592582, "get_robot_state": 0.003537946109530292, "sim_render-ego0": 0.0036880653115767465, "get_duckie_state": 1.3354458386384987e-06, "in-drivable-lane": 6.549999999999982, "deviation-heading": 0.891006057417327, "agent_compute-ego0": 0.011093714569188373, "complete-iteration": 0.1614939623241183, "set_robot_commands": 0.0021047712881353837, "deviation-center-line": 0.13094886067496936, "driven_lanedir_consec": 0.28134580805687714, "sim_compute_sim_state": 0.009327429759351514, "sim_compute_performance-ego0": 0.0019289209872861452}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5806581102909054, "get_ui_image": 0.034321137096570885, "step_physics": 0.13235541789428049, "survival_time": 4.549999999999992, "driven_lanedir": 0.3891269085945981, "get_state_dump": 0.0043248767438142195, "get_robot_state": 0.003571188968160878, "sim_render-ego0": 0.003630648488583772, "get_duckie_state": 1.368315323539402e-06, "in-drivable-lane": 2.449999999999993, "deviation-heading": 1.819139548495243, "agent_compute-ego0": 0.012094414752462637, "complete-iteration": 0.2034495965294216, "set_robot_commands": 0.0020547975664553437, "deviation-center-line": 0.16455763165774434, "driven_lanedir_consec": 0.37291744679479777, "sim_compute_sim_state": 0.009071023567863132, "sim_compute_performance-ego0": 0.001941271450208581}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0879641591031215, "get_ui_image": 0.030143916606903076, "step_physics": 0.12391946381992763, "survival_time": 3.5499999999999954, "driven_lanedir": 0.6643282900394009, "get_state_dump": 0.004709068271848891, "get_robot_state": 0.003610756662156847, "sim_render-ego0": 0.00372085968653361, "get_duckie_state": 1.629193623860677e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.41255266600258983, "agent_compute-ego0": 0.012187050448523628, "complete-iteration": 0.18981170654296875, "set_robot_commands": 0.0020691388183169896, "deviation-center-line": 0.08145571407246535, "driven_lanedir_consec": 0.6643282900394009, "sim_compute_sim_state": 0.007364971770180596, "sim_compute_performance-ego0": 0.0019917256302303737}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6633313991568863, "get_ui_image": 0.025218099355697632, "step_physics": 0.09430105984210968, "survival_time": 4.749999999999991, "driven_lanedir": 0.6351815788342716, "get_state_dump": 0.004509769380092621, "get_robot_state": 0.0035601183772087097, "sim_render-ego0": 0.003687781592210134, "get_duckie_state": 1.6490618387858074e-06, "in-drivable-lane": 2.89999999999999, "deviation-heading": 0.31761858761521594, "agent_compute-ego0": 0.011215393741925558, "complete-iteration": 0.15153202414512634, "set_robot_commands": 0.0021951297918955484, "deviation-center-line": 0.06014878912424249, "driven_lanedir_consec": 0.6351815788342716, "sim_compute_sim_state": 0.0048591941595077515, "sim_compute_performance-ego0": 0.001899801194667816}}
set_robot_commands_max0.0021951297918955484
set_robot_commands_mean0.0021059593662008163
set_robot_commands_median0.002086955053226186
set_robot_commands_min0.0020547975664553437
sim_compute_performance-ego0_max0.0019917256302303737
sim_compute_performance-ego0_mean0.001940429815598229
sim_compute_performance-ego0_median0.001935096218747363
sim_compute_performance-ego0_min0.001899801194667816
sim_compute_sim_state_max0.009327429759351514
sim_compute_sim_state_mean0.007655654814225748
sim_compute_sim_state_median0.008217997669021863
sim_compute_sim_state_min0.0048591941595077515
sim_render-ego0_max0.00372085968653361
sim_render-ego0_mean0.0036818387697260656
sim_render-ego0_median0.00368792345189344
sim_render-ego0_min0.003630648488583772
simulation-passed1
step_physics_max0.13235541789428049
step_physics_mean0.11225897967355232
step_physics_median0.1111897204789096
step_physics_min0.09430105984210968
survival_time_max7.84999999999998
survival_time_mean5.17499999999999
survival_time_min3.5499999999999954
No reset possible
5470111379Martin Cote 🇨🇦baseline-duckietownaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:34:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.8836249837629016
survival_time_median59.99999999999873
deviation-center-line_median3.2482313536699543
in-drivable-lane_median12.899999999999984


other stats
agent_compute-ego0_max0.011890809204457304
agent_compute-ego0_mean0.011544140600145709
agent_compute-ego0_median0.01158185098490846
agent_compute-ego0_min0.0111220512263086
complete-iteration_max0.20840821020013583
complete-iteration_mean0.18193558893235495
complete-iteration_median0.17877509214796689
complete-iteration_min0.16178396123335026
deviation-center-line_max3.8423620846880935
deviation-center-line_mean3.037460525655949
deviation-center-line_min1.8110173105957923
deviation-heading_max11.029195437229404
deviation-heading_mean9.22932832900694
deviation-heading_median9.20247517446823
deviation-heading_min7.483167529861897
driven_any_max7.921098126258872
driven_any_mean7.9192714082021585
driven_any_median7.920827088695004
driven_any_min7.914333329159752
driven_lanedir_consec_max7.117461460961419
driven_lanedir_consec_mean5.761609919387806
driven_lanedir_consec_min4.161728249064002
driven_lanedir_max7.117461460961419
driven_lanedir_mean5.832697498167486
driven_lanedir_median5.914799771519033
driven_lanedir_min4.383728988670459
get_duckie_state_max1.3252976137235895e-06
get_duckie_state_mean1.2765121301147564e-06
get_duckie_state_median1.2814254189014038e-06
get_duckie_state_min1.2179000689326278e-06
get_robot_state_max0.0037759628819982577
get_robot_state_mean0.0036907009935498135
get_robot_state_median0.003696656743255285
get_robot_state_min0.003593527605690428
get_state_dump_max0.004705451311020926
get_state_dump_mean0.00463980307289206
get_state_dump_median0.004676934979936662
get_state_dump_min0.004499891020673994
get_ui_image_max0.03581950845170478
get_ui_image_mean0.030221526469914346
get_ui_image_median0.02975326960132481
get_ui_image_min0.02556005822530297
in-drivable-lane_max24.349999999999657
in-drivable-lane_mean13.562499999999892
in-drivable-lane_min4.099999999999957
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.921098126258872, "get_ui_image": 0.027054077977443317, "step_physics": 0.09877089219327571, "survival_time": 59.99999999999873, "driven_lanedir": 5.3181043338852305, "get_state_dump": 0.004499891020673994, "get_robot_state": 0.003593527605690428, "sim_render-ego0": 0.003701002572001665, "get_duckie_state": 1.2179000689326278e-06, "in-drivable-lane": 17.600000000000048, "deviation-heading": 7.7148696659721505, "agent_compute-ego0": 0.0111220512263086, "complete-iteration": 0.16178396123335026, "set_robot_commands": 0.0021083605081016676, "deviation-center-line": 2.9794739442779785, "driven_lanedir_consec": 5.3181043338852305, "sim_compute_sim_state": 0.008918521406251524, "sim_compute_performance-ego0": 0.0019295809171678223}, "LF-norm-zigzag-000-ego0": {"driven_any": 7.92104118176319, "get_ui_image": 0.03581950845170478, "step_physics": 0.13113056213829935, "survival_time": 59.99999999999873, "driven_lanedir": 7.117461460961419, "get_state_dump": 0.004667306522048582, "get_robot_state": 0.0037759628819982577, "sim_render-ego0": 0.003904433969057768, "get_duckie_state": 1.259985613286942e-06, "in-drivable-lane": 4.099999999999957, "deviation-heading": 11.029195437229404, "agent_compute-ego0": 0.011890809204457304, "complete-iteration": 0.20840821020013583, "set_robot_commands": 0.0022645870116628475, "deviation-center-line": 3.8423620846880935, "driven_lanedir_consec": 7.117461460961419, "sim_compute_sim_state": 0.012746803965000783, "sim_compute_performance-ego0": 0.00211753634787122}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.914333329159752, "get_ui_image": 0.03245246122520631, "step_physics": 0.11926026249011291, "survival_time": 59.99999999999873, "driven_lanedir": 6.5114952091528355, "get_state_dump": 0.004705451311020926, "get_robot_state": 0.003760071023119975, "sim_render-ego0": 0.003870585677427218, "get_duckie_state": 1.3252976137235895e-06, "in-drivable-lane": 8.199999999999918, "deviation-heading": 10.69008068296431, "agent_compute-ego0": 0.01187996443463404, "complete-iteration": 0.19365082017388768, "set_robot_commands": 0.002248297722313823, "deviation-center-line": 3.51698876306193, "driven_lanedir_consec": 6.449145633640573, "sim_compute_sim_state": 0.0132978063737423, "sim_compute_performance-ego0": 0.002082154713105798}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.920612995626819, "get_ui_image": 0.02556005822530297, "step_physics": 0.1046094693907294, "survival_time": 59.99999999999873, "driven_lanedir": 4.383728988670459, "get_state_dump": 0.004686563437824741, "get_robot_state": 0.003633242463390595, "sim_render-ego0": 0.0037120869515043418, "get_duckie_state": 1.3028652245158656e-06, "in-drivable-lane": 24.349999999999657, "deviation-heading": 7.483167529861897, "agent_compute-ego0": 0.01128373753518288, "complete-iteration": 0.16389936412204612, "set_robot_commands": 0.0021685555415983304, "deviation-center-line": 1.8110173105957923, "driven_lanedir_consec": 4.161728249064002, "sim_compute_sim_state": 0.006232170141507545, "sim_compute_performance-ego0": 0.001925917886675248}}
set_robot_commands_max0.0022645870116628475
set_robot_commands_mean0.0021974501959191673
set_robot_commands_median0.0022084266319560768
set_robot_commands_min0.0021083605081016676
sim_compute_performance-ego0_max0.00211753634787122
sim_compute_performance-ego0_mean0.002013797466205022
sim_compute_performance-ego0_median0.00200586781513681
sim_compute_performance-ego0_min0.001925917886675248
sim_compute_sim_state_max0.0132978063737423
sim_compute_sim_state_mean0.010298825471625538
sim_compute_sim_state_median0.010832662685626156
sim_compute_sim_state_min0.006232170141507545
sim_render-ego0_max0.003904433969057768
sim_render-ego0_mean0.003797027292497748
sim_render-ego0_median0.003791336314465779
sim_render-ego0_min0.003701002572001665
simulation-passed1
step_physics_max0.13113056213829935
step_physics_mean0.11344279655310434
step_physics_median0.11193486594042117
step_physics_min0.09877089219327571
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5451711469Raphael Jeanexercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:40:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median12.73624070798918
survival_time_median59.99999999999873
deviation-center-line_median2.4947449335311997
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.05272075179018248
agent_compute-ego0_mean0.04841336858560402
agent_compute-ego0_median0.05150298591855166
agent_compute-ego0_min0.03792675071513028
complete-iteration_max0.2681422993900575
complete-iteration_mean0.23169136692542616
complete-iteration_median0.22812995068933645
complete-iteration_min0.20236326693297424
deviation-center-line_max2.66822822467615
deviation-center-line_mean2.4785574657550353
deviation-center-line_min2.256511771281591
deviation-heading_max10.869243163565503
deviation-heading_mean9.852478174402304
deviation-heading_median9.755908637635748
deviation-heading_min9.02885225877222
driven_any_max16.177196606955164
driven_any_mean13.66263698850903
driven_any_median12.974560310100175
driven_any_min12.524230726880608
driven_lanedir_consec_max15.92022363219598
driven_lanedir_consec_mean13.402807830400398
driven_lanedir_consec_min12.218526273427251
driven_lanedir_max15.92022363219598
driven_lanedir_mean13.402807830400398
driven_lanedir_median12.73624070798918
driven_lanedir_min12.218526273427251
get_duckie_state_max1.4154242039917907e-06
get_duckie_state_mean1.3335856867273283e-06
get_duckie_state_median1.342171534809046e-06
get_duckie_state_min1.2345754732994315e-06
get_robot_state_max0.003942976387017573
get_robot_state_mean0.003800460341371763
get_robot_state_median0.0038477994321684953
get_robot_state_min0.0035632661141324897
get_state_dump_max0.00517588650356423
get_state_dump_mean0.004916833501175778
get_state_dump_median0.0048787545204956664
get_state_dump_min0.0047339384601475494
get_ui_image_max0.036697607651836765
get_ui_image_mean0.0310325287561631
get_ui_image_median0.03112428363018687
get_ui_image_min0.0251839401124419
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 16.177196606955164, "get_ui_image": 0.029397303615382668, "step_physics": 0.11077523231506348, "survival_time": 59.99999999999873, "driven_lanedir": 15.92022363219598, "get_state_dump": 0.004851607061444869, "get_robot_state": 0.003942976387017573, "sim_render-ego0": 0.004020274826926454, "get_duckie_state": 1.4154242039917907e-06, "in-drivable-lane": 0.0, "deviation-heading": 9.02885225877222, "agent_compute-ego0": 0.052715156000917106, "complete-iteration": 0.2204480486845196, "set_robot_commands": 0.0024514770031372375, "deviation-center-line": 2.500678509668368, "driven_lanedir_consec": 15.92022363219598, "sim_compute_sim_state": 0.009987573242505128, "sim_compute_performance-ego0": 0.002206628467518523}, "LF-norm-zigzag-000-ego0": {"driven_any": 12.524230726880608, "get_ui_image": 0.036697607651836765, "step_physics": 0.14767638114370177, "survival_time": 59.99999999999873, "driven_lanedir": 12.218526273427251, "get_state_dump": 0.0047339384601475494, "get_robot_state": 0.003785846830903243, "sim_render-ego0": 0.003899562964332193, "get_duckie_state": 1.3572588054266302e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.869243163565503, "agent_compute-ego0": 0.05272075179018248, "complete-iteration": 0.2681422993900575, "set_robot_commands": 0.002344395695479089, "deviation-center-line": 2.66822822467615, "driven_lanedir_consec": 12.218526273427251, "sim_compute_sim_state": 0.014102407736544007, "sim_compute_performance-ego0": 0.0020855185789033634}, "LF-norm-techtrack-000-ego0": {"driven_any": 13.396824452642488, "get_ui_image": 0.03285126364499107, "step_physics": 0.13474823672209651, "survival_time": 59.99999999999873, "driven_lanedir": 13.132283787537336, "get_state_dump": 0.004905901979546464, "get_robot_state": 0.003909752033433748, "sim_render-ego0": 0.00396321933533528, "get_duckie_state": 1.3270842641914617e-06, "in-drivable-lane": 0.0, "deviation-heading": 9.9261293643868, "agent_compute-ego0": 0.03792675071513028, "complete-iteration": 0.23581185269415328, "set_robot_commands": 0.002366876522766164, "deviation-center-line": 2.256511771281591, "driven_lanedir_consec": 13.132283787537336, "sim_compute_sim_state": 0.012882840722724855, "sim_compute_performance-ego0": 0.002159530971568391}, "LF-norm-small_loop-000-ego0": {"driven_any": 12.552296167557865, "get_ui_image": 0.0251839401124419, "step_physics": 0.10430541284673917, "survival_time": 59.99999999999873, "driven_lanedir": 12.340197628441022, "get_state_dump": 0.00517588650356423, "get_robot_state": 0.0035632661141324897, "sim_render-ego0": 0.003644478708183041, "get_duckie_state": 1.2345754732994315e-06, "in-drivable-lane": 0.0, "deviation-heading": 9.585687910884694, "agent_compute-ego0": 0.05029081583618621, "complete-iteration": 0.20236326693297424, "set_robot_commands": 0.00224994660217895, "deviation-center-line": 2.488811357394031, "driven_lanedir_consec": 12.340197628441022, "sim_compute_sim_state": 0.005913464651814508, "sim_compute_performance-ego0": 0.0019495739329367457}}
set_robot_commands_max0.0024514770031372375
set_robot_commands_mean0.00235317395589036
set_robot_commands_median0.0023556361091226262
set_robot_commands_min0.00224994660217895
sim_compute_performance-ego0_max0.002206628467518523
sim_compute_performance-ego0_mean0.002100312987731756
sim_compute_performance-ego0_median0.002122524775235877
sim_compute_performance-ego0_min0.0019495739329367457
sim_compute_sim_state_max0.014102407736544007
sim_compute_sim_state_mean0.010721571588397124
sim_compute_sim_state_median0.011435206982614993
sim_compute_sim_state_min0.005913464651814508
sim_render-ego0_max0.004020274826926454
sim_render-ego0_mean0.003881883958694242
sim_render-ego0_median0.003931391149833737
sim_render-ego0_min0.003644478708183041
simulation-passed1
step_physics_max0.14767638114370177
step_physics_mean0.12437631575690024
step_physics_median0.12276173451858
step_physics_min0.10430541284673917
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5445211538Jean-Sébastien Grondin 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:28:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.209762307790865
survival_time_median45.74999999999952
deviation-center-line_median2.5754601481394293
in-drivable-lane_median13.099999999999914


other stats
agent_compute-ego0_max0.013127874459026356
agent_compute-ego0_mean0.012730784688425738
agent_compute-ego0_median0.012631365270526806
agent_compute-ego0_min0.012532533753622978
complete-iteration_max0.2767875386090914
complete-iteration_mean0.24327010541380173
complete-iteration_median0.2477947700294386
complete-iteration_min0.2007033429872384
deviation-center-line_max4.011912176095072
deviation-center-line_mean2.5285345396103587
deviation-center-line_min0.9513056860675054
deviation-heading_max15.419246979730213
deviation-heading_mean10.384772309644314
deviation-heading_median10.3408910552243
deviation-heading_min5.438060148398446
driven_any_max18.758848611934972
driven_any_mean13.53899194201026
driven_any_median14.152208762708677
driven_any_min7.092701630688716
driven_lanedir_consec_max13.872040713581123
driven_lanedir_consec_mean5.992213123560636
driven_lanedir_consec_min1.6772871650796892
driven_lanedir_max13.872040713581123
driven_lanedir_mean8.673215393710326
driven_lanedir_median9.294738903222733
driven_lanedir_min2.2313430548147175
get_duckie_state_max1.4014176067950795e-06
get_duckie_state_mean1.3650369348388743e-06
get_duckie_state_median1.36026446606221e-06
get_duckie_state_min1.3382012004359974e-06
get_robot_state_max0.0039355456356768
get_robot_state_mean0.003770440909405768
get_robot_state_median0.003747852219828559
get_robot_state_min0.003650513562289151
get_state_dump_max0.004964984540138305
get_state_dump_mean0.004761530684179246
get_state_dump_median0.004708389556973701
get_state_dump_min0.004664359082631278
get_ui_image_max0.035188443060909215
get_ui_image_mean0.031529726976146734
get_ui_image_median0.03202712554828928
get_ui_image_min0.026876213747099177
in-drivable-lane_max17.549999999999574
in-drivable-lane_mean13.22499999999988
in-drivable-lane_min9.150000000000116
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 18.758848611934972, "get_ui_image": 0.029529423042697577, "step_physics": 0.161746488820504, "survival_time": 59.99999999999873, "driven_lanedir": 13.872040713581123, "get_state_dump": 0.004726305492315364, "get_robot_state": 0.003742266654174195, "sim_render-ego0": 0.0038290218350095218, "get_duckie_state": 1.3701623921390377e-06, "in-drivable-lane": 12.1499999999997, "deviation-heading": 15.419246979730213, "agent_compute-ego0": 0.01256802139631616, "complete-iteration": 0.23014355122695657, "set_robot_commands": 0.0022466679000536865, "deviation-center-line": 4.011912176095072, "driven_lanedir_consec": 13.872040713581123, "sim_compute_sim_state": 0.009627567540596764, "sim_compute_performance-ego0": 0.002043844956740253}, "LF-norm-zigzag-000-ego0": {"driven_any": 7.092701630688716, "get_ui_image": 0.035188443060909215, "step_physics": 0.20134279390218143, "survival_time": 23.6000000000002, "driven_lanedir": 2.2313430548147175, "get_state_dump": 0.004664359082631278, "get_robot_state": 0.003650513562289151, "sim_render-ego0": 0.003662182463147927, "get_duckie_state": 1.3503665399853823e-06, "in-drivable-lane": 14.050000000000129, "deviation-heading": 5.438060148398446, "agent_compute-ego0": 0.012694709144737451, "complete-iteration": 0.2767875386090914, "set_robot_commands": 0.002149601567371199, "deviation-center-line": 0.9513056860675054, "driven_lanedir_consec": 1.6772871650796892, "sim_compute_sim_state": 0.011411262112994529, "sim_compute_performance-ego0": 0.0019416128606171237}, "LF-norm-techtrack-000-ego0": {"driven_any": 9.5458851052278, "get_ui_image": 0.03452482805388098, "step_physics": 0.18891955631092872, "survival_time": 31.500000000000313, "driven_lanedir": 6.279910969109235, "get_state_dump": 0.004964984540138305, "get_robot_state": 0.0039355456356768, "sim_render-ego0": 0.004014390395295979, "get_duckie_state": 1.4014176067950795e-06, "in-drivable-lane": 9.150000000000116, "deviation-heading": 7.672358384616172, "agent_compute-ego0": 0.013127874459026356, "complete-iteration": 0.2654459888319206, "set_robot_commands": 0.0023908150366104535, "deviation-center-line": 1.5906078993462869, "driven_lanedir_consec": 4.994549815912473, "sim_compute_sim_state": 0.011309404569647014, "sim_compute_performance-ego0": 0.002165432021659452}, "LF-norm-small_loop-000-ego0": {"driven_any": 18.75853242018955, "get_ui_image": 0.026876213747099177, "step_physics": 0.13849165735395624, "survival_time": 59.99999999999873, "driven_lanedir": 12.309566837336227, "get_state_dump": 0.004690473621632038, "get_robot_state": 0.0037534377854829223, "sim_render-ego0": 0.0038061487386863895, "get_duckie_state": 1.3382012004359974e-06, "in-drivable-lane": 17.549999999999574, "deviation-heading": 13.009423725832429, "agent_compute-ego0": 0.012532533753622978, "complete-iteration": 0.2007033429872384, "set_robot_commands": 0.002220856954811217, "deviation-center-line": 3.560312396932572, "driven_lanedir_consec": 3.4249747996692577, "sim_compute_sim_state": 0.006257949125558312, "sim_compute_performance-ego0": 0.0019906206393023513}}
set_robot_commands_max0.0023908150366104535
set_robot_commands_mean0.002251985364711639
set_robot_commands_median0.0022337624274324517
set_robot_commands_min0.002149601567371199
sim_compute_performance-ego0_max0.002165432021659452
sim_compute_performance-ego0_mean0.0020353776195797947
sim_compute_performance-ego0_median0.002017232798021302
sim_compute_performance-ego0_min0.0019416128606171237
sim_compute_sim_state_max0.011411262112994529
sim_compute_sim_state_mean0.009651545837199156
sim_compute_sim_state_median0.010468486055121889
sim_compute_sim_state_min0.006257949125558312
sim_render-ego0_max0.004014390395295979
sim_render-ego0_mean0.003827935858034955
sim_render-ego0_median0.003817585286847955
sim_render-ego0_min0.003662182463147927
simulation-passed1
step_physics_max0.20134279390218143
step_physics_mean0.17262512409689262
step_physics_median0.17533302256571637
step_physics_min0.13849165735395624
survival_time_max59.99999999999873
survival_time_mean43.77499999999949
survival_time_min23.6000000000002
No reset possible
5432311582Yishu Malhotra 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:35:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.645046954102662
survival_time_median59.99999999999873
deviation-center-line_median3.650824994223763
in-drivable-lane_median3.1249999999999076


other stats
agent_compute-ego0_max0.012410909314698812
agent_compute-ego0_mean0.012010613896532938
agent_compute-ego0_median0.011941537868966668
agent_compute-ego0_min0.011748470533499611
complete-iteration_max0.24748977594431196
complete-iteration_mean0.2199549552543812
complete-iteration_median0.21984273242734836
complete-iteration_min0.192644580218516
deviation-center-line_max3.970491025665531
deviation-center-line_mean3.4770130904542915
deviation-center-line_min2.6359113477041096
deviation-heading_max19.812015688771595
deviation-heading_mean16.94307739070128
deviation-heading_median18.201643518627563
deviation-heading_min11.557006836778406
driven_any_max12.324089678852692
driven_any_mean10.720880342975285
driven_any_median11.50449055617088
driven_any_min7.550450580706681
driven_lanedir_consec_max11.091728200345058
driven_lanedir_consec_mean7.876570529485618
driven_lanedir_consec_min3.1244600093920925
driven_lanedir_max11.091728200345058
driven_lanedir_mean9.256231162960107
driven_lanedir_median9.54700438497158
driven_lanedir_min6.839187681552211
get_duckie_state_max1.5577606912655795e-06
get_duckie_state_mean1.4507348627481198e-06
get_duckie_state_median1.4348282080921227e-06
get_duckie_state_min1.3755223435426532e-06
get_robot_state_max0.003935851326471643
get_robot_state_mean0.003878155046031061
get_robot_state_median0.003884766619965794
get_robot_state_min0.003807235617721011
get_state_dump_max0.004990926491628579
get_state_dump_mean0.004884690842952991
get_state_dump_median0.004899746030881615
get_state_dump_min0.004748344818420156
get_ui_image_max0.03651406862257323
get_ui_image_mean0.03142582068121249
get_ui_image_median0.03105160225606073
get_ui_image_min0.02708600959015528
in-drivable-lane_max9.15000000000001
in-drivable-lane_mean4.424999999999923
in-drivable-lane_min2.2999999999998693
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.065845399718816, "get_ui_image": 0.02903183354227668, "step_physics": 0.13437146270999703, "survival_time": 59.99999999999873, "driven_lanedir": 10.450906226653114, "get_state_dump": 0.004878971400010795, "get_robot_state": 0.003807235617721011, "sim_render-ego0": 0.003812481024977965, "get_duckie_state": 1.3755223435426532e-06, "in-drivable-lane": 3.749999999999901, "deviation-heading": 18.730954604510696, "agent_compute-ego0": 0.012040676820486613, "complete-iteration": 0.20207998575914116, "set_robot_commands": 0.0022490735256503165, "deviation-center-line": 3.560466546041781, "driven_lanedir_consec": 10.450906226653114, "sim_compute_sim_state": 0.00975001563041236, "sim_compute_performance-ego0": 0.0020537267219613337}, "LF-norm-zigzag-000-ego0": {"driven_any": 12.324089678852692, "get_ui_image": 0.03651406862257323, "step_physics": 0.16781614265473657, "survival_time": 59.99999999999873, "driven_lanedir": 11.091728200345058, "get_state_dump": 0.004748344818420156, "get_robot_state": 0.0038681820369977737, "sim_render-ego0": 0.003892493188430824, "get_duckie_state": 1.379691194634354e-06, "in-drivable-lane": 2.4999999999999147, "deviation-heading": 17.672332432744426, "agent_compute-ego0": 0.011748470533499611, "complete-iteration": 0.24748977594431196, "set_robot_commands": 0.002298233808823966, "deviation-center-line": 3.970491025665531, "driven_lanedir_consec": 11.091728200345058, "sim_compute_sim_state": 0.014385638685647297, "sim_compute_performance-ego0": 0.0021294906276350316}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.550450580706681, "get_ui_image": 0.03307137096984477, "step_physics": 0.1610446338412128, "survival_time": 39.449999999999896, "driven_lanedir": 6.839187681552211, "get_state_dump": 0.004920520661752435, "get_robot_state": 0.003935851326471643, "sim_render-ego0": 0.00393407616434218, "get_duckie_state": 1.4899652215498911e-06, "in-drivable-lane": 2.2999999999998693, "deviation-heading": 11.557006836778406, "agent_compute-ego0": 0.012410909314698812, "complete-iteration": 0.23760547909555557, "set_robot_commands": 0.002297292479985877, "deviation-center-line": 2.6359113477041096, "driven_lanedir_consec": 6.839187681552211, "sim_compute_sim_state": 0.013794419433497176, "sim_compute_performance-ego0": 0.00210534017297286}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.94313571262294, "get_ui_image": 0.02708600959015528, "step_physics": 0.1300267393047069, "survival_time": 59.99999999999873, "driven_lanedir": 8.643102543290047, "get_state_dump": 0.004990926491628579, "get_robot_state": 0.0039013512029338142, "sim_render-ego0": 0.0039362417073372895, "get_duckie_state": 1.5577606912655795e-06, "in-drivable-lane": 9.15000000000001, "deviation-heading": 19.812015688771595, "agent_compute-ego0": 0.011842398917446723, "complete-iteration": 0.192644580218516, "set_robot_commands": 0.002282481507198896, "deviation-center-line": 3.741183442405744, "driven_lanedir_consec": 3.1244600093920925, "sim_compute_sim_state": 0.00644567904126932, "sim_compute_performance-ego0": 0.002040135274818795}}
set_robot_commands_max0.002298233808823966
set_robot_commands_mean0.002281770330414764
set_robot_commands_median0.0022898869935923864
set_robot_commands_min0.0022490735256503165
sim_compute_performance-ego0_max0.0021294906276350316
sim_compute_performance-ego0_mean0.002082173199347005
sim_compute_performance-ego0_median0.002079533447467097
sim_compute_performance-ego0_min0.002040135274818795
sim_compute_sim_state_max0.014385638685647297
sim_compute_sim_state_mean0.011093938197706536
sim_compute_sim_state_median0.011772217531954768
sim_compute_sim_state_min0.00644567904126932
sim_render-ego0_max0.0039362417073372895
sim_render-ego0_mean0.0038938230212720647
sim_render-ego0_median0.003913284676386502
sim_render-ego0_min0.003812481024977965
simulation-passed1
step_physics_max0.16781614265473657
step_physics_mean0.1483147446276633
step_physics_median0.1477080482756049
step_physics_min0.1300267393047069
survival_time_max59.99999999999873
survival_time_mean54.86249999999902
survival_time_min39.449999999999896
No reset possible
5423011627Philippe Reddy 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:29:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.677018049437939
survival_time_median59.62499999999875
deviation-center-line_median3.698630828992956
in-drivable-lane_median14.799999999999466


other stats
agent_compute-ego0_max0.01305208403683273
agent_compute-ego0_mean0.012764339766688945
agent_compute-ego0_median0.012769015504359008
agent_compute-ego0_min0.012467244021205031
complete-iteration_max0.28088525766451683
complete-iteration_mean0.23650489365476357
complete-iteration_median0.23135516475715612
complete-iteration_min0.2024239874402252
deviation-center-line_max4.457789102093991
deviation-center-line_mean2.9880620185706297
deviation-center-line_min0.09719731420261568
deviation-heading_max18.63876368046052
deviation-heading_mean13.155539310712852
deviation-heading_median16.181864135548707
deviation-heading_min1.619665291293474
driven_any_max7.919724333692071
driven_any_mean6.134565350159404
driven_any_median7.820725793235454
driven_any_min0.9770854804746402
driven_lanedir_consec_max4.329375438305592
driven_lanedir_consec_mean3.040575482151148
driven_lanedir_consec_min0.47889039142312306
driven_lanedir_max5.825509141685673
driven_lanedir_mean3.4899426496264825
driven_lanedir_median3.8276855326985664
driven_lanedir_min0.47889039142312306
get_duckie_state_max1.4700127776557875e-06
get_duckie_state_mean1.4010214874274396e-06
get_duckie_state_median1.3871168246424297e-06
get_duckie_state_min1.3598395227691115e-06
get_robot_state_max0.0039138723407271345
get_robot_state_mean0.003807698877967746
get_robot_state_median0.003815279614419167
get_robot_state_min0.003686363942305515
get_state_dump_max0.0050642998260859206
get_state_dump_mean0.004860973437573662
get_state_dump_median0.00485841003087637
get_state_dump_min0.004662773862455989
get_ui_image_max0.03763791936389088
get_ui_image_mean0.03129352192911552
get_ui_image_median0.03071207563248812
get_ui_image_min0.026112017087594953
in-drivable-lane_max30.099999999999387
in-drivable-lane_mean15.924999999999578
in-drivable-lane_min3.999999999999992
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.727755860536705, "get_ui_image": 0.0285759382264007, "step_physics": 0.15019545044504973, "survival_time": 59.24999999999877, "driven_lanedir": 5.825509141685673, "get_state_dump": 0.004662773862455989, "get_robot_state": 0.003686363942305515, "sim_render-ego0": 0.003675474485297629, "get_duckie_state": 1.4116148924546138e-06, "in-drivable-lane": 9.099999999999532, "deviation-heading": 18.63876368046052, "agent_compute-ego0": 0.012467244021205031, "complete-iteration": 0.2161274893086837, "set_robot_commands": 0.0021761261793654444, "deviation-center-line": 4.457789102093991, "driven_lanedir_consec": 4.028040471784337, "sim_compute_sim_state": 0.008667459375339656, "sim_compute_performance-ego0": 0.0019322348284279117}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.9770854804746402, "get_ui_image": 0.03763791936389088, "step_physics": 0.20172861059741864, "survival_time": 8.399999999999984, "driven_lanedir": 0.47889039142312306, "get_state_dump": 0.0050642998260859206, "get_robot_state": 0.0039138723407271345, "sim_render-ego0": 0.003987993714372082, "get_duckie_state": 1.4700127776557875e-06, "in-drivable-lane": 3.999999999999992, "deviation-heading": 1.619665291293474, "agent_compute-ego0": 0.01305208403683273, "complete-iteration": 0.28088525766451683, "set_robot_commands": 0.002297905069836498, "deviation-center-line": 0.09719731420261568, "driven_lanedir_consec": 0.47889039142312306, "sim_compute_sim_state": 0.010918439492671447, "sim_compute_performance-ego0": 0.00218117025476941}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.913695725934202, "get_ui_image": 0.03284821303857554, "step_physics": 0.17029746485987274, "survival_time": 59.99999999999873, "driven_lanedir": 4.329375438305592, "get_state_dump": 0.004853204922513303, "get_robot_state": 0.0038563266185598502, "sim_render-ego0": 0.003876544355254288, "get_duckie_state": 1.3626187568302455e-06, "in-drivable-lane": 20.4999999999994, "deviation-heading": 18.136317110173408, "agent_compute-ego0": 0.012817029651257516, "complete-iteration": 0.2465828402056285, "set_robot_commands": 0.002316957111660388, "deviation-center-line": 3.671788064512126, "driven_lanedir_consec": 4.329375438305592, "sim_compute_sim_state": 0.01347599458337128, "sim_compute_performance-ego0": 0.002148021766287798}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.919724333692071, "get_ui_image": 0.026112017087594953, "step_physics": 0.1405803516048079, "survival_time": 59.99999999999873, "driven_lanedir": 3.325995627091541, "get_state_dump": 0.004863615139239436, "get_robot_state": 0.003774232610278483, "sim_render-ego0": 0.00380253057297223, "get_duckie_state": 1.3598395227691115e-06, "in-drivable-lane": 30.099999999999387, "deviation-heading": 14.227411160924008, "agent_compute-ego0": 0.012721001357460498, "complete-iteration": 0.2024239874402252, "set_robot_commands": 0.002214967162285518, "deviation-center-line": 3.7254735934737857, "driven_lanedir_consec": 3.325995627091541, "sim_compute_sim_state": 0.006298866399023356, "sim_compute_performance-ego0": 0.0019682984268734795}}
set_robot_commands_max0.002316957111660388
set_robot_commands_mean0.002251488880786962
set_robot_commands_median0.0022564361160610077
set_robot_commands_min0.0021761261793654444
sim_compute_performance-ego0_max0.00218117025476941
sim_compute_performance-ego0_mean0.00205743131908965
sim_compute_performance-ego0_median0.0020581600965806387
sim_compute_performance-ego0_min0.0019322348284279117
sim_compute_sim_state_max0.01347599458337128
sim_compute_sim_state_mean0.009840189962601435
sim_compute_sim_state_median0.009792949434005552
sim_compute_sim_state_min0.006298866399023356
sim_render-ego0_max0.003987993714372082
sim_render-ego0_mean0.003835635781974057
sim_render-ego0_median0.003839537464113259
sim_render-ego0_min0.003675474485297629
simulation-passed1
step_physics_max0.20172861059741864
step_physics_mean0.16570046937678723
step_physics_median0.16024645765246123
step_physics_min0.1405803516048079
survival_time_max59.99999999999873
survival_time_mean46.91249999999905
survival_time_min8.399999999999984
No reset possible
5418811638Dishank Bansal 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-080:18:05
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 1939 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5408411671Daniil Lisussim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:36:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.237776849489874
survival_time_median59.99999999999873
deviation-center-line_median3.312926316720614
in-drivable-lane_median15.399999999999448


other stats
agent_compute-ego0_max0.012497551137462048
agent_compute-ego0_mean0.012190802421659062
agent_compute-ego0_median0.012251656642587268
agent_compute-ego0_min0.011762345263999666
complete-iteration_max0.2532162009230462
complete-iteration_mean0.2163356376103125
complete-iteration_median0.21402376126327977
complete-iteration_min0.18407882699164424
deviation-center-line_max5.02368170869833
deviation-center-line_mean3.404512436926091
deviation-center-line_min1.968515405564806
deviation-heading_max17.12501799157715
deviation-heading_mean12.122636397174134
deviation-heading_median12.111943197440128
deviation-heading_min7.141641202239131
driven_any_max16.634148444782042
driven_any_mean14.908550648181809
driven_any_median15.080524970911178
driven_any_min12.839004206122835
driven_lanedir_consec_max11.625876477658874
driven_lanedir_consec_mean6.782660257428883
driven_lanedir_consec_min3.029210853076907
driven_lanedir_max13.083261699572546
driven_lanedir_mean9.807718483634666
driven_lanedir_median10.893525410343852
driven_lanedir_min4.360561414278415
get_duckie_state_max1.6115587220203867e-06
get_duckie_state_mean1.4274324803030758e-06
get_duckie_state_median1.4254453279493653e-06
get_duckie_state_min1.2472805432931866e-06
get_robot_state_max0.003998904303646802
get_robot_state_mean0.003875954986247227
get_robot_state_median0.003897510157205845
get_robot_state_min0.0037098953269304184
get_state_dump_max0.005011944449215904
get_state_dump_mean0.004865131815950458
get_state_dump_median0.0048499433107868795
get_state_dump_min0.004748696193012171
get_ui_image_max0.037505260613637605
get_ui_image_mean0.032024962676799884
get_ui_image_median0.032168812614888014
get_ui_image_min0.026256964863785898
in-drivable-lane_max39.199999999998745
in-drivable-lane_mean19.862499999999393
in-drivable-lane_min9.449999999999935
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 16.634148444782042, "get_ui_image": 0.02922031765892392, "step_physics": 0.12486359301067534, "survival_time": 52.44999999999916, "driven_lanedir": 11.625876477658874, "get_state_dump": 0.004837011155628023, "get_robot_state": 0.0039156786600748695, "sim_render-ego0": 0.004034169742039272, "get_duckie_state": 1.4209747314453125e-06, "in-drivable-lane": 17.399999999999192, "deviation-heading": 8.9617409958468, "agent_compute-ego0": 0.012168332054501489, "complete-iteration": 0.1939772131329491, "set_robot_commands": 0.0024574967793055944, "deviation-center-line": 2.981490389777582, "driven_lanedir_consec": 11.625876477658874, "sim_compute_sim_state": 0.010207970710027783, "sim_compute_performance-ego0": 0.0021830719993228003}, "LF-norm-zigzag-000-ego0": {"driven_any": 14.233929146867393, "get_ui_image": 0.037505260613637605, "step_physics": 0.1723802609804965, "survival_time": 59.99999999999873, "driven_lanedir": 10.161174343028827, "get_state_dump": 0.004862875465945737, "get_robot_state": 0.00387934165433682, "sim_render-ego0": 0.004066551456244958, "get_duckie_state": 1.6115587220203867e-06, "in-drivable-lane": 13.399999999999704, "deviation-heading": 17.12501799157715, "agent_compute-ego0": 0.012334981230673048, "complete-iteration": 0.2532162009230462, "set_robot_commands": 0.002364833587214512, "deviation-center-line": 3.644362243663646, "driven_lanedir_consec": 5.090408779808603, "sim_compute_sim_state": 0.01351610071752391, "sim_compute_performance-ego0": 0.002212513694159693}, "LF-norm-techtrack-000-ego0": {"driven_any": 15.927120794954964, "get_ui_image": 0.03511730757085211, "step_physics": 0.15499087872850606, "survival_time": 59.99999999999873, "driven_lanedir": 13.083261699572546, "get_state_dump": 0.005011944449215904, "get_robot_state": 0.003998904303646802, "sim_render-ego0": 0.004154481061987833, "get_duckie_state": 1.4299159244534178e-06, "in-drivable-lane": 9.449999999999935, "deviation-heading": 15.262145399033455, "agent_compute-ego0": 0.012497551137462048, "complete-iteration": 0.2340703093936104, "set_robot_commands": 0.0024444898101908283, "deviation-center-line": 5.02368170869833, "driven_lanedir_consec": 7.385144919171147, "sim_compute_sim_state": 0.013487689798817247, "sim_compute_performance-ego0": 0.00227262118178343}, "LF-norm-small_loop-000-ego0": {"driven_any": 12.839004206122835, "get_ui_image": 0.026256964863785898, "step_physics": 0.12333194818425236, "survival_time": 59.99999999999873, "driven_lanedir": 4.360561414278415, "get_state_dump": 0.004748696193012171, "get_robot_state": 0.0037098953269304184, "sim_render-ego0": 0.003798373235850211, "get_duckie_state": 1.2472805432931866e-06, "in-drivable-lane": 39.199999999998745, "deviation-heading": 7.141641202239131, "agent_compute-ego0": 0.011762345263999666, "complete-iteration": 0.18407882699164424, "set_robot_commands": 0.0022160286311801525, "deviation-center-line": 1.968515405564806, "driven_lanedir_consec": 3.029210853076907, "sim_compute_sim_state": 0.006205736449318663, "sim_compute_performance-ego0": 0.0019636259388665575}}
set_robot_commands_max0.0024574967793055944
set_robot_commands_mean0.0023707122019727715
set_robot_commands_median0.00240466169870267
set_robot_commands_min0.0022160286311801525
sim_compute_performance-ego0_max0.00227262118178343
sim_compute_performance-ego0_mean0.00215795820353312
sim_compute_performance-ego0_median0.002197792846741246
sim_compute_performance-ego0_min0.0019636259388665575
sim_compute_sim_state_max0.01351610071752391
sim_compute_sim_state_mean0.010854374418921902
sim_compute_sim_state_median0.011847830254422518
sim_compute_sim_state_min0.006205736449318663
sim_render-ego0_max0.004154481061987833
sim_render-ego0_mean0.004013393874030568
sim_render-ego0_median0.0040503605991421155
sim_render-ego0_min0.003798373235850211
simulation-passed1
step_physics_max0.1723802609804965
step_physics_mean0.1438916702259826
step_physics_median0.13992723586959072
step_physics_min0.12333194818425236
survival_time_max59.99999999999873
survival_time_mean58.11249999999883
survival_time_min52.44999999999916
No reset possible
5399511724Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:24:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.6730830936650118
survival_time_median50.24999999999928
deviation-center-line_median0.7702400244515315
in-drivable-lane_median38.89999999999935


other stats
agent_compute-ego0_max0.013042214301974367
agent_compute-ego0_mean0.012830552953855831
agent_compute-ego0_median0.012926587067919854
agent_compute-ego0_min0.012426823377609251
complete-iteration_max0.2601629062132402
complete-iteration_mean0.2211540603308642
complete-iteration_median0.2166339657282516
complete-iteration_min0.19118540365371345
deviation-center-line_max0.9875465306328964
deviation-center-line_mean0.6775293586116704
deviation-center-line_min0.18209085491072183
deviation-heading_max7.000977553484349
deviation-heading_mean3.9897417165546982
deviation-heading_median3.992284390278439
deviation-heading_min0.9734205321775669
driven_any_max10.640338809729757
driven_any_mean6.805884343544593
driven_any_median8.09063970967956
driven_any_min0.4019191450894945
driven_lanedir_consec_max1.884889258782392
driven_lanedir_consec_mean1.3626414190650304
driven_lanedir_consec_min0.21951023014770632
driven_lanedir_max1.884889258782392
driven_lanedir_mean1.3684580348940238
driven_lanedir_median1.6847163253229986
driven_lanedir_min0.21951023014770632
get_duckie_state_max1.3931903315027192e-06
get_duckie_state_mean1.3386612478062146e-06
get_duckie_state_median1.3765529760014285e-06
get_duckie_state_min1.2083487077192827e-06
get_robot_state_max0.0039790273342112424
get_robot_state_mean0.0038924086307350887
get_robot_state_median0.003940405694303382
get_robot_state_min0.003709795800122348
get_state_dump_max0.004942988006161972
get_state_dump_mean0.004870580959642248
get_state_dump_median0.004900174670125018
get_state_dump_min0.004738986492156982
get_ui_image_max0.03513145717707547
get_ui_image_mean0.031374235033645774
get_ui_image_median0.03163625080950333
get_ui_image_min0.027092981338500976
in-drivable-lane_max49.3999999999988
in-drivable-lane_mean32.38749999999938
in-drivable-lane_min2.3499999999999917
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.640338809729757, "get_ui_image": 0.029647582178806683, "step_physics": 0.13496362875145143, "survival_time": 59.99999999999873, "driven_lanedir": 1.884889258782392, "get_state_dump": 0.004910529801291689, "get_robot_state": 0.003947765602855063, "sim_render-ego0": 0.003984049099073322, "get_duckie_state": 1.3931903315027192e-06, "in-drivable-lane": 49.3999999999988, "deviation-heading": 4.437163726163125, "agent_compute-ego0": 0.012908747353025718, "complete-iteration": 0.20381983074915597, "set_robot_commands": 0.002396738400169455, "deviation-center-line": 0.8315720690308475, "driven_lanedir_consec": 1.884889258782392, "sim_compute_sim_state": 0.008806953223718394, "sim_compute_performance-ego0": 0.002168501545050857}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.4019191450894945, "get_ui_image": 0.03513145717707547, "step_physics": 0.18525399944999, "survival_time": 4.3499999999999925, "driven_lanedir": 0.21951023014770632, "get_state_dump": 0.004738986492156982, "get_robot_state": 0.003709795800122348, "sim_render-ego0": 0.003864767876538363, "get_duckie_state": 1.2083487077192827e-06, "in-drivable-lane": 2.3499999999999917, "deviation-heading": 0.9734205321775669, "agent_compute-ego0": 0.012426823377609251, "complete-iteration": 0.2601629062132402, "set_robot_commands": 0.002260777083310214, "deviation-center-line": 0.18209085491072183, "driven_lanedir_consec": 0.21951023014770632, "sim_compute_sim_state": 0.010683783076026222, "sim_compute_performance-ego0": 0.0020106705752286043}, "LF-norm-techtrack-000-ego0": {"driven_any": 8.256803055564275, "get_ui_image": 0.033624919440199984, "step_physics": 0.15314776668327035, "survival_time": 47.29999999999945, "driven_lanedir": 1.8610465372746796, "get_state_dump": 0.004889819538958347, "get_robot_state": 0.0039790273342112424, "sim_render-ego0": 0.003970824677689904, "get_duckie_state": 1.3879637028372903e-06, "in-drivable-lane": 36.899999999999565, "deviation-heading": 3.5474050543937543, "agent_compute-ego0": 0.013042214301974367, "complete-iteration": 0.2294481007073472, "set_robot_commands": 0.0023732389290959683, "deviation-center-line": 0.7089079798722155, "driven_lanedir_consec": 1.8610465372746796, "sim_compute_sim_state": 0.012194206999121146, "sim_compute_performance-ego0": 0.002136769742875064}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.924476363794846, "get_ui_image": 0.027092981338500976, "step_physics": 0.12671821531555463, "survival_time": 53.199999999999115, "driven_lanedir": 1.5083861133713177, "get_state_dump": 0.004942988006161972, "get_robot_state": 0.003933045785751701, "sim_render-ego0": 0.0040091700397187, "get_duckie_state": 1.3651422491655665e-06, "in-drivable-lane": 40.89999999999914, "deviation-heading": 7.000977553484349, "agent_compute-ego0": 0.012944426782813993, "complete-iteration": 0.19118540365371345, "set_robot_commands": 0.0024116791469950072, "deviation-center-line": 0.9875465306328964, "driven_lanedir_consec": 1.485119650055344, "sim_compute_sim_state": 0.006944360643485342, "sim_compute_performance-ego0": 0.0020997031753611675}}
set_robot_commands_max0.0024116791469950072
set_robot_commands_mean0.0023606083898926613
set_robot_commands_median0.002384988664632712
set_robot_commands_min0.002260777083310214
sim_compute_performance-ego0_max0.002168501545050857
sim_compute_performance-ego0_mean0.0021039112596289234
sim_compute_performance-ego0_median0.002118236459118116
sim_compute_performance-ego0_min0.0020106705752286043
sim_compute_sim_state_max0.012194206999121146
sim_compute_sim_state_mean0.009657325985587776
sim_compute_sim_state_median0.009745368149872308
sim_compute_sim_state_min0.006944360643485342
sim_render-ego0_max0.0040091700397187
sim_render-ego0_mean0.003957202923255072
sim_render-ego0_median0.003977436888381613
sim_render-ego0_min0.003864767876538363
simulation-passed1
step_physics_max0.18525399944999
step_physics_mean0.1500209025500666
step_physics_median0.14405569771736088
step_physics_min0.12671821531555463
survival_time_max59.99999999999873
survival_time_mean41.21249999999933
survival_time_min4.3499999999999925
No reset possible
5398411704brian wongtemplate-randomaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:04:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5040495128145347
survival_time_median4.6499999999999915
deviation-center-line_median0.10620228737371736
in-drivable-lane_median2.674999999999992


other stats
agent_compute-ego0_max0.012858036491605973
agent_compute-ego0_mean0.01169660348701002
agent_compute-ego0_median0.011564991388205454
agent_compute-ego0_min0.010798394680023192
complete-iteration_max0.21995294871537585
complete-iteration_mean0.1968194373740872
complete-iteration_median0.20189992884375713
complete-iteration_min0.1635249430934588
deviation-center-line_max0.16455763165774434
deviation-center-line_mean0.1092777488823554
deviation-center-line_min0.06014878912424249
deviation-heading_max1.819139548495243
deviation-heading_mean0.860079214882594
deviation-heading_median0.6517793617099584
deviation-heading_min0.31761858761521594
driven_any_max3.078535752374254
driven_any_mean1.8526223552312917
driven_any_median1.6219947547238958
driven_any_min1.0879641591031215
driven_lanedir_consec_max0.6643282900394009
driven_lanedir_consec_mean0.4884432809313369
driven_lanedir_consec_min0.28134580805687714
driven_lanedir_max0.6643282900394009
driven_lanedir_mean0.4924956463812869
driven_lanedir_median0.5121542437144349
driven_lanedir_min0.28134580805687714
get_duckie_state_max1.976761636854727e-06
get_duckie_state_mean1.7276934157203392e-06
get_duckie_state_median1.769134963768116e-06
get_duckie_state_min1.3957420984903972e-06
get_robot_state_max0.004153484030614925
get_robot_state_mean0.004038575121600176
get_robot_state_median0.004066545462262804
get_robot_state_min0.003867725531260173
get_state_dump_max0.00533316863907708
get_state_dump_mean0.0050657191193920975
get_state_dump_median0.00510700344711329
get_state_dump_min0.0047157009442647295
get_ui_image_max0.03741522716439288
get_ui_image_mean0.032029191630494995
get_ui_image_median0.03171509829167743
get_ui_image_min0.027271342774232227
in-drivable-lane_max6.549999999999982
in-drivable-lane_mean3.3499999999999894
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.078535752374254, "get_ui_image": 0.030461619171915175, "step_physics": 0.11504859411263768, "survival_time": 7.84999999999998, "driven_lanedir": 0.28134580805687714, "get_state_dump": 0.005143097684353213, "get_robot_state": 0.004153484030614925, "sim_render-ego0": 0.004129682915120185, "get_duckie_state": 1.976761636854727e-06, "in-drivable-lane": 6.549999999999982, "deviation-heading": 0.891006057417327, "agent_compute-ego0": 0.011834153646155247, "complete-iteration": 0.18600604202173937, "set_robot_commands": 0.0024235987965064714, "deviation-center-line": 0.13094886067496936, "driven_lanedir_consec": 0.28134580805687714, "sim_compute_sim_state": 0.010426387002196492, "sim_compute_performance-ego0": 0.002286666556249691}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5806581102909054, "get_ui_image": 0.03741522716439288, "step_physics": 0.14326460465140964, "survival_time": 4.549999999999992, "driven_lanedir": 0.3891269085945981, "get_state_dump": 0.005070909209873365, "get_robot_state": 0.0039815876794897995, "sim_render-ego0": 0.004082166630288829, "get_duckie_state": 1.829603443975034e-06, "in-drivable-lane": 2.449999999999993, "deviation-heading": 1.819139548495243, "agent_compute-ego0": 0.011295829130255656, "complete-iteration": 0.21995294871537585, "set_robot_commands": 0.0024407765139704165, "deviation-center-line": 0.16455763165774434, "driven_lanedir_consec": 0.37291744679479777, "sim_compute_sim_state": 0.01004536255546238, "sim_compute_performance-ego0": 0.002259347749793011}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0879641591031215, "get_ui_image": 0.03296857741143969, "step_physics": 0.1450369887881809, "survival_time": 3.5499999999999954, "driven_lanedir": 0.6643282900394009, "get_state_dump": 0.00533316863907708, "get_robot_state": 0.004151503245035808, "sim_render-ego0": 0.0041571491294436986, "get_duckie_state": 1.708666483561198e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.41255266600258983, "agent_compute-ego0": 0.012858036491605973, "complete-iteration": 0.21779381566577488, "set_robot_commands": 0.00241423315472073, "deviation-center-line": 0.08145571407246535, "driven_lanedir_consec": 0.6643282900394009, "sim_compute_sim_state": 0.008526835176679824, "sim_compute_performance-ego0": 0.002246648073196411}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6633313991568863, "get_ui_image": 0.027271342774232227, "step_physics": 0.10348597665627796, "survival_time": 4.749999999999991, "driven_lanedir": 0.6351815788342716, "get_state_dump": 0.0047157009442647295, "get_robot_state": 0.003867725531260173, "sim_render-ego0": 0.003795618812243144, "get_duckie_state": 1.3957420984903972e-06, "in-drivable-lane": 2.89999999999999, "deviation-heading": 0.31761858761521594, "agent_compute-ego0": 0.010798394680023192, "complete-iteration": 0.1635249430934588, "set_robot_commands": 0.0023261159658432007, "deviation-center-line": 0.06014878912424249, "driven_lanedir_consec": 0.6351815788342716, "sim_compute_sim_state": 0.005132339894771576, "sim_compute_performance-ego0": 0.002049016455809275}}
set_robot_commands_max0.0024407765139704165
set_robot_commands_mean0.002401181107760205
set_robot_commands_median0.002418915975613601
set_robot_commands_min0.0023261159658432007
sim_compute_performance-ego0_max0.002286666556249691
sim_compute_performance-ego0_mean0.002210419708762097
sim_compute_performance-ego0_median0.002252997911494711
sim_compute_performance-ego0_min0.002049016455809275
sim_compute_sim_state_max0.010426387002196492
sim_compute_sim_state_mean0.008532731157277568
sim_compute_sim_state_median0.009286098866071103
sim_compute_sim_state_min0.005132339894771576
sim_render-ego0_max0.0041571491294436986
sim_render-ego0_mean0.004041154371773965
sim_render-ego0_median0.004105924772704507
sim_render-ego0_min0.003795618812243144
simulation-passed1
step_physics_max0.1450369887881809
step_physics_mean0.12670904105212655
step_physics_median0.12915659938202367
step_physics_min0.10348597665627796
survival_time_max7.84999999999998
survival_time_mean5.17499999999999
survival_time_min3.5499999999999954
No reset possible
539589782Anastasiya Nikolskaya 🇷🇺jbr_sim2realaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:22:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.8094109977884565
survival_time_median9.074999999999996
deviation-center-line_median0.4100566928065677
in-drivable-lane_median4.350000000000005


other stats
agent_compute-ego0_max0.5094323569330675
agent_compute-ego0_mean0.4931180090533713
agent_compute-ego0_median0.49823567661501855
agent_compute-ego0_min0.4665683260503805
complete-iteration_max0.7619105413042265
complete-iteration_mean0.7174034928130639
complete-iteration_median0.7149306558176749
complete-iteration_min0.6778421183126792
deviation-center-line_max2.085843994556087
deviation-center-line_mean0.768397389686206
deviation-center-line_min0.16763217857560153
deviation-heading_max10.715939935152658
deviation-heading_mean3.7952132702878103
deviation-heading_median1.7056456725763844
deviation-heading_min1.0536218008458134
driven_any_max16.110754473891923
driven_any_mean5.9381451885515855
driven_any_median2.993548340267222
driven_any_min1.6547295997799754
driven_lanedir_consec_max6.968093017582005
driven_lanedir_consec_mean2.82146423291507
driven_lanedir_consec_min0.698941918501363
driven_lanedir_max6.968093017582005
driven_lanedir_mean2.82146423291507
driven_lanedir_median1.8094109977884565
driven_lanedir_min0.698941918501363
get_duckie_state_max1.5723294225232355e-06
get_duckie_state_mean1.384411347518783e-06
get_duckie_state_median1.3603704974416813e-06
get_duckie_state_min1.2445749726685338e-06
get_robot_state_max0.004073406087941137
get_robot_state_mean0.00387005549163992
get_robot_state_median0.0038554112875578313
get_robot_state_min0.0036959933035028807
get_state_dump_max0.004988105132662016
get_state_dump_mean0.00481597525786748
get_state_dump_median0.004846497471832644
get_state_dump_min0.004582800955142615
get_ui_image_max0.03678783877142545
get_ui_image_mean0.03324546978079625
get_ui_image_median0.034272259776973825
get_ui_image_min0.027649520797811883
in-drivable-lane_max23.79999999999938
in-drivable-lane_mean8.774999999999846
in-drivable-lane_min2.599999999999993
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.5290036421360136, "get_ui_image": 0.031763386957853745, "step_physics": 0.15290322928752714, "survival_time": 10.25000000000001, "driven_lanedir": 1.9332865247138091, "get_state_dump": 0.004952604330859138, "get_robot_state": 0.003862987444238755, "sim_render-ego0": 0.004007020042937936, "get_duckie_state": 1.393475578826608e-06, "in-drivable-lane": 5.35000000000002, "deviation-heading": 1.1554471909350217, "agent_compute-ego0": 0.507933064571862, "complete-iteration": 0.7208462421176502, "set_robot_commands": 0.002489148991779216, "deviation-center-line": 0.45119847948851266, "driven_lanedir_consec": 1.9332865247138091, "sim_compute_sim_state": 0.010771550021125276, "sim_compute_performance-ego0": 0.002068305478512662}, "LF-norm-zigzag-000-ego0": {"driven_any": 2.45809303839843, "get_ui_image": 0.036781132596093905, "step_physics": 0.1785632124486959, "survival_time": 7.89999999999998, "driven_lanedir": 1.685535470863104, "get_state_dump": 0.004582800955142615, "get_robot_state": 0.0036959933035028807, "sim_render-ego0": 0.003848020385646221, "get_duckie_state": 1.2445749726685338e-06, "in-drivable-lane": 2.599999999999993, "deviation-heading": 2.2558441542177476, "agent_compute-ego0": 0.4665683260503805, "complete-iteration": 0.7090150695176994, "set_robot_commands": 0.002322583828332289, "deviation-center-line": 0.36891490612462274, "driven_lanedir_consec": 1.685535470863104, "sim_compute_sim_state": 0.010472551081915322, "sim_compute_performance-ego0": 0.0020936360149263586}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.6547295997799754, "get_ui_image": 0.03678783877142545, "step_physics": 0.18855621691407828, "survival_time": 5.749999999999988, "driven_lanedir": 0.698941918501363, "get_state_dump": 0.004988105132662016, "get_robot_state": 0.004073406087941137, "sim_render-ego0": 0.004088816971614443, "get_duckie_state": 1.5723294225232355e-06, "in-drivable-lane": 3.34999999999999, "deviation-heading": 1.0536218008458134, "agent_compute-ego0": 0.5094323569330675, "complete-iteration": 0.7619105413042265, "set_robot_commands": 0.002480671323578933, "deviation-center-line": 0.16763217857560153, "driven_lanedir_consec": 0.698941918501363, "sim_compute_sim_state": 0.00915230759258928, "sim_compute_performance-ego0": 0.002247666490489039}, "LF-norm-small_loop-000-ego0": {"driven_any": 16.110754473891923, "get_ui_image": 0.027649520797811883, "step_physics": 0.13793988825178044, "survival_time": 46.24999999999951, "driven_lanedir": 6.968093017582005, "get_state_dump": 0.004740390612806151, "get_robot_state": 0.003847835130876908, "sim_render-ego0": 0.003950204756553436, "get_duckie_state": 1.3272654160567545e-06, "in-drivable-lane": 23.79999999999938, "deviation-heading": 10.715939935152658, "agent_compute-ego0": 0.48853828865817506, "complete-iteration": 0.6778421183126792, "set_robot_commands": 0.002399078455651038, "deviation-center-line": 2.085843994556087, "driven_lanedir_consec": 6.968093017582005, "sim_compute_sim_state": 0.00657465751433733, "sim_compute_performance-ego0": 0.002112677751298085}}
set_robot_commands_max0.002489148991779216
set_robot_commands_mean0.0024228706498353693
set_robot_commands_median0.0024398748896149857
set_robot_commands_min0.002322583828332289
sim_compute_performance-ego0_max0.002247666490489039
sim_compute_performance-ego0_mean0.002130571433806536
sim_compute_performance-ego0_median0.0021031568831122215
sim_compute_performance-ego0_min0.002068305478512662
sim_compute_sim_state_max0.010771550021125276
sim_compute_sim_state_mean0.009242766552491802
sim_compute_sim_state_median0.0098124293372523
sim_compute_sim_state_min0.00657465751433733
sim_render-ego0_max0.004088816971614443
sim_render-ego0_mean0.003973515539188009
sim_render-ego0_median0.003978612399745686
sim_render-ego0_min0.003848020385646221
simulation-passed1
step_physics_max0.18855621691407828
step_physics_mean0.16449063672552044
step_physics_median0.16573322086811154
step_physics_min0.13793988825178044
survival_time_max46.24999999999951
survival_time_mean17.537499999999874
survival_time_min5.749999999999988
No reset possible
5395611705brian wongbaseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:01:04
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5393411314Anastasiya Nikolskaya 🇷🇺JetBrains Researchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:03:15
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible