Duckietown Challenges Home Challenges Submissions

Evaluator 4880

ID4880
evaluatornogpu-prod-03
ownerI don't have one 😀
machinenogpu-prod_10db3794b871
processnogpu-prod-03_10db3794b871
version6.2.5
first heard
last heard
statusinactive
# evaluating
# success19 53913
# timeout2 55812
# failed3 54981
# error
# aborted4 57299
# host-error36 54371
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5753610899Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-03----No reset possible
5753011010Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-030:00:44
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5752611010Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-030:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5750110933Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-030:05:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5737610948Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:26:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.9263802094937414
survival_time_median45.84999999999953
deviation-center-line_median1.6199037866208013
in-drivable-lane_median6.8499999999999135


other stats
agent_compute-ego0_max0.012832112985443386
agent_compute-ego0_mean0.012523007585785909
agent_compute-ego0_median0.012712065689150964
agent_compute-ego0_min0.01183578597939832
complete-iteration_max0.20211514262063823
complete-iteration_mean0.17814812305129774
complete-iteration_median0.1761994643453712
complete-iteration_min0.15807842089381027
deviation-center-line_max2.2130345588034026
deviation-center-line_mean1.4169028216274049
deviation-center-line_min0.214769154464614
deviation-heading_max9.733342049057544
deviation-heading_mean6.039862400951661
deviation-heading_median6.758307077220756
deviation-heading_min0.9094934003075824
driven_any_max8.332500871563104
driven_any_mean5.562755334951762
driven_any_median6.0954881853823375
driven_any_min1.7275440974792724
driven_lanedir_consec_max7.615922491136805
driven_lanedir_consec_mean4.026300823823629
driven_lanedir_consec_min0.6365203851702277
driven_lanedir_max7.615922491136805
driven_lanedir_mean4.026300823823629
driven_lanedir_median3.9263802094937414
driven_lanedir_min0.6365203851702277
get_duckie_state_max1.5410852868987758e-06
get_duckie_state_mean1.327768603922295e-06
get_duckie_state_median1.2799675869303011e-06
get_duckie_state_min1.210053954929801e-06
get_robot_state_max0.0038222105278758385
get_robot_state_mean0.0036996261277230736
get_robot_state_median0.003694507545803246
get_robot_state_min0.0035872788914099616
get_state_dump_max0.004762507596678976
get_state_dump_mean0.004609707596516002
get_state_dump_median0.004614312568840517
get_state_dump_min0.004447697651703996
get_ui_image_max0.03543959324443905
get_ui_image_mean0.030115313418156143
get_ui_image_median0.02928751653131627
get_ui_image_min0.02644662736555299
in-drivable-lane_max26.69999999999928
in-drivable-lane_mean10.924999999999764
in-drivable-lane_min3.299999999999951
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.332500871563104, "get_ui_image": 0.028511127365518862, "step_physics": 0.1071542774410073, "survival_time": 59.99999999999873, "driven_lanedir": 7.615922491136805, "get_state_dump": 0.004762507596678976, "get_robot_state": 0.0038222105278758385, "sim_render-ego0": 0.003973466172801962, "get_duckie_state": 1.5410852868987758e-06, "in-drivable-lane": 3.299999999999951, "deviation-heading": 9.733342049057544, "agent_compute-ego0": 0.012765888766781874, "complete-iteration": 0.17553776467868829, "set_robot_commands": 0.002264662249499217, "deviation-center-line": 2.2130345588034026, "driven_lanedir_consec": 7.615922491136805, "sim_compute_sim_state": 0.010098756104087354, "sim_compute_performance-ego0": 0.002088278556842788}, "LF-norm-zigzag-000-ego0": {"driven_any": 5.5445253353098085, "get_ui_image": 0.03543959324443905, "step_physics": 0.12656901152136835, "survival_time": 41.0999999999998, "driven_lanedir": 4.7212785160081765, "get_state_dump": 0.0045869321463898926, "get_robot_state": 0.003655924976668699, "sim_render-ego0": 0.0038872106976236783, "get_duckie_state": 1.210053954929801e-06, "in-drivable-lane": 5.049999999999756, "deviation-heading": 8.834651984900459, "agent_compute-ego0": 0.012658242611520055, "complete-iteration": 0.20211514262063823, "set_robot_commands": 0.00216165666591788, "deviation-center-line": 2.023846611379585, "driven_lanedir_consec": 4.7212785160081765, "sim_compute_sim_state": 0.0110999156029517, "sim_compute_performance-ego0": 0.001972343333581528}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.6464510354548665, "get_ui_image": 0.03006390569711367, "step_physics": 0.10845242883469486, "survival_time": 50.59999999999926, "driven_lanedir": 3.131481902979307, "get_state_dump": 0.004447697651703996, "get_robot_state": 0.0035872788914099616, "sim_render-ego0": 0.003844067233689093, "get_duckie_state": 1.2542276486004211e-06, "in-drivable-lane": 26.69999999999928, "deviation-heading": 4.6819621695410545, "agent_compute-ego0": 0.012832112985443386, "complete-iteration": 0.17686116401205412, "set_robot_commands": 0.0021463043009540524, "deviation-center-line": 1.2159609618620173, "driven_lanedir_consec": 3.131481902979307, "sim_compute_sim_state": 0.009416984076090417, "sim_compute_performance-ego0": 0.0019831130205702333}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.7275440974792724, "get_ui_image": 0.02644662736555299, "step_physics": 0.09806975733071888, "survival_time": 13.80000000000006, "driven_lanedir": 0.6365203851702277, "get_state_dump": 0.004641692991291142, "get_robot_state": 0.003733090114937793, "sim_render-ego0": 0.00387380097316921, "get_duckie_state": 1.3057075252601816e-06, "in-drivable-lane": 8.650000000000071, "deviation-heading": 0.9094934003075824, "agent_compute-ego0": 0.01183578597939832, "complete-iteration": 0.15807842089381027, "set_robot_commands": 0.0021967009947188062, "deviation-center-line": 0.214769154464614, "driven_lanedir_consec": 0.6365203851702277, "sim_compute_sim_state": 0.005279733816208822, "sim_compute_performance-ego0": 0.0019164636246994512}}
set_robot_commands_max0.002264662249499217
set_robot_commands_mean0.002192331052772489
set_robot_commands_median0.002179178830318343
set_robot_commands_min0.0021463043009540524
sim_compute_performance-ego0_max0.002088278556842788
sim_compute_performance-ego0_mean0.0019900496339235003
sim_compute_performance-ego0_median0.0019777281770758806
sim_compute_performance-ego0_min0.0019164636246994512
sim_compute_sim_state_max0.0110999156029517
sim_compute_sim_state_mean0.008973847399834574
sim_compute_sim_state_median0.009757870090088884
sim_compute_sim_state_min0.005279733816208822
sim_render-ego0_max0.003973466172801962
sim_render-ego0_mean0.0038946362693209856
sim_render-ego0_median0.003880505835396444
sim_render-ego0_min0.003844067233689093
simulation-passed1
step_physics_max0.12656901152136835
step_physics_mean0.11006136878194736
step_physics_median0.10780335313785108
step_physics_min0.09806975733071888
survival_time_max59.99999999999873
survival_time_mean41.37499999999947
survival_time_min13.80000000000006
No reset possible
5730310965Himanshu Arora 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:18:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.603234669151191
survival_time_median27.175000000000253
deviation-center-line_median1.1791437757685823
in-drivable-lane_median12.274999999999867


other stats
agent_compute-ego0_max0.013104039142724651
agent_compute-ego0_mean0.01224615634875433
agent_compute-ego0_median0.012125595968484464
agent_compute-ego0_min0.01162939431532374
complete-iteration_max0.2013708845660597
complete-iteration_mean0.1772261929388591
complete-iteration_median0.18040810575624555
complete-iteration_min0.14671767567688565
deviation-center-line_max2.775382428514279
deviation-center-line_mean1.4471610474481988
deviation-center-line_min0.654974209741351
deviation-heading_max12.42289828312
deviation-heading_mean6.495197138318199
deviation-heading_median5.324756224112662
deviation-heading_min2.9083778219274734
driven_any_max9.597226160109589
driven_any_mean6.647486190277972
driven_any_median5.7777546477393695
driven_any_min5.437209305523561
driven_lanedir_consec_max5.646686895274053
driven_lanedir_consec_mean3.6893393640506247
driven_lanedir_consec_min1.9042012226260632
driven_lanedir_max6.403196366022625
driven_lanedir_mean3.8797297624979015
driven_lanedir_median3.605644029255046
driven_lanedir_min1.9044346254588889
get_duckie_state_max1.5678474003172518e-06
get_duckie_state_mean1.3516804413530209e-06
get_duckie_state_median1.286053485613454e-06
get_duckie_state_min1.2667673938679245e-06
get_robot_state_max0.003816163603862838
get_robot_state_mean0.0035850951164019627
get_robot_state_median0.0035818613669658193
get_robot_state_min0.0033604941278133752
get_state_dump_max0.004919713213106814
get_state_dump_mean0.004669884973345857
get_state_dump_median0.004703324004102921
get_state_dump_min0.004353178672070773
get_ui_image_max0.03301800139711041
get_ui_image_mean0.028654755687049237
get_ui_image_median0.028827141778914944
get_ui_image_min0.02394673779325665
in-drivable-lane_max18.750000000000252
in-drivable-lane_mean12.45000000000002
in-drivable-lane_min6.500000000000092
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.613152801267365, "get_ui_image": 0.02638841869935611, "step_physics": 0.09762092223993672, "survival_time": 23.900000000000205, "driven_lanedir": 3.386855723683451, "get_state_dump": 0.004836858936540767, "get_robot_state": 0.0036548269068771714, "sim_render-ego0": 0.0038014438804355698, "get_duckie_state": 1.2996052401548636e-06, "in-drivable-lane": 10.700000000000138, "deviation-heading": 3.1443201083219434, "agent_compute-ego0": 0.011733354755632565, "complete-iteration": 0.16239364759409353, "set_robot_commands": 0.002113757900007085, "deviation-center-line": 1.0459270003535406, "driven_lanedir_consec": 3.386855723683451, "sim_compute_sim_state": 0.010170307438159536, "sim_compute_performance-ego0": 0.001986292060581279}, "LF-norm-zigzag-000-ego0": {"driven_any": 9.597226160109589, "get_ui_image": 0.03301800139711041, "step_physics": 0.12342522716113792, "survival_time": 46.64999999999949, "driven_lanedir": 6.403196366022625, "get_state_dump": 0.004569789071665075, "get_robot_state": 0.0035088958270544667, "sim_render-ego0": 0.0037789063994838543, "get_duckie_state": 1.272501731072044e-06, "in-drivable-lane": 13.849999999999596, "deviation-heading": 12.42289828312, "agent_compute-ego0": 0.012517837181336365, "complete-iteration": 0.1984225639183976, "set_robot_commands": 0.002176844843960421, "deviation-center-line": 2.775382428514279, "driven_lanedir_consec": 5.646686895274053, "sim_compute_sim_state": 0.013434565552296936, "sim_compute_performance-ego0": 0.0019113335925966056}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.942356494211374, "get_ui_image": 0.031265864858473776, "step_physics": 0.12881990878014743, "survival_time": 27.90000000000026, "driven_lanedir": 1.9044346254588889, "get_state_dump": 0.004919713213106814, "get_robot_state": 0.003816163603862838, "sim_render-ego0": 0.004053065090145322, "get_duckie_state": 1.5678474003172518e-06, "in-drivable-lane": 18.750000000000252, "deviation-heading": 2.9083778219274734, "agent_compute-ego0": 0.013104039142724651, "complete-iteration": 0.2013708845660597, "set_robot_commands": 0.0023048525419559376, "deviation-center-line": 0.654974209741351, "driven_lanedir_consec": 1.9042012226260632, "sim_compute_sim_state": 0.010854419953921187, "sim_compute_performance-ego0": 0.00213323680147502}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.437209305523561, "get_ui_image": 0.02394673779325665, "step_physics": 0.09078523797808952, "survival_time": 26.45000000000024, "driven_lanedir": 3.82443233482664, "get_state_dump": 0.004353178672070773, "get_robot_state": 0.0033604941278133752, "sim_render-ego0": 0.003484217625743938, "get_duckie_state": 1.2667673938679245e-06, "in-drivable-lane": 6.500000000000092, "deviation-heading": 7.50519233990338, "agent_compute-ego0": 0.01162939431532374, "complete-iteration": 0.14671767567688565, "set_robot_commands": 0.001989202229481823, "deviation-center-line": 1.3123605511836245, "driven_lanedir_consec": 3.8196136146189312, "sim_compute_sim_state": 0.0053340947852944425, "sim_compute_performance-ego0": 0.0017579843413155034}}
set_robot_commands_max0.0023048525419559376
set_robot_commands_mean0.0021461643788513165
set_robot_commands_median0.002145301371983753
set_robot_commands_min0.001989202229481823
sim_compute_performance-ego0_max0.00213323680147502
sim_compute_performance-ego0_mean0.001947211698992102
sim_compute_performance-ego0_median0.001948812826588942
sim_compute_performance-ego0_min0.0017579843413155034
sim_compute_sim_state_max0.013434565552296936
sim_compute_sim_state_mean0.009948346932418023
sim_compute_sim_state_median0.01051236369604036
sim_compute_sim_state_min0.0053340947852944425
sim_render-ego0_max0.004053065090145322
sim_render-ego0_mean0.003779408248952171
sim_render-ego0_median0.003790175139959712
sim_render-ego0_min0.003484217625743938
simulation-passed1
step_physics_max0.12881990878014743
step_physics_mean0.1101628240398279
step_physics_median0.11052307470053732
step_physics_min0.09078523797808952
survival_time_max46.64999999999949
survival_time_mean31.22500000000005
survival_time_min23.900000000000205
No reset possible
5729911012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-030:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5726110983Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-030:05:28
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 185 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5721310990Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:15:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.3503844897113728
survival_time_median19.52500000000014
deviation-center-line_median0.5907820774188185
in-drivable-lane_median9.6750000000001


other stats
agent_compute-ego0_max0.035073387499937075
agent_compute-ego0_mean0.019186856113031177
agent_compute-ego0_median0.01411469742981504
agent_compute-ego0_min0.013444642092557562
complete-iteration_max0.23083813531058175
complete-iteration_mean0.19329086782404495
complete-iteration_median0.186179607706582
complete-iteration_min0.16996612057243426
deviation-center-line_max0.6241910560827437
deviation-center-line_mean0.4841165940924948
deviation-center-line_min0.13071116544959854
deviation-heading_max1.6855699240442534
deviation-heading_mean1.3771271389956814
deviation-heading_median1.467375810988969
deviation-heading_min0.8881870099605342
driven_any_max5.9163231575379776
driven_any_mean3.107822218438933
driven_any_median2.888982495773834
driven_any_min0.7370007246700855
driven_lanedir_consec_max2.1102686277046985
driven_lanedir_consec_mean1.2631874894939277
driven_lanedir_consec_min0.2417123508482657
driven_lanedir_max2.1102686277046985
driven_lanedir_mean1.2631874894939277
driven_lanedir_median1.3503844897113728
driven_lanedir_min0.2417123508482657
get_duckie_state_max1.948151812457398e-06
get_duckie_state_mean1.739343731579034e-06
get_duckie_state_median1.789110027275092e-06
get_duckie_state_min1.4310030593085537e-06
get_robot_state_max0.004319158976510067
get_robot_state_mean0.004083652131694664
get_robot_state_median0.004149490369616716
get_robot_state_min0.003716468811035156
get_state_dump_max0.005138415382021949
get_state_dump_mean0.004905162225842468
get_state_dump_median0.004975740355056336
get_state_dump_min0.004530752811235251
get_ui_image_max0.036634967440650576
get_ui_image_mean0.03115078128079285
get_ui_image_median0.031440867071448525
get_ui_image_min0.025086423539623773
in-drivable-lane_max47.89999999999891
in-drivable-lane_mean17.44999999999979
in-drivable-lane_min2.550000000000036
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.4033189406881608, "get_ui_image": 0.029429419728733548, "step_physics": 0.11082185354808834, "survival_time": 14.850000000000076, "driven_lanedir": 2.1102686277046985, "get_state_dump": 0.005093729736020901, "get_robot_state": 0.004319158976510067, "sim_render-ego0": 0.004290043907677567, "get_duckie_state": 1.948151812457398e-06, "in-drivable-lane": 2.550000000000036, "deviation-heading": 1.5642886871439767, "agent_compute-ego0": 0.013444642092557562, "complete-iteration": 0.1831492177592028, "set_robot_commands": 0.002497130592397395, "deviation-center-line": 0.6201845609808305, "driven_lanedir_consec": 2.1102686277046985, "sim_compute_sim_state": 0.010732358734079656, "sim_compute_performance-ego0": 0.002409962199678357}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.7370007246700855, "get_ui_image": 0.036634967440650576, "step_physics": 0.1500767435346331, "survival_time": 5.1999999999999895, "driven_lanedir": 0.2417123508482657, "get_state_dump": 0.005138415382021949, "get_robot_state": 0.004256666274297805, "sim_render-ego0": 0.00441823232741583, "get_duckie_state": 1.8369583856491816e-06, "in-drivable-lane": 3.2999999999999883, "deviation-heading": 0.8881870099605342, "agent_compute-ego0": 0.013751929146902904, "complete-iteration": 0.23083813531058175, "set_robot_commands": 0.0025690555572509765, "deviation-center-line": 0.13071116544959854, "driven_lanedir_consec": 0.2417123508482657, "sim_compute_sim_state": 0.01150514284769694, "sim_compute_performance-ego0": 0.0023758729298909505}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.9163231575379776, "get_ui_image": 0.0334523144141635, "step_physics": 0.1120288876876288, "survival_time": 56.14999999999895, "driven_lanedir": 1.339685515352663, "get_state_dump": 0.004857750974091771, "get_robot_state": 0.004042314464935628, "sim_render-ego0": 0.00439004096272153, "get_duckie_state": 1.7412616689010023e-06, "in-drivable-lane": 47.89999999999891, "deviation-heading": 1.6855699240442534, "agent_compute-ego0": 0.014477465712727176, "complete-iteration": 0.1892099976539612, "set_robot_commands": 0.002413782784947297, "deviation-center-line": 0.5613795938568065, "driven_lanedir_consec": 1.339685515352663, "sim_compute_sim_state": 0.011179385024033407, "sim_compute_performance-ego0": 0.0022641568845701387}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.3746460508595075, "get_ui_image": 0.025086423539623773, "step_physics": 0.0870883096124708, "survival_time": 24.20000000000021, "driven_lanedir": 1.3610834640700828, "get_state_dump": 0.004530752811235251, "get_robot_state": 0.003716468811035156, "sim_render-ego0": 0.0038423675851723582, "get_duckie_state": 1.4310030593085537e-06, "in-drivable-lane": 16.05000000000021, "deviation-heading": 1.370462934833961, "agent_compute-ego0": 0.035073387499937075, "complete-iteration": 0.16996612057243426, "set_robot_commands": 0.0022252018918696138, "deviation-center-line": 0.6241910560827437, "driven_lanedir_consec": 1.3610834640700828, "sim_compute_sim_state": 0.006376182910093327, "sim_compute_performance-ego0": 0.0019398305833954173}}
set_robot_commands_max0.0025690555572509765
set_robot_commands_mean0.0024262927066163206
set_robot_commands_median0.002455456688672346
set_robot_commands_min0.0022252018918696138
sim_compute_performance-ego0_max0.002409962199678357
sim_compute_performance-ego0_mean0.002247455649383716
sim_compute_performance-ego0_median0.0023200149072305446
sim_compute_performance-ego0_min0.0019398305833954173
sim_compute_sim_state_max0.01150514284769694
sim_compute_sim_state_mean0.009948267378975834
sim_compute_sim_state_median0.010955871879056531
sim_compute_sim_state_min0.006376182910093327
sim_render-ego0_max0.00441823232741583
sim_render-ego0_mean0.004235171195746822
sim_render-ego0_median0.004340042435199549
sim_render-ego0_min0.0038423675851723582
simulation-passed1
step_physics_max0.1500767435346331
step_physics_mean0.11500394859570526
step_physics_median0.11142537061785857
step_physics_min0.0870883096124708
survival_time_max56.14999999999895
survival_time_mean25.0999999999998
survival_time_min5.1999999999999895
No reset possible
5711411011Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:21:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.75797194398709
survival_time_median46.02499999999952
deviation-center-line_median2.24404729322509
in-drivable-lane_median4.425000000000049


other stats
agent_compute-ego0_max0.013003858499582565
agent_compute-ego0_mean0.012507692168015712
agent_compute-ego0_median0.012609229540731305
agent_compute-ego0_min0.01180845109101768
complete-iteration_max0.23387963431222095
complete-iteration_mean0.18133714757818944
complete-iteration_median0.17063174636834466
complete-iteration_min0.15020546326384737
deviation-center-line_max4.265086003065298
deviation-center-line_mean2.2168587924445697
deviation-center-line_min0.11425458026280086
deviation-heading_max10.984918353266076
deviation-heading_mean6.878378352631122
deviation-heading_median7.4878348678194175
deviation-heading_min1.5529253216195782
driven_any_max8.316675874059532
driven_any_mean5.335520988996597
driven_any_median6.371275206902803
driven_any_min0.2828576681212511
driven_lanedir_consec_max8.139758514574586
driven_lanedir_consec_mean4.44114340716061
driven_lanedir_consec_min0.1088712260936755
driven_lanedir_max8.139758514574586
driven_lanedir_mean4.44114340716061
driven_lanedir_median4.75797194398709
driven_lanedir_min0.1088712260936755
get_duckie_state_max2.5612967354910712e-06
get_duckie_state_mean2.074011307548436e-06
get_duckie_state_median1.99985742370453e-06
get_duckie_state_min1.7350336472936137e-06
get_robot_state_max0.0036709185543902017
get_robot_state_mean0.0035271377833832125
get_robot_state_median0.0035512302184737337
get_robot_state_min0.003335172142195182
get_state_dump_max0.004651306868592071
get_state_dump_mean0.0044741395875389734
get_state_dump_median0.00451324150113355
get_state_dump_min0.004218768479296723
get_ui_image_max0.03482691560472761
get_ui_image_mean0.029630140607020537
get_ui_image_median0.02910794095855923
get_ui_image_min0.025477764906236076
in-drivable-lane_max15.150000000000215
in-drivable-lane_mean6.000000000000078
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.873004950410802, "get_ui_image": 0.02565951585026916, "step_physics": 0.0880777917547018, "survival_time": 32.05000000000032, "driven_lanedir": 3.8680199432576017, "get_state_dump": 0.004218768479296723, "get_robot_state": 0.003335172142195182, "sim_render-ego0": 0.0035380121332091335, "get_duckie_state": 1.7350336472936137e-06, "in-drivable-lane": 7.450000000000102, "deviation-heading": 4.157003807664016, "agent_compute-ego0": 0.01180845109101768, "complete-iteration": 0.15020546326384737, "set_robot_commands": 0.001978642472596926, "deviation-center-line": 1.287274397907784, "driven_lanedir_consec": 3.8680199432576017, "sim_compute_sim_state": 0.009737520203040768, "sim_compute_performance-ego0": 0.001768400735944231}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.2828576681212511, "get_ui_image": 0.03482691560472761, "step_physics": 0.1609881741659982, "survival_time": 3.4499999999999957, "driven_lanedir": 0.1088712260936755, "get_state_dump": 0.004442395482744489, "get_robot_state": 0.003492804936000279, "sim_render-ego0": 0.00374711241040911, "get_duckie_state": 2.5612967354910712e-06, "in-drivable-lane": 1.399999999999995, "deviation-heading": 1.5529253216195782, "agent_compute-ego0": 0.012854715756007605, "complete-iteration": 0.23387963431222095, "set_robot_commands": 0.0021381752831595283, "deviation-center-line": 0.11425458026280086, "driven_lanedir_consec": 0.1088712260936755, "sim_compute_sim_state": 0.0094597475869315, "sim_compute_performance-ego0": 0.0018412930624825616}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.869545463394805, "get_ui_image": 0.0325563660668493, "step_physics": 0.1140575762295306, "survival_time": 59.99999999999873, "driven_lanedir": 5.647923944716577, "get_state_dump": 0.00458408751952261, "get_robot_state": 0.0036709185543902017, "sim_render-ego0": 0.003899483557644732, "get_duckie_state": 1.9434786756072412e-06, "in-drivable-lane": 15.150000000000215, "deviation-heading": 10.984918353266076, "agent_compute-ego0": 0.013003858499582565, "complete-iteration": 0.1883136003241749, "set_robot_commands": 0.002196389967754024, "deviation-center-line": 3.200820188542396, "driven_lanedir_consec": 5.647923944716577, "sim_compute_sim_state": 0.012259853968116068, "sim_compute_performance-ego0": 0.0019928982216154507}, "LF-norm-small_loop-000-ego0": {"driven_any": 8.316675874059532, "get_ui_image": 0.025477764906236076, "step_physics": 0.0927810897239539, "survival_time": 59.99999999999873, "driven_lanedir": 8.139758514574586, "get_state_dump": 0.004651306868592071, "get_robot_state": 0.003609655500947189, "sim_render-ego0": 0.003775774688148975, "get_duckie_state": 2.0562361718018187e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.818665927974818, "agent_compute-ego0": 0.012363743325455003, "complete-iteration": 0.15294989241251442, "set_robot_commands": 0.002138195982781378, "deviation-center-line": 4.265086003065298, "driven_lanedir_consec": 8.139758514574586, "sim_compute_sim_state": 0.006130777925972538, "sim_compute_performance-ego0": 0.001929540618273936}}
set_robot_commands_max0.002196389967754024
set_robot_commands_mean0.002112850926572964
set_robot_commands_median0.002138185632970453
set_robot_commands_min0.001978642472596926
sim_compute_performance-ego0_max0.0019928982216154507
sim_compute_performance-ego0_mean0.0018830331595790447
sim_compute_performance-ego0_median0.0018854168403782488
sim_compute_performance-ego0_min0.001768400735944231
sim_compute_sim_state_max0.012259853968116068
sim_compute_sim_state_mean0.009396974921015218
sim_compute_sim_state_median0.009598633894986134
sim_compute_sim_state_min0.006130777925972538
sim_render-ego0_max0.003899483557644732
sim_render-ego0_mean0.003740095697352987
sim_render-ego0_median0.003761443549279042
sim_render-ego0_min0.0035380121332091335
simulation-passed1
step_physics_max0.1609881741659982
step_physics_mean0.11397615796854613
step_physics_median0.10341933297674225
step_physics_min0.0880777917547018
survival_time_max59.99999999999873
survival_time_mean38.87499999999944
survival_time_min3.4499999999999957
No reset possible
5701311042Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:16:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.851681632626355
survival_time_median22.450000000000184
deviation-center-line_median0.9314523628453856
in-drivable-lane_median10.075000000000102


other stats
agent_compute-ego0_max0.013143087027805997
agent_compute-ego0_mean0.012759718520955563
agent_compute-ego0_median0.012794713774796893
agent_compute-ego0_min0.012306359506422473
complete-iteration_max0.20854466641708755
complete-iteration_mean0.1785668415293016
complete-iteration_median0.17607280746154952
complete-iteration_min0.15357708477701978
deviation-center-line_max1.239011082988793
deviation-center-line_mean0.8256107731406793
deviation-center-line_min0.2005272838831528
deviation-heading_max4.6708598209924554
deviation-heading_mean3.434807437237988
deviation-heading_median4.05163755423591
deviation-heading_min0.9650948194876768
driven_any_max8.413900548435564
driven_any_mean4.165783219827891
driven_any_median3.2181145394523165
driven_any_min1.8130032519713692
driven_lanedir_consec_max4.0719871135765455
driven_lanedir_consec_mean2.1225338377429255
driven_lanedir_consec_min0.7147849721424471
driven_lanedir_max4.0719871135765455
driven_lanedir_mean2.1225338377429255
driven_lanedir_median1.851681632626355
driven_lanedir_min0.7147849721424471
get_duckie_state_max1.4075100532489271e-06
get_duckie_state_mean1.2414858835447544e-06
get_duckie_state_median1.1899822521140407e-06
get_duckie_state_min1.178468976702009e-06
get_robot_state_max0.003733715011370204
get_robot_state_mean0.003633627941772669
get_robot_state_median0.0036167810769389135
get_robot_state_min0.003567234601842643
get_state_dump_max0.005042239332464497
get_state_dump_mean0.004711205845320771
get_state_dump_median0.004632318990310893
get_state_dump_min0.004537946068196802
get_ui_image_max0.03567390560773601
get_ui_image_mean0.03022012819841645
get_ui_image_median0.029544941241340657
get_ui_image_min0.026116724703248464
in-drivable-lane_max28.799999999999415
in-drivable-lane_mean13.649999999999917
in-drivable-lane_min5.650000000000045
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.413900548435564, "get_ui_image": 0.02669586344248688, "step_physics": 0.0957327212056806, "survival_time": 54.19999999999906, "driven_lanedir": 4.0719871135765455, "get_state_dump": 0.004537946068196802, "get_robot_state": 0.003567234601842643, "sim_render-ego0": 0.003775220202960177, "get_duckie_state": 1.178468976702009e-06, "in-drivable-lane": 28.799999999999415, "deviation-heading": 4.089527615772002, "agent_compute-ego0": 0.012306359506422473, "complete-iteration": 0.1613250925793626, "set_robot_commands": 0.0021084846988801034, "deviation-center-line": 1.0081914513192451, "driven_lanedir_consec": 4.0719871135765455, "sim_compute_sim_state": 0.010578641144361365, "sim_compute_performance-ego0": 0.0019417749571910103}, "LF-norm-zigzag-000-ego0": {"driven_any": 2.5281930386272955, "get_ui_image": 0.03567390560773601, "step_physics": 0.13219378529493167, "survival_time": 18.00000000000012, "driven_lanedir": 1.6907705570320866, "get_state_dump": 0.004624159382320837, "get_robot_state": 0.003645603676581977, "sim_render-ego0": 0.003981952191720048, "get_duckie_state": 1.1815258670711785e-06, "in-drivable-lane": 5.650000000000045, "deviation-heading": 4.013747492699819, "agent_compute-ego0": 0.013143087027805997, "complete-iteration": 0.20854466641708755, "set_robot_commands": 0.0022217263145129767, "deviation-center-line": 0.8547132743715263, "driven_lanedir_consec": 1.6907705570320866, "sim_compute_sim_state": 0.01093538051827132, "sim_compute_performance-ego0": 0.0020392588301048386}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.908036040277338, "get_ui_image": 0.03239401904019443, "step_physics": 0.11741646499492242, "survival_time": 26.900000000000247, "driven_lanedir": 2.012592708220623, "get_state_dump": 0.005042239332464497, "get_robot_state": 0.003733715011370204, "sim_render-ego0": 0.004082961073611796, "get_duckie_state": 1.4075100532489271e-06, "in-drivable-lane": 12.150000000000144, "deviation-heading": 4.6708598209924554, "agent_compute-ego0": 0.013135727349813884, "complete-iteration": 0.1908205223437364, "set_robot_commands": 0.0022270467153064394, "deviation-center-line": 1.239011082988793, "driven_lanedir_consec": 2.012592708220623, "sim_compute_sim_state": 0.01060708201661402, "sim_compute_performance-ego0": 0.0020879013861265167}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.8130032519713692, "get_ui_image": 0.026116724703248464, "step_physics": 0.09352053798196888, "survival_time": 13.100000000000051, "driven_lanedir": 0.7147849721424471, "get_state_dump": 0.004640478598300949, "get_robot_state": 0.00358795847729585, "sim_render-ego0": 0.003823417221185373, "get_duckie_state": 1.1984386371569034e-06, "in-drivable-lane": 8.00000000000006, "deviation-heading": 0.9650948194876768, "agent_compute-ego0": 0.012453700199779904, "complete-iteration": 0.15357708477701978, "set_robot_commands": 0.0021981739725903415, "deviation-center-line": 0.2005272838831528, "driven_lanedir_consec": 0.7147849721424471, "sim_compute_sim_state": 0.005236831454723053, "sim_compute_performance-ego0": 0.0019179605259188228}}
set_robot_commands_max0.0022270467153064394
set_robot_commands_mean0.002188857925322465
set_robot_commands_median0.002209950143551659
set_robot_commands_min0.0021084846988801034
sim_compute_performance-ego0_max0.0020879013861265167
sim_compute_performance-ego0_mean0.001996723924835297
sim_compute_performance-ego0_median0.001990516893647924
sim_compute_performance-ego0_min0.0019179605259188228
sim_compute_sim_state_max0.01093538051827132
sim_compute_sim_state_mean0.00933948378349244
sim_compute_sim_state_median0.010592861580487691
sim_compute_sim_state_min0.005236831454723053
sim_render-ego0_max0.004082961073611796
sim_render-ego0_mean0.003915887672369348
sim_render-ego0_median0.0039026847064527106
sim_render-ego0_min0.003775220202960177
simulation-passed1
step_physics_max0.13219378529493167
step_physics_mean0.1097158773693759
step_physics_median0.10657459310030153
step_physics_min0.09352053798196888
survival_time_max54.19999999999906
survival_time_mean28.04999999999987
survival_time_min13.100000000000051
No reset possible
5699111313Moustafa Elarabitemplate-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-030:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5694211054Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:13:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6456454384235769
survival_time_median18.800000000000132
deviation-center-line_median0.5431785528261694
in-drivable-lane_median12.050000000000065


other stats
agent_compute-ego0_max0.013667882770620369
agent_compute-ego0_mean0.013191243616553584
agent_compute-ego0_median0.013383873736095696
agent_compute-ego0_min0.012329344223402572
complete-iteration_max0.22116704717759164
complete-iteration_mean0.1840851765751103
complete-iteration_median0.1825445470021556
complete-iteration_min0.15008456511853852
deviation-center-line_max0.5757002983581085
deviation-center-line_mean0.454529812252469
deviation-center-line_min0.1560618449994286
deviation-heading_max4.017017323927924
deviation-heading_mean2.7336240665548956
deviation-heading_median3.1642710107649967
deviation-heading_min0.5889369207616638
driven_any_max4.345201214166436
driven_any_mean3.2293487600205255
driven_any_median2.9555848639643254
driven_any_min2.6610240979870152
driven_lanedir_consec_max0.8951301907769306
driven_lanedir_consec_mean0.6258053880371894
driven_lanedir_consec_min0.31680048452467346
driven_lanedir_max0.8951301907769306
driven_lanedir_mean0.6258053880371894
driven_lanedir_median0.6456454384235769
driven_lanedir_min0.31680048452467346
get_duckie_state_max1.415532296864774e-06
get_duckie_state_mean1.3170064399931275e-06
get_duckie_state_median1.3518361335049083e-06
get_duckie_state_min1.1488211960979203e-06
get_robot_state_max0.0038284687535718727
get_robot_state_mean0.0037108092360325223
get_robot_state_median0.0037277429743956146
get_robot_state_min0.0035592822417669875
get_state_dump_max0.0048804376159605305
get_state_dump_mean0.004715274600332437
get_state_dump_median0.004795538042268526
get_state_dump_min0.004389584700832163
get_ui_image_max0.03469045499232427
get_ui_image_mean0.03017449608383628
get_ui_image_median0.030636838128705664
get_ui_image_min0.024733853085609516
in-drivable-lane_max17.600000000000136
in-drivable-lane_mean13.175000000000091
in-drivable-lane_min11.000000000000096
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.345201214166436, "get_ui_image": 0.028499250294491497, "step_physics": 0.09367320327053814, "survival_time": 24.30000000000021, "driven_lanedir": 0.7470577032937699, "get_state_dump": 0.0048804376159605305, "get_robot_state": 0.0038284687535718727, "sim_render-ego0": 0.004086736291340979, "get_duckie_state": 1.385960980362471e-06, "in-drivable-lane": 17.600000000000136, "deviation-heading": 2.461110848173687, "agent_compute-ego0": 0.013598258490435153, "complete-iteration": 0.16399388930146455, "set_robot_commands": 0.0022961545039496137, "deviation-center-line": 0.562154494064496, "driven_lanedir_consec": 0.7470577032937699, "sim_compute_sim_state": 0.010982459575488582, "sim_compute_performance-ego0": 0.002058738066185671}, "LF-norm-zigzag-000-ego0": {"driven_any": 2.914700075106957, "get_ui_image": 0.03469045499232427, "step_physics": 0.12590718269348145, "survival_time": 19.050000000000136, "driven_lanedir": 0.8951301907769306, "get_state_dump": 0.0047143547946870016, "get_robot_state": 0.003700453573496554, "sim_render-ego0": 0.003975440694399529, "get_duckie_state": 1.415532296864774e-06, "in-drivable-lane": 11.000000000000096, "deviation-heading": 3.8674311733563065, "agent_compute-ego0": 0.01316948898175624, "complete-iteration": 0.20109520470284667, "set_robot_commands": 0.0022118640819769257, "deviation-center-line": 0.5757002983581085, "driven_lanedir_consec": 0.8951301907769306, "sim_compute_sim_state": 0.01065012297705206, "sim_compute_performance-ego0": 0.0019911952043703087}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.996469652821694, "get_ui_image": 0.03277442596291983, "step_physics": 0.1466254758578475, "survival_time": 18.55000000000013, "driven_lanedir": 0.544233173553384, "get_state_dump": 0.004876721289850051, "get_robot_state": 0.0037550323752946753, "sim_render-ego0": 0.004024930538669709, "get_duckie_state": 1.3177112866473455e-06, "in-drivable-lane": 12.100000000000064, "deviation-heading": 4.017017323927924, "agent_compute-ego0": 0.013667882770620369, "complete-iteration": 0.22116704717759164, "set_robot_commands": 0.002325959103081816, "deviation-center-line": 0.5242026115878429, "driven_lanedir_consec": 0.544233173553384, "sim_compute_sim_state": 0.010980580442695209, "sim_compute_performance-ego0": 0.0020470317973885484}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6610240979870152, "get_ui_image": 0.024733853085609516, "step_physics": 0.09187213466685014, "survival_time": 14.000000000000064, "driven_lanedir": 0.31680048452467346, "get_state_dump": 0.004389584700832163, "get_robot_state": 0.0035592822417669875, "sim_render-ego0": 0.003725068000712005, "get_duckie_state": 1.1488211960979203e-06, "in-drivable-lane": 12.000000000000064, "deviation-heading": 0.5889369207616638, "agent_compute-ego0": 0.012329344223402572, "complete-iteration": 0.15008456511853852, "set_robot_commands": 0.002121987291926591, "deviation-center-line": 0.1560618449994286, "driven_lanedir_consec": 0.31680048452467346, "sim_compute_sim_state": 0.005366505252933163, "sim_compute_performance-ego0": 0.0019108290349885665}}
set_robot_commands_max0.002325959103081816
set_robot_commands_mean0.0022389912452337366
set_robot_commands_median0.0022540092929632695
set_robot_commands_min0.002121987291926591
sim_compute_performance-ego0_max0.002058738066185671
sim_compute_performance-ego0_mean0.0020019485257332737
sim_compute_performance-ego0_median0.0020191135008794285
sim_compute_performance-ego0_min0.0019108290349885665
sim_compute_sim_state_max0.010982459575488582
sim_compute_sim_state_mean0.009494917062042254
sim_compute_sim_state_median0.010815351709873631
sim_compute_sim_state_min0.005366505252933163
sim_render-ego0_max0.004086736291340979
sim_render-ego0_mean0.003953043881280555
sim_render-ego0_median0.004000185616534619
sim_render-ego0_min0.003725068000712005
simulation-passed1
step_physics_max0.1466254758578475
step_physics_mean0.1145194991221793
step_physics_median0.1097901929820098
step_physics_min0.09187213466685014
survival_time_max24.30000000000021
survival_time_mean18.975000000000136
survival_time_min14.000000000000064
No reset possible
5693811278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5693111278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:55
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5691511281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:02:25
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5690511284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5689311284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5688311284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:47
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5668311095Olga (Ge Ya) Xu 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-030:31:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.470932596202625
survival_time_median59.79999999999874
deviation-center-line_median2.745635570807507
in-drivable-lane_median17.37499999999952


other stats
agent_compute-ego0_max0.012602317938697429
agent_compute-ego0_mean0.012312332631940736
agent_compute-ego0_median0.01240001387473902
agent_compute-ego0_min0.011846984839587476
complete-iteration_max0.18460959121147724
complete-iteration_mean0.16644896814505147
complete-iteration_median0.1664714294282593
complete-iteration_min0.1482434225122101
deviation-center-line_max3.9396891835673298
deviation-center-line_mean2.8704192327055034
deviation-center-line_min2.0507166056396704
deviation-heading_max14.494973780386635
deviation-heading_mean12.72521441799103
deviation-heading_median13.847411862006702
deviation-heading_min8.71106016756408
driven_any_max12.224686121696076
driven_any_mean9.423669488359442
driven_any_median8.909470099923801
driven_any_min7.6510516318940915
driven_lanedir_consec_max10.932895134333078
driven_lanedir_consec_mean6.602135983271543
driven_lanedir_consec_min4.533783606347838
driven_lanedir_max10.932895134333078
driven_lanedir_mean7.184196456157842
driven_lanedir_median6.630277238851716
driven_lanedir_min4.54333621259486
get_duckie_state_max1.4313055414839847e-06
get_duckie_state_mean1.357276581371809e-06
get_duckie_state_median1.3898223697378196e-06
get_duckie_state_min1.2181560445276107e-06
get_robot_state_max0.003674131348964872
get_robot_state_mean0.0035497478941724538
get_robot_state_median0.003552297479444861
get_robot_state_min0.003420265268835222
get_state_dump_max0.004577904120770025
get_state_dump_mean0.0044343423995193525
get_state_dump_median0.004442017707012083
get_state_dump_min0.004275430063283222
get_ui_image_max0.0337642460237105
get_ui_image_mean0.028985873678273277
get_ui_image_median0.02865535090759152
get_ui_image_min0.02486854687419958
in-drivable-lane_max24.04999999999951
in-drivable-lane_mean16.612499999999528
in-drivable-lane_min7.649999999999565
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.224686121696076, "get_ui_image": 0.02818873601591061, "step_physics": 0.10004956339122256, "survival_time": 59.99999999999873, "driven_lanedir": 10.932895134333078, "get_state_dump": 0.004577904120770025, "get_robot_state": 0.003674131348964872, "sim_render-ego0": 0.003932326362095308, "get_duckie_state": 1.4313055414839847e-06, "in-drivable-lane": 7.649999999999565, "deviation-heading": 14.494973780386635, "agent_compute-ego0": 0.012602317938697429, "complete-iteration": 0.16637586971603763, "set_robot_commands": 0.002240494625653752, "deviation-center-line": 3.9396891835673298, "driven_lanedir_consec": 10.932895134333078, "sim_compute_sim_state": 0.008979733837931282, "sim_compute_performance-ego0": 0.0020469839030955853}, "LF-norm-zigzag-000-ego0": {"driven_any": 7.6510516318940915, "get_ui_image": 0.0337642460237105, "step_physics": 0.11075583127798, "survival_time": 59.59999999999875, "driven_lanedir": 4.54333621259486, "get_state_dump": 0.004450728687436662, "get_robot_state": 0.0035959808944356813, "sim_render-ego0": 0.003888333433331907, "get_duckie_state": 1.3729552711045553e-06, "in-drivable-lane": 22.749999999999275, "deviation-heading": 13.737414034311014, "agent_compute-ego0": 0.012583034136469188, "complete-iteration": 0.18460959121147724, "set_robot_commands": 0.0021773641687027065, "deviation-center-line": 2.685340150939774, "driven_lanedir_consec": 4.533783606347838, "sim_compute_sim_state": 0.011310429904806645, "sim_compute_performance-ego0": 0.0020031033697392093}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.9516352271261805, "get_ui_image": 0.02912196579927243, "step_physics": 0.10034540484410635, "survival_time": 40.199999999999854, "driven_lanedir": 6.371202044509692, "get_state_dump": 0.004275430063283222, "get_robot_state": 0.003420265268835222, "sim_render-ego0": 0.0035853984193031834, "get_duckie_state": 1.2181560445276107e-06, "in-drivable-lane": 11.999999999999766, "deviation-heading": 8.71106016756408, "agent_compute-ego0": 0.011846984839587476, "complete-iteration": 0.16656698914048093, "set_robot_commands": 0.00200335609246485, "deviation-center-line": 2.0507166056396704, "driven_lanedir_consec": 5.578704984870008, "sim_compute_sim_state": 0.010102265517904153, "sim_compute_performance-ego0": 0.0017887417574106537}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.867304972721422, "get_ui_image": 0.02486854687419958, "step_physics": 0.08949349226304434, "survival_time": 59.99999999999873, "driven_lanedir": 6.889352433193739, "get_state_dump": 0.004433306726587503, "get_robot_state": 0.00350861406445404, "sim_render-ego0": 0.0037263952425179337, "get_duckie_state": 1.406689468371084e-06, "in-drivable-lane": 24.04999999999951, "deviation-heading": 13.95740968970239, "agent_compute-ego0": 0.012216993613008853, "complete-iteration": 0.1482434225122101, "set_robot_commands": 0.0021178295570646695, "deviation-center-line": 2.80593099067524, "driven_lanedir_consec": 5.363160207535243, "sim_compute_sim_state": 0.005921353110663599, "sim_compute_performance-ego0": 0.0018753161736074632}}
set_robot_commands_max0.002240494625653752
set_robot_commands_mean0.0021347611109714944
set_robot_commands_median0.002147596862883688
set_robot_commands_min0.00200335609246485
sim_compute_performance-ego0_max0.0020469839030955853
sim_compute_performance-ego0_mean0.001928536300963228
sim_compute_performance-ego0_median0.001939209771673336
sim_compute_performance-ego0_min0.0017887417574106537
sim_compute_sim_state_max0.011310429904806645
sim_compute_sim_state_mean0.00907844559282642
sim_compute_sim_state_median0.009540999677917716
sim_compute_sim_state_min0.005921353110663599
sim_render-ego0_max0.003932326362095308
sim_render-ego0_mean0.003783113364312083
sim_render-ego0_median0.00380736433792492
sim_render-ego0_min0.0035853984193031834
simulation-passed1
step_physics_max0.11075583127798
step_physics_mean0.1001610729440883
step_physics_median0.10019748411766444
step_physics_min0.08949349226304434
survival_time_max59.99999999999873
survival_time_mean54.949999999999015
survival_time_min40.199999999999854
No reset possible
5666311283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:51
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5661011425Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-030:04:56
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5654411282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:07:23
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5647911285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:35
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5646711285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:34
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5645411291Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:35
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5641611291Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-030:00:40
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5637311450Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-testingLFt-simhost-erroryesnogpu-prod-030:01:50
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5628811427Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-030:03:10
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5581211420Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LFVI-sim-validationLFVIv-simtimeoutyesnogpu-prod-030:41:18
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible