Duckietown Challenges Home Challenges Submissions

Evaluator 4885

ID4885
evaluatornogpu-prod-02
ownerI don't have one 😀
machinenogpu-prod_5627c37a4d5b
processnogpu-prod-02_5627c37a4d5b
version6.2.5
first heard
last heard
statusinactive
# evaluating
# success20 53960
# timeout2 57459
# failed1 54601
# error
# aborted3 57239
# host-error33 53923
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5753910897Dishank Bansal 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-02----No reset possible
5745910919Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-020:23:40
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5741810933Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-020:05:48
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5735410953Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:12:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.401823289503607
survival_time_median14.700000000000074
deviation-center-line_median0.22846900722551847
in-drivable-lane_median12.17500000000006


other stats
agent_compute-ego0_max0.01325585698121346
agent_compute-ego0_mean0.012999186346053048
agent_compute-ego0_median0.01301594566487994
agent_compute-ego0_min0.012708997073238843
complete-iteration_max0.22158767905416368
complete-iteration_mean0.18414257802900735
complete-iteration_median0.17503193167477415
complete-iteration_min0.16491876971231748
deviation-center-line_max0.39301405185034827
deviation-center-line_mean0.23252517462086017
deviation-center-line_min0.08014863218205541
deviation-heading_max3.1305297590947827
deviation-heading_mean1.5159085888542942
deviation-heading_median1.1882808206275872
deviation-heading_min0.5565429550672207
driven_any_max6.322775699740005
driven_any_mean3.1624386335108876
driven_any_median2.8637013055749287
driven_any_min0.599576223153689
driven_lanedir_consec_max0.6876102657834492
driven_lanedir_consec_mean0.4159500981109645
driven_lanedir_consec_min0.1725435476531949
driven_lanedir_max0.6876102657834492
driven_lanedir_mean0.4159500981109645
driven_lanedir_median0.401823289503607
driven_lanedir_min0.1725435476531949
get_duckie_state_max1.6134079188516696e-06
get_duckie_state_mean1.484110013345169e-06
get_duckie_state_median1.5287165957353563e-06
get_duckie_state_min1.2655989430582942e-06
get_robot_state_max0.004053111844414832
get_robot_state_mean0.0038597911371105144
get_robot_state_median0.003826604210691682
get_robot_state_min0.003732844282643863
get_state_dump_max0.004986294163953538
get_state_dump_mean0.00482374744557614
get_state_dump_median0.004794479204249896
get_state_dump_min0.004719737209851229
get_ui_image_max0.03540285629562185
get_ui_image_mean0.03074980418312121
get_ui_image_median0.030105147969723273
get_ui_image_min0.02738606449741645
in-drivable-lane_max26.350000000000268
in-drivable-lane_mean13.312500000000096
in-drivable-lane_min2.5499999999999936
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.832276134657606, "get_ui_image": 0.02833083966006971, "step_physics": 0.09662061031550576, "survival_time": 14.550000000000072, "driven_lanedir": 0.5088751412116059, "get_state_dump": 0.004779810774816226, "get_robot_state": 0.003854919786322607, "sim_render-ego0": 0.0039312504742243515, "get_duckie_state": 1.6134079188516696e-06, "in-drivable-lane": 11.350000000000044, "deviation-heading": 1.6102360458891942, "agent_compute-ego0": 0.012708997073238843, "complete-iteration": 0.16491876971231748, "set_robot_commands": 0.0022753174990823824, "deviation-center-line": 0.33888860510868013, "driven_lanedir_consec": 0.5088751412116059, "sim_compute_sim_state": 0.01021448063523802, "sim_compute_performance-ego0": 0.0021059113006069237}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.599576223153689, "get_ui_image": 0.03540285629562185, "step_physics": 0.14543636841110036, "survival_time": 3.899999999999994, "driven_lanedir": 0.1725435476531949, "get_state_dump": 0.004719737209851229, "get_robot_state": 0.003798288635060757, "sim_render-ego0": 0.004061677787877336, "get_duckie_state": 1.4637089982817444e-06, "in-drivable-lane": 2.5499999999999936, "deviation-heading": 0.7663255953659801, "agent_compute-ego0": 0.013204659087748469, "complete-iteration": 0.22158767905416368, "set_robot_commands": 0.00238416768327544, "deviation-center-line": 0.08014863218205541, "driven_lanedir_consec": 0.1725435476531949, "sim_compute_sim_state": 0.010381351543378228, "sim_compute_performance-ego0": 0.0020987414106538025}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.322775699740005, "get_ui_image": 0.03187945627937683, "step_physics": 0.1062468103707408, "survival_time": 31.25000000000031, "driven_lanedir": 0.6876102657834492, "get_state_dump": 0.004809147633683567, "get_robot_state": 0.003732844282643863, "sim_render-ego0": 0.003941501291415181, "get_duckie_state": 1.2655989430582942e-06, "in-drivable-lane": 26.350000000000268, "deviation-heading": 3.1305297590947827, "agent_compute-ego0": 0.012827232242011415, "complete-iteration": 0.17883294954086645, "set_robot_commands": 0.002167648019882056, "deviation-center-line": 0.39301405185034827, "driven_lanedir_consec": 0.6876102657834492, "sim_compute_sim_state": 0.011135010292735724, "sim_compute_performance-ego0": 0.0020083303268725117}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.895126476492251, "get_ui_image": 0.02738606449741645, "step_physics": 0.10691355618854496, "survival_time": 14.850000000000076, "driven_lanedir": 0.2947714377956081, "get_state_dump": 0.004986294163953538, "get_robot_state": 0.004053111844414832, "sim_render-ego0": 0.004061634908586541, "get_duckie_state": 1.593724193188968e-06, "in-drivable-lane": 13.000000000000076, "deviation-heading": 0.5565429550672207, "agent_compute-ego0": 0.01325585698121346, "complete-iteration": 0.17123091380868183, "set_robot_commands": 0.0022979542713037274, "deviation-center-line": 0.1180494093423568, "driven_lanedir_consec": 0.2947714377956081, "sim_compute_sim_state": 0.0060733500743072304, "sim_compute_performance-ego0": 0.0021007989076960005}}
set_robot_commands_max0.00238416768327544
set_robot_commands_mean0.0022812718683859015
set_robot_commands_median0.0022866358851930547
set_robot_commands_min0.002167648019882056
sim_compute_performance-ego0_max0.0021059113006069237
sim_compute_performance-ego0_mean0.0020784454864573097
sim_compute_performance-ego0_median0.0020997701591749015
sim_compute_performance-ego0_min0.0020083303268725117
sim_compute_sim_state_max0.011135010292735724
sim_compute_sim_state_mean0.0094510481364148
sim_compute_sim_state_median0.010297916089308124
sim_compute_sim_state_min0.0060733500743072304
sim_render-ego0_max0.004061677787877336
sim_render-ego0_mean0.003999016115525852
sim_render-ego0_median0.0040015681000008605
sim_render-ego0_min0.0039312504742243515
simulation-passed1
step_physics_max0.14543636841110036
step_physics_mean0.11380433632147298
step_physics_median0.10658018327964287
step_physics_min0.09662061031550576
survival_time_max31.25000000000031
survival_time_mean16.137500000000113
survival_time_min3.899999999999994
No reset possible
5727810980Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:17:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.6288269768663497
survival_time_median22.525000000000183
deviation-center-line_median0.6923648568302725
in-drivable-lane_median6.9000000000000865


other stats
agent_compute-ego0_max0.013357819565924677
agent_compute-ego0_mean0.01298508206487207
agent_compute-ego0_median0.012926484602604867
agent_compute-ego0_min0.01272953948835387
complete-iteration_max0.19445973867976993
complete-iteration_mean0.17324982604728073
complete-iteration_median0.17447832609359132
complete-iteration_min0.14958291332217027
deviation-center-line_max1.282878233648001
deviation-center-line_mean0.7913184385074573
deviation-center-line_min0.497665806721283
deviation-heading_max3.929232334375037
deviation-heading_mean2.180227893863551
deviation-heading_median1.8202211997228697
deviation-heading_min1.151236841633429
driven_any_max11.354896295533914
driven_any_mean5.479527825845192
driven_any_median4.08150960392847
driven_any_min2.4001957999899153
driven_lanedir_consec_max3.9115905248000553
driven_lanedir_consec_mean2.6062691359056314
driven_lanedir_consec_min1.2558320650897714
driven_lanedir_max3.9115905248000553
driven_lanedir_mean2.6062691359056314
driven_lanedir_median2.6288269768663497
driven_lanedir_min1.2558320650897714
get_duckie_state_max1.5952204929963322e-06
get_duckie_state_mean1.3992373553068e-06
get_duckie_state_median1.3861099891692517e-06
get_duckie_state_min1.2295089498923642e-06
get_robot_state_max0.003824984299317571
get_robot_state_mean0.003689238008592374
get_robot_state_median0.00369022793305433
get_robot_state_min0.003551511868943263
get_state_dump_max0.004655329611378844
get_state_dump_mean0.0045934525642462865
get_state_dump_median0.004596363037165383
get_state_dump_min0.004525754571275535
get_ui_image_max0.034914211667050526
get_ui_image_mean0.02975807617074064
get_ui_image_median0.02943347535797682
get_ui_image_min0.02525114229995839
in-drivable-lane_max42.4499999999986
in-drivable-lane_mean15.312499999999712
in-drivable-lane_min5.000000000000071
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.77781113356002, "get_ui_image": 0.027360871547961053, "step_physics": 0.09728707703015275, "survival_time": 26.150000000000237, "driven_lanedir": 3.9115905248000553, "get_state_dump": 0.004626682241454379, "get_robot_state": 0.003824984299317571, "sim_render-ego0": 0.003931186581385955, "get_duckie_state": 1.5952204929963322e-06, "in-drivable-lane": 5.000000000000071, "deviation-heading": 2.3366424601582874, "agent_compute-ego0": 0.01281150060755606, "complete-iteration": 0.1646753181028002, "set_robot_commands": 0.0022347783314362736, "deviation-center-line": 0.8672092702232062, "driven_lanedir_consec": 3.9115905248000553, "sim_compute_sim_state": 0.01039194105235675, "sim_compute_performance-ego0": 0.002106217482617793}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.354896295533914, "get_ui_image": 0.034914211667050526, "step_physics": 0.1149135211623777, "survival_time": 59.99999999999873, "driven_lanedir": 3.152541608690435, "get_state_dump": 0.004655329611378844, "get_robot_state": 0.003704218145810396, "sim_render-ego0": 0.003993090543024347, "get_duckie_state": 1.4845080220828343e-06, "in-drivable-lane": 42.4499999999986, "deviation-heading": 3.929232334375037, "agent_compute-ego0": 0.013357819565924677, "complete-iteration": 0.19445973867976993, "set_robot_commands": 0.002221129319749208, "deviation-center-line": 1.282878233648001, "driven_lanedir_consec": 3.152541608690435, "sim_compute_sim_state": 0.014537139300204236, "sim_compute_performance-ego0": 0.002069228892520901}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.385208074296922, "get_ui_image": 0.031506079167992584, "step_physics": 0.11298507310469735, "survival_time": 18.900000000000137, "driven_lanedir": 2.1051123450422637, "get_state_dump": 0.004525754571275535, "get_robot_state": 0.003676237720298264, "sim_render-ego0": 0.003824475572732005, "get_duckie_state": 1.2877119562556688e-06, "in-drivable-lane": 7.300000000000104, "deviation-heading": 1.3037999392874513, "agent_compute-ego0": 0.013041468597653674, "complete-iteration": 0.1842813340843825, "set_robot_commands": 0.0021662296909141037, "deviation-center-line": 0.497665806721283, "driven_lanedir_consec": 2.1051123450422637, "sim_compute_sim_state": 0.010492089555886302, "sim_compute_performance-ego0": 0.00197620630893355}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.4001957999899153, "get_ui_image": 0.02525114229995839, "step_physics": 0.09013473378480788, "survival_time": 13.65000000000006, "driven_lanedir": 1.2558320650897714, "get_state_dump": 0.0045660438328763864, "get_robot_state": 0.003551511868943263, "sim_render-ego0": 0.0037814587572195233, "get_duckie_state": 1.2295089498923642e-06, "in-drivable-lane": 6.500000000000069, "deviation-heading": 1.151236841633429, "agent_compute-ego0": 0.01272953948835387, "complete-iteration": 0.14958291332217027, "set_robot_commands": 0.002145567949670945, "deviation-center-line": 0.5175204434373389, "driven_lanedir_consec": 1.2558320650897714, "sim_compute_sim_state": 0.005442375684306569, "sim_compute_performance-ego0": 0.00189836268877461}}
set_robot_commands_max0.0022347783314362736
set_robot_commands_mean0.002191926322942633
set_robot_commands_median0.002193679505331656
set_robot_commands_min0.002145567949670945
sim_compute_performance-ego0_max0.002106217482617793
sim_compute_performance-ego0_mean0.0020125038432117137
sim_compute_performance-ego0_median0.0020227176007272256
sim_compute_performance-ego0_min0.00189836268877461
sim_compute_sim_state_max0.014537139300204236
sim_compute_sim_state_mean0.010215886398188464
sim_compute_sim_state_median0.010442015304121526
sim_compute_sim_state_min0.005442375684306569
sim_render-ego0_max0.003993090543024347
sim_render-ego0_mean0.0038825528635904574
sim_render-ego0_median0.00387783107705898
sim_render-ego0_min0.0037814587572195233
simulation-passed1
step_physics_max0.1149135211623777
step_physics_mean0.10383010127050891
step_physics_median0.10513607506742506
step_physics_min0.09013473378480788
survival_time_max59.99999999999873
survival_time_mean29.67499999999979
survival_time_min13.65000000000006
No reset possible
5725611012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-020:01:49
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5723911010Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-020:01:39
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5721510992Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:10:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.3845425386708304
survival_time_median15.22500000000008
deviation-center-line_median0.2648344860591804
in-drivable-lane_median11.925000000000056


other stats
agent_compute-ego0_max0.013919389958158977
agent_compute-ego0_mean0.013056401111134446
agent_compute-ego0_median0.012895988344909876
agent_compute-ego0_min0.012514237796559054
complete-iteration_max0.21078035130220305
complete-iteration_mean0.17778886564530305
complete-iteration_median0.17310462618821346
complete-iteration_min0.1541658589025823
deviation-center-line_max0.4509025014834449
deviation-center-line_mean0.2674519909718316
deviation-center-line_min0.08923649028552058
deviation-heading_max4.1882431419970505
deviation-heading_mean2.106012479342495
deviation-heading_median1.8385862029005848
deviation-heading_min0.5586343695717592
driven_any_max5.891841494328425
driven_any_mean3.0188402978437043
driven_any_median2.784983727618747
driven_any_min0.6135522418088981
driven_lanedir_consec_max0.6893962510036289
driven_lanedir_consec_mean0.40834690024670817
driven_lanedir_consec_min0.17490627264154313
driven_lanedir_max0.6893962510036289
driven_lanedir_mean0.40834690024670817
driven_lanedir_median0.3845425386708304
driven_lanedir_min0.17490627264154313
get_duckie_state_max1.526109262365074e-06
get_duckie_state_mean1.376735720762513e-06
get_duckie_state_median1.3408736142742754e-06
get_duckie_state_min1.2990863921364288e-06
get_robot_state_max0.003878163062815889
get_robot_state_mean0.003680136042321872
get_robot_state_median0.003625770176158232
get_robot_state_min0.0035908407541551356
get_state_dump_max0.005162393797230989
get_state_dump_mean0.004784692453248364
get_state_dump_median0.004718818875775275
get_state_dump_min0.004538738264211918
get_ui_image_max0.03470522656160242
get_ui_image_mean0.03036865833652753
get_ui_image_median0.03027857460077041
get_ui_image_min0.026212257582966875
in-drivable-lane_max24.950000000000248
in-drivable-lane_mean12.850000000000088
in-drivable-lane_min2.5999999999999925
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.3885417652824357, "get_ui_image": 0.027296568395392212, "step_physics": 0.09149256106400236, "survival_time": 14.100000000000064, "driven_lanedir": 0.5047105161634341, "get_state_dump": 0.004538738264211918, "get_robot_state": 0.0035908407541551356, "sim_render-ego0": 0.003852101181084192, "get_duckie_state": 1.2990863921364288e-06, "in-drivable-lane": 9.250000000000014, "deviation-heading": 2.775122986687395, "agent_compute-ego0": 0.012801198150580848, "complete-iteration": 0.1572751788284248, "set_robot_commands": 0.0022401253663187734, "deviation-center-line": 0.4244228118777055, "driven_lanedir_consec": 0.5047105161634341, "sim_compute_sim_state": 0.009424940857364936, "sim_compute_performance-ego0": 0.001956979293284062}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.6135522418088981, "get_ui_image": 0.03470522656160242, "step_physics": 0.1369651626138126, "survival_time": 4.199999999999993, "driven_lanedir": 0.17490627264154313, "get_state_dump": 0.004749009188483743, "get_robot_state": 0.0036135112538057215, "sim_render-ego0": 0.003883072909186868, "get_duckie_state": 1.3239243451286764e-06, "in-drivable-lane": 2.5999999999999925, "deviation-heading": 0.9020494191137745, "agent_compute-ego0": 0.012514237796559054, "complete-iteration": 0.21078035130220305, "set_robot_commands": 0.0021451529334573184, "deviation-center-line": 0.08923649028552058, "driven_lanedir_consec": 0.17490627264154313, "sim_compute_sim_state": 0.010131154340856216, "sim_compute_performance-ego0": 0.001990500618429745}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.891841494328425, "get_ui_image": 0.03326058080614861, "step_physics": 0.11076265256761927, "survival_time": 31.000000000000306, "driven_lanedir": 0.6893962510036289, "get_state_dump": 0.005162393797230989, "get_robot_state": 0.003878163062815889, "sim_render-ego0": 0.004230085778351567, "get_duckie_state": 1.526109262365074e-06, "in-drivable-lane": 24.950000000000248, "deviation-heading": 4.1882431419970505, "agent_compute-ego0": 0.013919389958158977, "complete-iteration": 0.18893407354800212, "set_robot_commands": 0.002374742726005026, "deviation-center-line": 0.4509025014834449, "driven_lanedir_consec": 0.6893962510036289, "sim_compute_sim_state": 0.013068578477451, "sim_compute_performance-ego0": 0.002176529735374758}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.181425689955058, "get_ui_image": 0.026212257582966875, "step_physics": 0.09352731632023324, "survival_time": 16.350000000000097, "driven_lanedir": 0.26437456117822666, "get_state_dump": 0.004688628563066808, "get_robot_state": 0.003638029098510742, "sim_render-ego0": 0.003791512512579197, "get_duckie_state": 1.3578228834198745e-06, "in-drivable-lane": 14.600000000000096, "deviation-heading": 0.5586343695717592, "agent_compute-ego0": 0.012990778539238908, "complete-iteration": 0.1541658589025823, "set_robot_commands": 0.0022201240062713623, "deviation-center-line": 0.10524616024065536, "driven_lanedir_consec": 0.26437456117822666, "sim_compute_sim_state": 0.005084729049263931, "sim_compute_performance-ego0": 0.0019269469307690132}}
set_robot_commands_max0.002374742726005026
set_robot_commands_mean0.00224503625801312
set_robot_commands_median0.0022301246862950676
set_robot_commands_min0.0021451529334573184
sim_compute_performance-ego0_max0.002176529735374758
sim_compute_performance-ego0_mean0.002012739144464395
sim_compute_performance-ego0_median0.0019737399558569036
sim_compute_performance-ego0_min0.0019269469307690132
sim_compute_sim_state_max0.013068578477451
sim_compute_sim_state_mean0.00942735068123402
sim_compute_sim_state_median0.009778047599110575
sim_compute_sim_state_min0.005084729049263931
sim_render-ego0_max0.004230085778351567
sim_render-ego0_mean0.003939193095300456
sim_render-ego0_median0.00386758704513553
sim_render-ego0_min0.003791512512579197
simulation-passed1
step_physics_max0.1369651626138126
step_physics_mean0.10818692314141688
step_physics_median0.10214498444392624
step_physics_min0.09149256106400236
survival_time_max31.000000000000306
survival_time_mean16.412500000000115
survival_time_min4.199999999999993
No reset possible
5720012729Eric Lutemplate-randomaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:05:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5040495128145347
survival_time_median4.6499999999999915
deviation-center-line_median0.10620228737371736
in-drivable-lane_median2.674999999999992


other stats
agent_compute-ego0_max0.011935783757103814
agent_compute-ego0_mean0.010826450133306536
agent_compute-ego0_median0.010651428422628685
agent_compute-ego0_min0.01006715993086497
complete-iteration_max0.20721736550331116
complete-iteration_mean0.18615014036463676
complete-iteration_median0.18984321117335515
complete-iteration_min0.1576967736085256
deviation-center-line_max0.16455763165774434
deviation-center-line_mean0.1092777488823554
deviation-center-line_min0.06014878912424249
deviation-heading_max1.819139548495243
deviation-heading_mean0.860079214882594
deviation-heading_median0.6517793617099584
deviation-heading_min0.31761858761521594
driven_any_max3.078535752374254
driven_any_mean1.8526223552312917
driven_any_median1.6219947547238958
driven_any_min1.0879641591031215
driven_lanedir_consec_max0.6643282900394009
driven_lanedir_consec_mean0.4884432809313369
driven_lanedir_consec_min0.28134580805687714
driven_lanedir_max0.6643282900394009
driven_lanedir_mean0.4924956463812869
driven_lanedir_median0.5121542437144349
driven_lanedir_min0.28134580805687714
get_duckie_state_max1.4395653447018394e-06
get_duckie_state_mean1.3548227927228884e-06
get_duckie_state_median1.347871218326587e-06
get_duckie_state_min1.2839833895365398e-06
get_robot_state_max0.003909899128807915
get_robot_state_mean0.003737985317135764
get_robot_state_median0.0037442885654997418
get_robot_state_min0.0035534650087356567
get_state_dump_max0.0048283735911051435
get_state_dump_mean0.004659818027491767
get_state_dump_median0.004702840326379536
get_state_dump_min0.004405217866102855
get_ui_image_max0.03511952835580577
get_ui_image_mean0.03065365481747423
get_ui_image_median0.03043932460363739
get_ui_image_min0.026616441706816357
in-drivable-lane_max6.549999999999982
in-drivable-lane_mean3.3499999999999894
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.078535752374254, "get_ui_image": 0.028313876707342606, "step_physics": 0.10927492304693294, "survival_time": 7.84999999999998, "driven_lanedir": 0.28134580805687714, "get_state_dump": 0.004787621618826178, "get_robot_state": 0.003723574590079392, "sim_render-ego0": 0.003981191900712025, "get_duckie_state": 1.4395653447018394e-06, "in-drivable-lane": 6.549999999999982, "deviation-heading": 0.891006057417327, "agent_compute-ego0": 0.010900607591942895, "complete-iteration": 0.17518995834302298, "set_robot_commands": 0.002239957640442667, "deviation-center-line": 0.13094886067496936, "driven_lanedir_consec": 0.28134580805687714, "sim_compute_sim_state": 0.009864754314664045, "sim_compute_performance-ego0": 0.0020171162448351897}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5806581102909054, "get_ui_image": 0.03511952835580577, "step_physics": 0.13316065591314566, "survival_time": 4.549999999999992, "driven_lanedir": 0.3891269085945981, "get_state_dump": 0.004618059033932893, "get_robot_state": 0.003765002540920091, "sim_render-ego0": 0.00385206937789917, "get_duckie_state": 1.3579492983610737e-06, "in-drivable-lane": 2.449999999999993, "deviation-heading": 1.819139548495243, "agent_compute-ego0": 0.010402249253314474, "complete-iteration": 0.20449646400368732, "set_robot_commands": 0.0022075072578761888, "deviation-center-line": 0.16455763165774434, "driven_lanedir_consec": 0.37291744679479777, "sim_compute_sim_state": 0.009297953999560812, "sim_compute_performance-ego0": 0.001990696658258853}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0879641591031215, "get_ui_image": 0.032564772499932185, "step_physics": 0.13738200730747646, "survival_time": 3.5499999999999954, "driven_lanedir": 0.6643282900394009, "get_state_dump": 0.0048283735911051435, "get_robot_state": 0.003909899128807915, "sim_render-ego0": 0.004147234890196059, "get_duckie_state": 1.3377931382921006e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.41255266600258983, "agent_compute-ego0": 0.011935783757103814, "complete-iteration": 0.20721736550331116, "set_robot_commands": 0.0023026929961310495, "deviation-center-line": 0.08145571407246535, "driven_lanedir_consec": 0.6643282900394009, "sim_compute_sim_state": 0.0079059898853302, "sim_compute_performance-ego0": 0.002155125141143799}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6633313991568863, "get_ui_image": 0.026616441706816357, "step_physics": 0.10028726359208424, "survival_time": 4.749999999999991, "driven_lanedir": 0.6351815788342716, "get_state_dump": 0.004405217866102855, "get_robot_state": 0.0035534650087356567, "sim_render-ego0": 0.00379219651222229, "get_duckie_state": 1.2839833895365398e-06, "in-drivable-lane": 2.89999999999999, "deviation-heading": 0.31761858761521594, "agent_compute-ego0": 0.01006715993086497, "complete-iteration": 0.1576967736085256, "set_robot_commands": 0.002184132734934489, "deviation-center-line": 0.06014878912424249, "driven_lanedir_consec": 0.6351815788342716, "sim_compute_sim_state": 0.004888524611790975, "sim_compute_performance-ego0": 0.0018248260021209717}}
set_robot_commands_max0.0023026929961310495
set_robot_commands_mean0.002233572657346098
set_robot_commands_median0.002223732449159428
set_robot_commands_min0.002184132734934489
sim_compute_performance-ego0_max0.002155125141143799
sim_compute_performance-ego0_mean0.0019969410115897035
sim_compute_performance-ego0_median0.0020039064515470213
sim_compute_performance-ego0_min0.0018248260021209717
sim_compute_sim_state_max0.009864754314664045
sim_compute_sim_state_mean0.007989305702836508
sim_compute_sim_state_median0.008601971942445507
sim_compute_sim_state_min0.004888524611790975
sim_render-ego0_max0.004147234890196059
sim_render-ego0_mean0.003943173170257386
sim_render-ego0_median0.003916630639305597
sim_render-ego0_min0.00379219651222229
simulation-passed1
step_physics_max0.13738200730747646
step_physics_mean0.1200262124649098
step_physics_median0.1212177894800393
step_physics_min0.10028726359208424
survival_time_max7.84999999999998
survival_time_mean5.17499999999999
survival_time_min3.5499999999999954
No reset possible
5719411313Moustafa Elarabitemplate-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-020:01:06
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5718911312Moustafa Elarabitemplate-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-020:00:46
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5717011278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:01:57
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5706911029Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:18:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.261860996748414
survival_time_median28.625000000000075
deviation-center-line_median1.1698795725551423
in-drivable-lane_median12.299999999999978


other stats
agent_compute-ego0_max0.013355210461194002
agent_compute-ego0_mean0.012681622669251998
agent_compute-ego0_median0.012620632709569986
agent_compute-ego0_min0.012130014796674023
complete-iteration_max0.2042589181586157
complete-iteration_mean0.17368956151982318
complete-iteration_median0.16747168806757154
complete-iteration_min0.155555951785534
deviation-center-line_max2.3896420161537333
deviation-center-line_mean1.2895633282210046
deviation-center-line_min0.42885215162
deviation-heading_max5.653120902988238
deviation-heading_mean3.1895811490018224
deviation-heading_median3.00056885882794
deviation-heading_min1.104065975363173
driven_any_max12.373410503958914
driven_any_mean6.424846121153673
driven_any_median5.457016471100411
driven_any_min2.411941038454958
driven_lanedir_consec_max5.3315605405447375
driven_lanedir_consec_mean3.4647202280629674
driven_lanedir_consec_min2.0035983782103033
driven_lanedir_max6.109577961794743
driven_lanedir_mean3.659224583375469
driven_lanedir_median3.261860996748414
driven_lanedir_min2.0035983782103033
get_duckie_state_max1.3933435592361276e-06
get_duckie_state_mean1.3578449047894188e-06
get_duckie_state_median1.352653191155394e-06
get_duckie_state_min1.3327296776107594e-06
get_robot_state_max0.003685370571508098
get_robot_state_mean0.0035704572994996703
get_robot_state_median0.0035531517195470435
get_robot_state_min0.0034901551873964955
get_state_dump_max0.004526811860813387
get_state_dump_mean0.004485203492219242
get_state_dump_median0.004482948848032944
get_state_dump_min0.004448104411997693
get_ui_image_max0.03494545900368992
get_ui_image_mean0.029786850388046272
get_ui_image_median0.02923226569213615
get_ui_image_min0.02573741116422288
in-drivable-lane_max29.699999999999783
in-drivable-lane_mean14.462499999999944
in-drivable-lane_min3.5500000000000504
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.411941038454958, "get_ui_image": 0.02739813572553627, "step_physics": 0.09084023087650198, "survival_time": 13.100000000000051, "driven_lanedir": 2.0317301089304833, "get_state_dump": 0.004526811860813387, "get_robot_state": 0.0034901551873964955, "sim_render-ego0": 0.003751268858238771, "get_duckie_state": 1.3933435592361276e-06, "in-drivable-lane": 3.5500000000000504, "deviation-heading": 1.104065975363173, "agent_compute-ego0": 0.012130014796674023, "complete-iteration": 0.15569902282250697, "set_robot_commands": 0.0021289392115951943, "deviation-center-line": 0.42885215162, "driven_lanedir_consec": 2.0317301089304833, "sim_compute_sim_state": 0.00950086524731306, "sim_compute_performance-ego0": 0.0018528604688753192}, "LF-norm-zigzag-000-ego0": {"driven_any": 3.3507014189928293, "get_ui_image": 0.03494545900368992, "step_physics": 0.12830858170231688, "survival_time": 19.700000000000145, "driven_lanedir": 2.0035983782103033, "get_state_dump": 0.00448835469499419, "get_robot_state": 0.00358025212831135, "sim_render-ego0": 0.003949509994893134, "get_duckie_state": 1.3327296776107594e-06, "in-drivable-lane": 9.20000000000013, "deviation-heading": 1.8483144427068845, "agent_compute-ego0": 0.013355210461194002, "complete-iteration": 0.2042589181586157, "set_robot_commands": 0.0021556129938439476, "deviation-center-line": 0.754110232546526, "driven_lanedir_consec": 2.0035983782103033, "sim_compute_sim_state": 0.011421772196323055, "sim_compute_performance-ego0": 0.0019714373576490184}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.373410503958914, "get_ui_image": 0.031066395658736028, "step_physics": 0.1071756356562504, "survival_time": 59.99999999999873, "driven_lanedir": 6.109577961794743, "get_state_dump": 0.004477543001071698, "get_robot_state": 0.003685370571508098, "sim_render-ego0": 0.0039242544737981815, "get_duckie_state": 1.3397893341852168e-06, "in-drivable-lane": 29.699999999999783, "deviation-heading": 5.653120902988238, "agent_compute-ego0": 0.01259853023970554, "complete-iteration": 0.1792443533126361, "set_robot_commands": 0.002209984988197498, "deviation-center-line": 2.3896420161537333, "driven_lanedir_consec": 5.3315605405447375, "sim_compute_sim_state": 0.01198055206190835, "sim_compute_performance-ego0": 0.0020426784725014516}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.563331523207993, "get_ui_image": 0.02573741116422288, "step_physics": 0.0951665516229386, "survival_time": 37.55, "driven_lanedir": 4.491991884566345, "get_state_dump": 0.004448104411997693, "get_robot_state": 0.003526051310782737, "sim_render-ego0": 0.003788978495496385, "get_duckie_state": 1.3655170481255717e-06, "in-drivable-lane": 15.399999999999824, "deviation-heading": 4.152823274948995, "agent_compute-ego0": 0.012642735179434431, "complete-iteration": 0.155555951785534, "set_robot_commands": 0.002148841289763755, "deviation-center-line": 1.585648912563759, "driven_lanedir_consec": 4.491991884566345, "sim_compute_sim_state": 0.006127971600978933, "sim_compute_performance-ego0": 0.001888268805564718}}
set_robot_commands_max0.002209984988197498
set_robot_commands_mean0.002160844620850098
set_robot_commands_median0.0021522271418038513
set_robot_commands_min0.0021289392115951943
sim_compute_performance-ego0_max0.0020426784725014516
sim_compute_performance-ego0_mean0.0019388112761476268
sim_compute_performance-ego0_median0.001929853081606868
sim_compute_performance-ego0_min0.0018528604688753192
sim_compute_sim_state_max0.01198055206190835
sim_compute_sim_state_mean0.009757790276630847
sim_compute_sim_state_median0.01046131872181806
sim_compute_sim_state_min0.006127971600978933
sim_render-ego0_max0.003949509994893134
sim_render-ego0_mean0.003853502955606618
sim_render-ego0_median0.003856616484647283
sim_render-ego0_min0.003751268858238771
simulation-passed1
step_physics_max0.12830858170231688
step_physics_mean0.10537274996450197
step_physics_median0.1011710936395945
step_physics_min0.09084023087650198
survival_time_max59.99999999999873
survival_time_mean32.58749999999973
survival_time_min13.100000000000051
No reset possible
5691911063Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:26:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.712272325317809
survival_time_median59.99999999999873
deviation-center-line_median2.679496011513158
in-drivable-lane_median1.0499999999999965


other stats
agent_compute-ego0_max0.043152770610770795
agent_compute-ego0_mean0.033422078811307925
agent_compute-ego0_median0.03605264251575581
agent_compute-ego0_min0.018430259602949285
complete-iteration_max0.23200635476545853
complete-iteration_mean0.1915725231549641
complete-iteration_median0.1840142307432367
complete-iteration_min0.1662552763679244
deviation-center-line_max3.7382954733556857
deviation-center-line_mean2.3254306326097183
deviation-center-line_min0.2044350340568711
deviation-heading_max10.334807819807338
deviation-heading_mean5.954175123880642
deviation-heading_median5.767274237451526
deviation-heading_min1.9473442008121824
driven_any_max8.339782680818251
driven_any_mean5.961913350088906
driven_any_median7.601782295795141
driven_any_min0.3043061279470948
driven_lanedir_consec_max7.7503999929579
driven_lanedir_consec_mean4.827755060223082
driven_lanedir_consec_min0.13607559729881125
driven_lanedir_max7.7503999929579
driven_lanedir_mean5.44350033080474
driven_lanedir_median6.943762866481125
driven_lanedir_min0.13607559729881125
get_duckie_state_max1.323510963255718e-06
get_duckie_state_mean1.2680696552187324e-06
get_duckie_state_median1.2518464278221924e-06
get_duckie_state_min1.2450748019748264e-06
get_robot_state_max0.003718058334401406
get_robot_state_mean0.003556864988659588
get_robot_state_median0.003531210229954652
get_robot_state_min0.003446981160327642
get_state_dump_max0.004781503860003545
get_state_dump_mean0.004673602162955981
get_state_dump_median0.004651470942660037
get_state_dump_min0.004609962906500306
get_ui_image_max0.03281279043717818
get_ui_image_mean0.02910911811759746
get_ui_image_median0.02928128329840032
get_ui_image_min0.025061115436411024
in-drivable-lane_max14.199999999999545
in-drivable-lane_mean4.074999999999884
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.339782680818251, "get_ui_image": 0.027232413486477536, "step_physics": 0.09144676337134929, "survival_time": 59.99999999999873, "driven_lanedir": 6.65453789435043, "get_state_dump": 0.004613436826758341, "get_robot_state": 0.0035069848377440593, "sim_render-ego0": 0.003805899401687762, "get_duckie_state": 1.256213795632546e-06, "in-drivable-lane": 14.199999999999545, "deviation-heading": 4.826954354183848, "agent_compute-ego0": 0.03669019702272947, "complete-iteration": 0.1812518190880997, "set_robot_commands": 0.0021343562724092024, "deviation-center-line": 2.42740201603804, "driven_lanedir_consec": 4.191556812023798, "sim_compute_sim_state": 0.009838458401872951, "sim_compute_performance-ego0": 0.0018953371802337164}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.3043061279470948, "get_ui_image": 0.03281279043717818, "step_physics": 0.13033556938171387, "survival_time": 4.899999999999991, "driven_lanedir": 0.13607559729881125, "get_state_dump": 0.004609962906500306, "get_robot_state": 0.003446981160327642, "sim_render-ego0": 0.003735390576449307, "get_duckie_state": 1.2450748019748264e-06, "in-drivable-lane": 2.0999999999999925, "deviation-heading": 1.9473442008121824, "agent_compute-ego0": 0.043152770610770795, "complete-iteration": 0.23200635476545853, "set_robot_commands": 0.002153755438448203, "deviation-center-line": 0.2044350340568711, "driven_lanedir_consec": 0.13607559729881125, "sim_compute_sim_state": 0.009831869240963098, "sim_compute_performance-ego0": 0.001842289259939483}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.388213782531776, "get_ui_image": 0.031330153110323104, "step_physics": 0.10779597380079892, "survival_time": 59.99999999999873, "driven_lanedir": 7.232987838611819, "get_state_dump": 0.004781503860003545, "get_robot_state": 0.003718058334401406, "sim_render-ego0": 0.003913429754163502, "get_duckie_state": 1.323510963255718e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.334807819807338, "agent_compute-ego0": 0.018430259602949285, "complete-iteration": 0.1867766423983737, "set_robot_commands": 0.002186953475532881, "deviation-center-line": 3.7382954733556857, "driven_lanedir_consec": 7.232987838611819, "sim_compute_sim_state": 0.012508262305533658, "sim_compute_performance-ego0": 0.002019168336028163}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.815350809058506, "get_ui_image": 0.025061115436411024, "step_physics": 0.08372834342206943, "survival_time": 59.99999999999873, "driven_lanedir": 7.7503999929579, "get_state_dump": 0.004689505058561734, "get_robot_state": 0.0035554356221652447, "sim_render-ego0": 0.0037316497815439447, "get_duckie_state": 1.2474790600118392e-06, "in-drivable-lane": 0.0, "deviation-heading": 6.707594120719202, "agent_compute-ego0": 0.03541508800878215, "complete-iteration": 0.1662552763679244, "set_robot_commands": 0.00213767706007882, "deviation-center-line": 2.9315900069882757, "driven_lanedir_consec": 7.7503999929579, "sim_compute_sim_state": 0.0060176317340428385, "sim_compute_performance-ego0": 0.001832864762940673}}
set_robot_commands_max0.002186953475532881
set_robot_commands_mean0.0021531855616172767
set_robot_commands_median0.0021457162492635115
set_robot_commands_min0.0021343562724092024
sim_compute_performance-ego0_max0.002019168336028163
sim_compute_performance-ego0_mean0.0018974148847855088
sim_compute_performance-ego0_median0.0018688132200865995
sim_compute_performance-ego0_min0.001832864762940673
sim_compute_sim_state_max0.012508262305533658
sim_compute_sim_state_mean0.009549055420603136
sim_compute_sim_state_median0.009835163821418025
sim_compute_sim_state_min0.0060176317340428385
sim_render-ego0_max0.003913429754163502
sim_render-ego0_mean0.003796592378461129
sim_render-ego0_median0.003770644989068534
sim_render-ego0_min0.0037316497815439447
simulation-passed1
step_physics_max0.13033556938171387
step_physics_mean0.10332666249398288
step_physics_median0.0996213685860741
step_physics_min0.08372834342206943
survival_time_max59.99999999999873
survival_time_mean46.22499999999904
survival_time_min4.899999999999991
No reset possible
5690311282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5688711425Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-020:00:44
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5685811285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:02:19
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5681711079Étienne Boucher 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:04:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5421232533589626
survival_time_median5.474999999999989
deviation-center-line_median0.14258752155014676
in-drivable-lane_median2.9749999999999894


other stats
agent_compute-ego0_max0.013074104602520283
agent_compute-ego0_mean0.012038237580355372
agent_compute-ego0_median0.012038293171089853
agent_compute-ego0_min0.011002259376721503
complete-iteration_max0.1956741505480827
complete-iteration_mean0.1719852204764062
complete-iteration_median0.17369792246083332
complete-iteration_min0.1448708864358755
deviation-center-line_max0.2264256960389139
deviation-center-line_mean0.14911646769757775
deviation-center-line_min0.0848651316511036
deviation-heading_max2.296491387142445
deviation-heading_mean1.155194197174774
deviation-heading_median0.9622225466145684
deviation-heading_min0.39984030832751527
driven_any_max3.584892340967327
driven_any_mean1.981996613978529
driven_any_median1.540872526162007
driven_any_min1.2613490626227757
driven_lanedir_consec_max0.6935156511660254
driven_lanedir_consec_mean0.5011603005189074
driven_lanedir_consec_min0.22687904419167904
driven_lanedir_max0.6935156511660254
driven_lanedir_mean0.5048368725998398
driven_lanedir_median0.5494763975208273
driven_lanedir_min0.22687904419167904
get_duckie_state_max1.3731993161714995e-06
get_duckie_state_mean1.264459449853148e-06
get_duckie_state_median1.2544238564000267e-06
get_duckie_state_min1.175790770441039e-06
get_robot_state_max0.0039176459495837875
get_robot_state_mean0.0035549533495037805
get_robot_state_median0.0034583617378525907
get_robot_state_min0.0033854439727261537
get_state_dump_max0.004921019077301025
get_state_dump_mean0.004641133192925258
get_state_dump_median0.004611182574848615
get_state_dump_min0.004421148544702775
get_ui_image_max0.03387240653342389
get_ui_image_mean0.029363049681270993
get_ui_image_median0.029355403332538985
get_ui_image_min0.02486898552658211
in-drivable-lane_max9.750000000000016
in-drivable-lane_mean4.424999999999997
in-drivable-lane_min1.9999999999999951
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.584892340967327, "get_ui_image": 0.02727050508171451, "step_physics": 0.0950988987994089, "survival_time": 11.300000000000026, "driven_lanedir": 0.22687904419167904, "get_state_dump": 0.004716238786470523, "get_robot_state": 0.0035171991928033366, "sim_render-ego0": 0.00381502285927928, "get_duckie_state": 1.2330546778204157e-06, "in-drivable-lane": 9.750000000000016, "deviation-heading": 1.2376476894062516, "agent_compute-ego0": 0.01202858282080831, "complete-iteration": 0.15967566536386632, "set_robot_commands": 0.0020546545541233955, "deviation-center-line": 0.15769364294070565, "driven_lanedir_consec": 0.22687904419167904, "sim_compute_sim_state": 0.009169875787743389, "sim_compute_performance-ego0": 0.0019152500555903904}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.2613490626227757, "get_ui_image": 0.03387240653342389, "step_physics": 0.12556611730697306, "survival_time": 4.6499999999999915, "driven_lanedir": 0.4127998105203827, "get_state_dump": 0.004506126363226708, "get_robot_state": 0.003399524282901845, "sim_render-ego0": 0.003661591955955992, "get_duckie_state": 1.2757930349796375e-06, "in-drivable-lane": 1.9999999999999951, "deviation-heading": 2.296491387142445, "agent_compute-ego0": 0.012048003521371396, "complete-iteration": 0.1956741505480827, "set_robot_commands": 0.0019755617101141747, "deviation-center-line": 0.2264256960389139, "driven_lanedir_consec": 0.3980935221966533, "sim_compute_sim_state": 0.008761367899306278, "sim_compute_performance-ego0": 0.001795413646292179}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.4185786724850264, "get_ui_image": 0.03144030158336346, "step_physics": 0.11716584517405584, "survival_time": 5.14999999999999, "driven_lanedir": 0.6935156511660254, "get_state_dump": 0.004921019077301025, "get_robot_state": 0.0039176459495837875, "sim_render-ego0": 0.0040986033586355355, "get_duckie_state": 1.3731993161714995e-06, "in-drivable-lane": 2.549999999999991, "deviation-heading": 0.686797403822885, "agent_compute-ego0": 0.013074104602520283, "complete-iteration": 0.1877201795578003, "set_robot_commands": 0.002273282179465661, "deviation-center-line": 0.12748140015958784, "driven_lanedir_consec": 0.6935156511660254, "sim_compute_sim_state": 0.008621509258563701, "sim_compute_performance-ego0": 0.0021084730441753683}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6631663798389875, "get_ui_image": 0.02486898552658211, "step_physics": 0.08911847049354488, "survival_time": 5.799999999999987, "driven_lanedir": 0.6861529845212719, "get_state_dump": 0.004421148544702775, "get_robot_state": 0.0033854439727261537, "sim_render-ego0": 0.0035882994659945495, "get_duckie_state": 1.175790770441039e-06, "in-drivable-lane": 3.399999999999988, "deviation-heading": 0.39984030832751527, "agent_compute-ego0": 0.011002259376721503, "complete-iteration": 0.1448708864358755, "set_robot_commands": 0.0019115994119236611, "deviation-center-line": 0.0848651316511036, "driven_lanedir_consec": 0.6861529845212719, "sim_compute_sim_state": 0.004691792349530081, "sim_compute_performance-ego0": 0.001799923741919363}}
set_robot_commands_max0.002273282179465661
set_robot_commands_mean0.0020537744639067233
set_robot_commands_median0.002015108132118786
set_robot_commands_min0.0019115994119236611
sim_compute_performance-ego0_max0.0021084730441753683
sim_compute_performance-ego0_mean0.001904765121994325
sim_compute_performance-ego0_median0.0018575868987548768
sim_compute_performance-ego0_min0.001795413646292179
sim_compute_sim_state_max0.009169875787743389
sim_compute_sim_state_mean0.0078111363237858625
sim_compute_sim_state_median0.00869143857893499
sim_compute_sim_state_min0.004691792349530081
sim_render-ego0_max0.0040986033586355355
sim_render-ego0_mean0.003790879409966339
sim_render-ego0_median0.003738307407617636
sim_render-ego0_min0.0035882994659945495
simulation-passed1
step_physics_max0.12556611730697306
step_physics_mean0.10673733294349567
step_physics_median0.10613237198673237
step_physics_min0.08911847049354488
survival_time_max11.300000000000026
survival_time_mean6.724999999999999
survival_time_min4.6499999999999915
No reset possible
5679111283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:03:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5677611284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5675311281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:02:07
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5673011290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:02:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5670211427Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-020:01:39
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5668111296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:02:15
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5580111422Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LFVI_multi-sim-validationLFVIv-simsuccessyesnogpu-prod-021:21:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.00627423012839
survival_time_median59.99999999999873
deviation-center-line_median2.851622727852673
in-drivable-lane_median0.29999999999998694


other stats
agent_compute-ego0_max0.04501443858150638
agent_compute-ego0_mean0.04420001292407364
agent_compute-ego0_median0.04420001292407364
agent_compute-ego0_min0.0433855872666409
agent_compute-ego1_max0.04314861984475269
agent_compute-ego1_mean0.04291249314116796
agent_compute-ego1_median0.04291249314116796
agent_compute-ego1_min0.04267636643758324
agent_compute-ego2_max0.04440310714048311
agent_compute-ego2_mean0.0436639401636751
agent_compute-ego2_median0.0436639401636751
agent_compute-ego2_min0.04292477318686709
agent_compute-ego3_max0.0433818154489865
agent_compute-ego3_mean0.042804939363719416
agent_compute-ego3_median0.042804939363719416
agent_compute-ego3_min0.04222806327845234
complete-iteration_max1.1209499421068077
complete-iteration_mean1.120512246986313
complete-iteration_median1.120512246986313
complete-iteration_min1.1200745518658184
deviation-center-line_max6.993703781443878
deviation-center-line_mean3.9920540805709623
deviation-center-line_min1.81559138318626
deviation-heading_max42.18086494270505
deviation-heading_mean21.576090648227545
deviation-heading_median13.045436950124678
deviation-heading_min9.36000672480762
driven_any_max14.590320803977198
driven_any_mean7.924562479268612
driven_any_median8.415174702140996
driven_any_min0.4793280258511935
driven_lanedir_consec_max14.286955591091672
driven_lanedir_consec_mean7.611218517557059
driven_lanedir_consec_min0.4607971690050614
driven_lanedir_max14.286955591091672
driven_lanedir_mean7.63295552871756
driven_lanedir_median8.006275032326903
driven_lanedir_min0.4607971690050614
get_duckie_state_max1.4483779792880933e-06
get_duckie_state_mean1.4256478150023906e-06
get_duckie_state_median1.4256478150023906e-06
get_duckie_state_min1.402917650716688e-06
get_robot_state_max0.01421338533183915
get_robot_state_mean0.014212008419878576
get_robot_state_median0.014212008419878576
get_robot_state_min0.014210631507918002
get_state_dump_max0.009369456698555037
get_state_dump_mean0.009361090112189071
get_state_dump_median0.009361090112189071
get_state_dump_min0.009352723525823106
get_ui_image_max0.04901897500297807
get_ui_image_mean0.04864840225613584
get_ui_image_median0.04864840225613584
get_ui_image_min0.04827782950929361
in-drivable-lane_max1.2999999999999954
in-drivable-lane_mean0.43124999999999425
in-drivable-lane_min0.0
per-episodes
details{"LFVI_multi-norm-4way-000-ego0": {"driven_any": 14.590274250259988, "get_ui_image": 0.04901897500297807, "step_physics": 0.8186401894050871, "survival_time": 59.99999999999873, "driven_lanedir": 14.285870115338705, "get_state_dump": 0.009352723525823106, "get_robot_state": 0.01421338533183915, "sim_render-ego0": 0.003749904584924347, "sim_render-ego1": 0.003729821244048437, "sim_render-ego2": 0.003699620101572969, "sim_render-ego3": 0.0036867486745689832, "get_duckie_state": 1.402917650716688e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.65141140471916, "agent_compute-ego0": 0.0433855872666409, "agent_compute-ego1": 0.04267636643758324, "agent_compute-ego2": 0.04292477318686709, "agent_compute-ego3": 0.04222806327845234, "complete-iteration": 1.1200745518658184, "set_robot_commands": 0.0024865564954568704, "deviation-center-line": 2.869488261734872, "driven_lanedir_consec": 14.285870115338705, "sim_compute_sim_state": 0.02520927402200945, "sim_compute_performance-ego0": 0.002074803043463943, "sim_compute_performance-ego1": 0.0019589725084646257, "sim_compute_performance-ego2": 0.0019872115116929333, "sim_compute_performance-ego3": 0.001933575867613984}, "LFVI_multi-norm-4way-000-ego1": {"driven_any": 2.2400890986044524, "get_ui_image": 0.04901897500297807, "step_physics": 0.8186401894050871, "survival_time": 59.99999999999873, "driven_lanedir": 1.900791696912732, "get_state_dump": 0.009352723525823106, "get_robot_state": 0.01421338533183915, "sim_render-ego0": 0.003749904584924347, "sim_render-ego1": 0.003729821244048437, "sim_render-ego2": 0.003699620101572969, "sim_render-ego3": 0.0036867486745689832, "get_duckie_state": 1.402917650716688e-06, "in-drivable-lane": 0.8999999999999968, "deviation-heading": 38.71500900662983, "agent_compute-ego0": 0.0433855872666409, "agent_compute-ego1": 0.04267636643758324, "agent_compute-ego2": 0.04292477318686709, "agent_compute-ego3": 0.04222806327845234, "complete-iteration": 1.1200745518658184, "set_robot_commands": 0.0024865564954568704, "deviation-center-line": 6.000862773390881, "driven_lanedir_consec": 1.900791696912732, "sim_compute_sim_state": 0.02520927402200945, "sim_compute_performance-ego0": 0.002074803043463943, "sim_compute_performance-ego1": 0.0019589725084646257, "sim_compute_performance-ego2": 0.0019872115116929333, "sim_compute_performance-ego3": 0.001933575867613984}, "LFVI_multi-norm-4way-000-ego2": {"driven_any": 14.59026030567754, "get_ui_image": 0.04901897500297807, "step_physics": 0.8186401894050871, "survival_time": 59.99999999999873, "driven_lanedir": 14.286955591091672, "get_state_dump": 0.009352723525823106, "get_robot_state": 0.01421338533183915, "sim_render-ego0": 0.003749904584924347, "sim_render-ego1": 0.003729821244048437, "sim_render-ego2": 0.003699620101572969, "sim_render-ego3": 0.0036867486745689832, "get_duckie_state": 1.402917650716688e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.77473756366386, "agent_compute-ego0": 0.0433855872666409, "agent_compute-ego1": 0.04267636643758324, "agent_compute-ego2": 0.04292477318686709, "agent_compute-ego3": 0.04222806327845234, "complete-iteration": 1.1200745518658184, "set_robot_commands": 0.0024865564954568704, "deviation-center-line": 2.7783831944119246, "driven_lanedir_consec": 14.286955591091672, "sim_compute_sim_state": 0.02520927402200945, "sim_compute_performance-ego0": 0.002074803043463943, "sim_compute_performance-ego1": 0.0019589725084646257, "sim_compute_performance-ego2": 0.0019872115116929333, "sim_compute_performance-ego3": 0.001933575867613984}, "LFVI_multi-norm-4way-000-ego3": {"driven_any": 0.9929039380973048, "get_ui_image": 0.04901897500297807, "step_physics": 0.8186401894050871, "survival_time": 59.99999999999873, "driven_lanedir": 0.8491677542940025, "get_state_dump": 0.009352723525823106, "get_robot_state": 0.01421338533183915, "sim_render-ego0": 0.003749904584924347, "sim_render-ego1": 0.003729821244048437, "sim_render-ego2": 0.003699620101572969, "sim_render-ego3": 0.0036867486745689832, "get_duckie_state": 1.402917650716688e-06, "in-drivable-lane": 0.14999999999999947, "deviation-heading": 42.18086494270505, "agent_compute-ego0": 0.0433855872666409, "agent_compute-ego1": 0.04267636643758324, "agent_compute-ego2": 0.04292477318686709, "agent_compute-ego3": 0.04222806327845234, "complete-iteration": 1.1200745518658184, "set_robot_commands": 0.0024865564954568704, "deviation-center-line": 6.993703781443878, "driven_lanedir_consec": 0.6900128436534608, "sim_compute_sim_state": 0.02520927402200945, "sim_compute_performance-ego0": 0.002074803043463943, "sim_compute_performance-ego1": 0.0019589725084646257, "sim_compute_performance-ego2": 0.0019872115116929333, "sim_compute_performance-ego3": 0.001933575867613984}, "LFVI_multi-norm-udem1-000-ego0": {"driven_any": 14.590320803977198, "get_ui_image": 0.04827782950929361, "step_physics": 0.8167762226308017, "survival_time": 59.99999999999873, "driven_lanedir": 14.21564614875529, "get_state_dump": 0.009369456698555037, "get_robot_state": 0.014210631507918002, "sim_render-ego0": 0.003774412466425582, "sim_render-ego1": 0.003806877295043844, "sim_render-ego2": 0.003760845039806001, "sim_render-ego3": 0.003721927028214505, "get_duckie_state": 1.4483779792880933e-06, "in-drivable-lane": 0.4499999999999744, "deviation-heading": 9.36000672480762, "agent_compute-ego0": 0.04501443858150638, "agent_compute-ego1": 0.04314861984475269, "agent_compute-ego2": 0.04440310714048311, "agent_compute-ego3": 0.0433818154489865, "complete-iteration": 1.1209499421068077, "set_robot_commands": 0.002496278256202717, "deviation-center-line": 2.518317448732518, "driven_lanedir_consec": 14.21564352094802, "sim_compute_sim_state": 0.023431085527786905, "sim_compute_performance-ego0": 0.0020710443279129778, "sim_compute_performance-ego1": 0.001986162549351574, "sim_compute_performance-ego2": 0.0019680889917353013, "sim_compute_performance-ego3": 0.002001847752325739}, "LFVI_multi-norm-udem1-000-ego1": {"driven_any": 14.590300851084324, "get_ui_image": 0.04827782950929361, "step_physics": 0.8167762226308017, "survival_time": 59.99999999999873, "driven_lanedir": 14.111758367741071, "get_state_dump": 0.009369456698555037, "get_robot_state": 0.014210631507918002, "sim_render-ego0": 0.003774412466425582, "sim_render-ego1": 0.003806877295043844, "sim_render-ego2": 0.003760845039806001, "sim_render-ego3": 0.003721927028214505, "get_duckie_state": 1.4483779792880933e-06, "in-drivable-lane": 0.6499999999999879, "deviation-heading": 10.218220944182727, "agent_compute-ego0": 0.04501443858150638, "agent_compute-ego1": 0.04314861984475269, "agent_compute-ego2": 0.04440310714048311, "agent_compute-ego3": 0.0433818154489865, "complete-iteration": 1.1209499421068077, "set_robot_commands": 0.002496278256202717, "deviation-center-line": 2.8337571939704747, "driven_lanedir_consec": 14.111756763344047, "sim_compute_sim_state": 0.023431085527786905, "sim_compute_performance-ego0": 0.0020710443279129778, "sim_compute_performance-ego1": 0.001986162549351574, "sim_compute_performance-ego2": 0.0019680889917353013, "sim_compute_performance-ego3": 0.002001847752325739}, "LFVI_multi-norm-udem1-000-ego2": {"driven_any": 0.4793280258511935, "get_ui_image": 0.04827782950929361, "step_physics": 0.8167762226308017, "survival_time": 59.99999999999873, "driven_lanedir": 0.4607971690050614, "get_state_dump": 0.009369456698555037, "get_robot_state": 0.014210631507918002, "sim_render-ego0": 0.003774412466425582, "sim_render-ego1": 0.003806877295043844, "sim_render-ego2": 0.003760845039806001, "sim_render-ego3": 0.003721927028214505, "get_duckie_state": 1.4483779792880933e-06, "in-drivable-lane": 0.0, "deviation-heading": 35.39233826252663, "agent_compute-ego0": 0.04501443858150638, "agent_compute-ego1": 0.04314861984475269, "agent_compute-ego2": 0.04440310714048311, "agent_compute-ego3": 0.0433818154489865, "complete-iteration": 1.1209499421068077, "set_robot_commands": 0.002496278256202717, "deviation-center-line": 6.126328607696888, "driven_lanedir_consec": 0.4607971690050614, "sim_compute_sim_state": 0.023431085527786905, "sim_compute_performance-ego0": 0.0020710443279129778, "sim_compute_performance-ego1": 0.001986162549351574, "sim_compute_performance-ego2": 0.0019680889917353013, "sim_compute_performance-ego3": 0.002001847752325739}, "LFVI_multi-norm-udem1-000-ego3": {"driven_any": 1.323022560596903, "get_ui_image": 0.04827782950929361, "step_physics": 0.8167762226308017, "survival_time": 59.99999999999873, "driven_lanedir": 0.9526573866019388, "get_state_dump": 0.009369456698555037, "get_robot_state": 0.014210631507918002, "sim_render-ego0": 0.003774412466425582, "sim_render-ego1": 0.003806877295043844, "sim_render-ego2": 0.003760845039806001, "sim_render-ego3": 0.003721927028214505, "get_duckie_state": 1.4483779792880933e-06, "in-drivable-lane": 1.2999999999999954, "deviation-heading": 15.316136336585496, "agent_compute-ego0": 0.04501443858150638, "agent_compute-ego1": 0.04314861984475269, "agent_compute-ego2": 0.04440310714048311, "agent_compute-ego3": 0.0433818154489865, "complete-iteration": 1.1209499421068077, "set_robot_commands": 0.002496278256202717, "deviation-center-line": 1.81559138318626, "driven_lanedir_consec": 0.93792044016277, "sim_compute_sim_state": 0.023431085527786905, "sim_compute_performance-ego0": 0.0020710443279129778, "sim_compute_performance-ego1": 0.001986162549351574, "sim_compute_performance-ego2": 0.0019680889917353013, "sim_compute_performance-ego3": 0.002001847752325739}}
set_robot_commands_max0.002496278256202717
set_robot_commands_mean0.0024914173758297936
set_robot_commands_median0.0024914173758297936
set_robot_commands_min0.0024865564954568704
sim_compute_performance-ego0_max0.002074803043463943
sim_compute_performance-ego0_mean0.0020729236856884604
sim_compute_performance-ego0_median0.0020729236856884604
sim_compute_performance-ego0_min0.0020710443279129778
sim_compute_performance-ego1_max0.001986162549351574
sim_compute_performance-ego1_mean0.0019725675289081
sim_compute_performance-ego1_median0.0019725675289081
sim_compute_performance-ego1_min0.0019589725084646257
sim_compute_performance-ego2_max0.0019872115116929333
sim_compute_performance-ego2_mean0.0019776502517141175
sim_compute_performance-ego2_median0.0019776502517141175
sim_compute_performance-ego2_min0.0019680889917353013
sim_compute_performance-ego3_max0.002001847752325739
sim_compute_performance-ego3_mean0.0019677118099698616
sim_compute_performance-ego3_median0.0019677118099698616
sim_compute_performance-ego3_min0.001933575867613984
sim_compute_sim_state_max0.02520927402200945
sim_compute_sim_state_mean0.024320179774898178
sim_compute_sim_state_median0.024320179774898178
sim_compute_sim_state_min0.023431085527786905
sim_render-ego0_max0.003774412466425582
sim_render-ego0_mean0.003762158525674964
sim_render-ego0_median0.003762158525674964
sim_render-ego0_min0.003749904584924347
sim_render-ego1_max0.003806877295043844
sim_render-ego1_mean0.003768349269546141
sim_render-ego1_median0.003768349269546141
sim_render-ego1_min0.003729821244048437
sim_render-ego2_max0.003760845039806001
sim_render-ego2_mean0.003730232570689485
sim_render-ego2_median0.003730232570689485
sim_render-ego2_min0.003699620101572969
sim_render-ego3_max0.003721927028214505
sim_render-ego3_mean0.003704337851391744
sim_render-ego3_median0.003704337851391744
sim_render-ego3_min0.0036867486745689832
simulation-passed1
step_physics_max0.8186401894050871
step_physics_mean0.8177082060179444
step_physics_median0.8177082060179444
step_physics_min0.8167762226308017
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5573911291Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:03:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5564611301Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:03:02
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5561511310Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:01:23
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5557011450Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-testingLFt-simhost-erroryesnogpu-prod-020:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5550111289Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5548411307Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:49
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5545011307Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5539611287Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5534811309Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:58
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5531811424Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:38
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5529911302Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:36
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5528311424Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:46
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5524411286Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:01:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5519211303Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:01:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5498911413Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LFP-sim-validationLFP-simsuccessnonogpu-prod-020:13:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median8.025
in-drivable-lane_median0.5250000000000075
driven_lanedir_consec_median1.5425060846496597
deviation-center-line_median0.3402907324174555


other stats
agent_compute-ego0_max0.048485596592609696
agent_compute-ego0_mean0.047105682499159675
agent_compute-ego0_median0.048008361321350015
agent_compute-ego0_min0.04392041076132896
complete-iteration_max0.30308914545810584
complete-iteration_mean0.26148859513351375
complete-iteration_median0.2684083904011991
complete-iteration_min0.20604845427355103
deviation-center-line_max3.9835052701742866
deviation-center-line_mean1.2374055632002845
deviation-center-line_min0.28553551779193964
deviation-heading_max15.44160972654172
deviation-heading_mean4.849693625177389
deviation-heading_median1.5530721167059474
deviation-heading_min0.851020540755943
driven_any_max14.5898100863319
driven_any_mean4.636677183524886
driven_any_median1.7082199312507167
driven_any_min0.5404587852662095
driven_lanedir_consec_max11.088425839240465
driven_lanedir_consec_mean3.668158047567891
driven_lanedir_consec_min0.4991941817317804
driven_lanedir_max11.088425839240465
driven_lanedir_mean3.668158047567891
driven_lanedir_median1.5425060846496597
driven_lanedir_min0.4991941817317804
get_duckie_state_max0.025047992284481343
get_duckie_state_mean0.018065522290838847
get_duckie_state_median0.02149979009615731
get_duckie_state_min0.0042145166865594184
get_robot_state_max0.0037768509076989217
get_robot_state_mean0.00370905885594941
get_robot_state_median0.003705515380783793
get_robot_state_min0.003648353754531136
get_state_dump_max0.008738689697705783
get_state_dump_mean0.007684188366903155
get_state_dump_median0.008211876014161644
get_state_dump_min0.005574311741583552
get_ui_image_max0.035902731346361565
get_ui_image_mean0.03230665688770622
get_ui_image_median0.033422381074532215
get_ui_image_min0.02647913405539888
in-drivable-lane_max11.099999999999756
in-drivable-lane_mean3.037499999999943
in-drivable-lane_min0.0
per-episodes
details{"LFP-norm-loop-000-ego0": {"driven_any": 2.276806063496427, "get_ui_image": 0.03147825598716736, "step_physics": 0.12247610206787404, "survival_time": 10.350000000000012, "driven_lanedir": 1.9665277864594513, "get_state_dump": 0.008738689697705783, "get_robot_state": 0.003712035142458402, "sim_render-ego0": 0.0037776621488424447, "get_duckie_state": 0.025047992284481343, "in-drivable-lane": 1.050000000000015, "deviation-heading": 1.747008755262681, "agent_compute-ego0": 0.048485596592609696, "complete-iteration": 0.2558950437949254, "set_robot_commands": 0.0022229609581140373, "deviation-center-line": 0.3724996356547582, "driven_lanedir_consec": 1.9665277864594513, "sim_compute_sim_state": 0.007903449810468234, "sim_compute_performance-ego0": 0.0019520887961754431}, "LFP-norm-zigzag-000-ego0": {"driven_any": 0.5404587852662095, "get_ui_image": 0.035902731346361565, "step_physics": 0.16944136041583438, "survival_time": 3.2499999999999964, "driven_lanedir": 0.4991941817317804, "get_state_dump": 0.008072011398546623, "get_robot_state": 0.003698995619109183, "sim_render-ego0": 0.0036663647853966913, "get_duckie_state": 0.021026950893980084, "in-drivable-lane": 0.0, "deviation-heading": 1.3591354781492135, "agent_compute-ego0": 0.048157337940100464, "complete-iteration": 0.30308914545810584, "set_robot_commands": 0.0022541826421564274, "deviation-center-line": 0.28553551779193964, "driven_lanedir_consec": 0.4991941817317804, "sim_compute_sim_state": 0.008755431030735825, "sim_compute_performance-ego0": 0.0020115158774636007}, "LFP-norm-techtrack-000-ego0": {"driven_any": 1.1396337990050067, "get_ui_image": 0.03536650616189708, "step_physics": 0.1417240619659424, "survival_time": 5.699999999999988, "driven_lanedir": 1.118484382839868, "get_state_dump": 0.008351740629776665, "get_robot_state": 0.0037768509076989217, "sim_render-ego0": 0.003906256219615107, "get_duckie_state": 0.021972629298334535, "in-drivable-lane": 0.0, "deviation-heading": 0.851020540755943, "agent_compute-ego0": 0.04785938470259957, "complete-iteration": 0.2809217370074728, "set_robot_commands": 0.00229023435841436, "deviation-center-line": 0.3080818291801528, "driven_lanedir_consec": 1.118484382839868, "sim_compute_sim_state": 0.013432826166567597, "sim_compute_performance-ego0": 0.0021268803140391474}, "LFP-norm-small_loop-000-ego0": {"driven_any": 14.5898100863319, "get_ui_image": 0.02647913405539888, "step_physics": 0.1079750057064822, "survival_time": 59.99999999999873, "driven_lanedir": 11.088425839240465, "get_state_dump": 0.005574311741583552, "get_robot_state": 0.003648353754531136, "sim_render-ego0": 0.003660811671210169, "get_duckie_state": 0.0042145166865594184, "in-drivable-lane": 11.099999999999756, "deviation-heading": 15.44160972654172, "agent_compute-ego0": 0.04392041076132896, "complete-iteration": 0.20604845427355103, "set_robot_commands": 0.002246688347275708, "deviation-center-line": 3.9835052701742866, "driven_lanedir_consec": 11.088425839240465, "sim_compute_sim_state": 0.006318425457245305, "sim_compute_performance-ego0": 0.0019180937472429995}}
set_robot_commands_max0.00229023435841436
set_robot_commands_mean0.002253516576490133
set_robot_commands_median0.002250435494716068
set_robot_commands_min0.0022229609581140373
sim_compute_performance-ego0_max0.0021268803140391474
sim_compute_performance-ego0_mean0.0020021446837302976
sim_compute_performance-ego0_median0.001981802336819522
sim_compute_performance-ego0_min0.0019180937472429995
sim_compute_sim_state_max0.013432826166567597
sim_compute_sim_state_mean0.00910253311625424
sim_compute_sim_state_median0.00832944042060203
sim_compute_sim_state_min0.006318425457245305
sim_render-ego0_max0.003906256219615107
sim_render-ego0_mean0.003752773706266103
sim_render-ego0_median0.003722013467119568
sim_render-ego0_min0.003660811671210169
simulation-passed1
step_physics_max0.16944136041583438
step_physics_mean0.13540413253903325
step_physics_median0.1321000820169082
step_physics_min0.1079750057064822
survival_time_max59.99999999999873
survival_time_mean19.824999999999683
survival_time_min3.2499999999999964
No reset possible
5496311436Charlie Gauthier 🇨🇦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5481011359Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:22:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8698293376206834
survival_time_median45.974999999999525
deviation-center-line_median0.4248566832600076
in-drivable-lane_median32.224999999999625


other stats
agent_compute-ego0_max0.012370208382769584
agent_compute-ego0_mean0.011725223606868628
agent_compute-ego0_median0.011579032417466571
agent_compute-ego0_min0.011372621209771784
complete-iteration_max0.20333228411374393
complete-iteration_mean0.18146906984727065
complete-iteration_median0.18450358992321303
complete-iteration_min0.15353681542891248
deviation-center-line_max1.0927204222835858
deviation-center-line_mean0.5275726787647103
deviation-center-line_min0.1678569262552397
deviation-heading_max4.613863479987735
deviation-heading_mean2.7221511918269035
deviation-heading_median2.685484328991754
deviation-heading_min0.903772629336369
driven_any_max13.731428903054857
driven_any_mean8.527252118894957
driven_any_median9.545419440093063
driven_any_min1.2867406923388456
driven_lanedir_consec_max4.406185580535196
driven_lanedir_consec_mean1.692538271882334
driven_lanedir_consec_min0.6243088317527725
driven_lanedir_max4.6205575848686955
driven_lanedir_mean1.8917368548203033
driven_lanedir_median1.161040501329873
driven_lanedir_min0.6243088317527725
get_duckie_state_max1.342146546690614e-06
get_duckie_state_mean1.2947913084761576e-06
get_duckie_state_median1.3100556009373252e-06
get_duckie_state_min1.2169074853393655e-06
get_robot_state_max0.003700778194067416
get_robot_state_mean0.003605569161874757
get_robot_state_median0.003596065975807426
get_robot_state_min0.003529366501816758
get_state_dump_max0.004610075017822099
get_state_dump_mean0.004514701082365933
get_state_dump_median0.004489870595021771
get_state_dump_min0.00446898812159809
get_ui_image_max0.03443535391267363
get_ui_image_mean0.02953783856445502
get_ui_image_median0.02962123826472096
get_ui_image_min0.024473523815704525
in-drivable-lane_max56.69999999999873
in-drivable-lane_mean31.187499999999496
in-drivable-lane_min3.599999999999987
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.68317451574296, "get_ui_image": 0.02693881709296424, "step_physics": 0.10506135837451833, "survival_time": 55.44999999999899, "driven_lanedir": 4.6205575848686955, "get_state_dump": 0.004492230028719515, "get_robot_state": 0.003529366501816758, "sim_render-ego0": 0.00360326573655412, "get_duckie_state": 1.3327813363290048e-06, "in-drivable-lane": 37.049999999999216, "deviation-heading": 4.2843449823179744, "agent_compute-ego0": 0.011372621209771784, "complete-iteration": 0.1679538093171678, "set_robot_commands": 0.002094885250469586, "deviation-center-line": 1.0927204222835858, "driven_lanedir_consec": 4.406185580535196, "sim_compute_sim_state": 0.008907689060176815, "sim_compute_performance-ego0": 0.0018724512409519505}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.2867406923388456, "get_ui_image": 0.03443535391267363, "step_physics": 0.13179024616321483, "survival_time": 7.099999999999983, "driven_lanedir": 0.6243088317527725, "get_state_dump": 0.0044875111613240275, "get_robot_state": 0.003623052076859907, "sim_render-ego0": 0.0036467338775421354, "get_duckie_state": 1.342146546690614e-06, "in-drivable-lane": 3.599999999999987, "deviation-heading": 1.086623675665534, "agent_compute-ego0": 0.011657086285677824, "complete-iteration": 0.20333228411374393, "set_robot_commands": 0.00215690786188299, "deviation-center-line": 0.1763537569365236, "driven_lanedir_consec": 0.6243088317527725, "sim_compute_sim_state": 0.009558077458735112, "sim_compute_performance-ego0": 0.0018940205340618856}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.407664364443165, "get_ui_image": 0.03230365943647768, "step_physics": 0.12865046861233692, "survival_time": 36.500000000000064, "driven_lanedir": 1.592372704515827, "get_state_dump": 0.004610075017822099, "get_robot_state": 0.003700778194067416, "sim_render-ego0": 0.003819950628215409, "get_duckie_state": 1.287329865545646e-06, "in-drivable-lane": 27.40000000000003, "deviation-heading": 4.613863479987735, "agent_compute-ego0": 0.012370208382769584, "complete-iteration": 0.2010533705292583, "set_robot_commands": 0.002240699880263385, "deviation-center-line": 0.6733596095834916, "driven_lanedir_consec": 1.0099503770974474, "sim_compute_sim_state": 0.011267229241972584, "sim_compute_performance-ego0": 0.0020051051425542454}, "LF-norm-small_loop-000-ego0": {"driven_any": 13.731428903054857, "get_ui_image": 0.024473523815704525, "step_physics": 0.09569605105524752, "survival_time": 59.99999999999873, "driven_lanedir": 0.7297082981439194, "get_state_dump": 0.00446898812159809, "get_robot_state": 0.0035690798747549444, "sim_render-ego0": 0.0036560020081506583, "get_duckie_state": 1.2169074853393655e-06, "in-drivable-lane": 56.69999999999873, "deviation-heading": 0.903772629336369, "agent_compute-ego0": 0.01150097854925532, "complete-iteration": 0.15353681542891248, "set_robot_commands": 0.002156897051745311, "deviation-center-line": 0.1678569262552397, "driven_lanedir_consec": 0.7297082981439194, "sim_compute_sim_state": 0.006013030116504475, "sim_compute_performance-ego0": 0.0019178741877521703}}
set_robot_commands_max0.002240699880263385
set_robot_commands_mean0.002162347511090318
set_robot_commands_median0.0021569024568141503
set_robot_commands_min0.002094885250469586
sim_compute_performance-ego0_max0.0020051051425542454
sim_compute_performance-ego0_mean0.001922362776330063
sim_compute_performance-ego0_median0.001905947360907028
sim_compute_performance-ego0_min0.0018724512409519505
sim_compute_sim_state_max0.011267229241972584
sim_compute_sim_state_mean0.008936506469347245
sim_compute_sim_state_median0.009232883259455964
sim_compute_sim_state_min0.006013030116504475
sim_render-ego0_max0.003819950628215409
sim_render-ego0_mean0.003681488062615581
sim_render-ego0_median0.003651367942846397
sim_render-ego0_min0.00360326573655412
simulation-passed1
step_physics_max0.13179024616321483
step_physics_mean0.1152995310513294
step_physics_median0.1168559134934276
step_physics_min0.09569605105524752
survival_time_max59.99999999999873
survival_time_mean39.76249999999944
survival_time_min7.099999999999983
No reset possible
5474011384Yishu Malhotra 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:12:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.106594534466961
survival_time_median13.45000000000006
deviation-center-line_median0.36526841213184225
in-drivable-lane_median8.125000000000034


other stats
agent_compute-ego0_max0.012938876325885454
agent_compute-ego0_mean0.012109955856550184
agent_compute-ego0_median0.011953850778674445
agent_compute-ego0_min0.011593245542966405
complete-iteration_max0.22061968947711744
complete-iteration_mean0.18382189130710672
complete-iteration_median0.17870628579296696
complete-iteration_min0.15725530416537553
deviation-center-line_max0.5434734649513698
deviation-center-line_mean0.3637523953402067
deviation-center-line_min0.18099929214577248
deviation-heading_max2.864535169113714
deviation-heading_mean1.838032610625348
deviation-heading_median1.824187986412532
deviation-heading_min0.8392193005626126
driven_any_max11.49095064559107
driven_any_mean4.573320655862596
driven_any_median2.740709952823009
driven_any_min1.3209120722132934
driven_lanedir_consec_max2.097935576261713
driven_lanedir_consec_mean1.2554420584715351
driven_lanedir_consec_min0.7106435886905049
driven_lanedir_max2.097935576261713
driven_lanedir_mean1.2554420584715351
driven_lanedir_median1.106594534466961
driven_lanedir_min0.7106435886905049
get_duckie_state_max1.529852549235026e-06
get_duckie_state_mean1.4292095666900494e-06
get_duckie_state_median1.4382335338515307e-06
get_duckie_state_min1.3105186498221098e-06
get_robot_state_max0.004066485290726026
get_robot_state_mean0.003770585839385485
get_robot_state_median0.003723638079427032
get_robot_state_min0.0035685819079618497
get_state_dump_max0.005101094120427182
get_state_dump_mean0.004929208294899973
get_state_dump_median0.004998378264598357
get_state_dump_min0.004618982529975999
get_ui_image_max0.035064990583219026
get_ui_image_mean0.030225988548340417
get_ui_image_median0.03005761711600241
get_ui_image_min0.02572372937813783
in-drivable-lane_max43.69999999999919
in-drivable-lane_mean15.737499999999812
in-drivable-lane_min2.9999999999999893
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.49095064559107, "get_ui_image": 0.02643391358460619, "step_physics": 0.0982096642955368, "survival_time": 53.199999999999115, "driven_lanedir": 2.097935576261713, "get_state_dump": 0.004618982529975999, "get_robot_state": 0.0035685819079618497, "sim_render-ego0": 0.003707269211889992, "get_duckie_state": 1.3105186498221098e-06, "in-drivable-lane": 43.69999999999919, "deviation-heading": 2.864535169113714, "agent_compute-ego0": 0.01181315368329975, "complete-iteration": 0.16089765342748222, "set_robot_commands": 0.002106855508867004, "deviation-center-line": 0.5434734649513698, "driven_lanedir_consec": 2.097935576261713, "sim_compute_sim_state": 0.008432919318687188, "sim_compute_performance-ego0": 0.0019204820265792343}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3209120722132934, "get_ui_image": 0.035064990583219026, "step_physics": 0.14589926914164894, "survival_time": 7.549999999999981, "driven_lanedir": 0.7973086526370299, "get_state_dump": 0.005101094120427182, "get_robot_state": 0.003819815422359266, "sim_render-ego0": 0.0038740729030809903, "get_duckie_state": 1.4367856477436266e-06, "in-drivable-lane": 2.9999999999999893, "deviation-heading": 1.233588713664136, "agent_compute-ego0": 0.012094547874049136, "complete-iteration": 0.22061968947711744, "set_robot_commands": 0.002186234060086702, "deviation-center-line": 0.2983481752063457, "driven_lanedir_consec": 0.7973086526370299, "sim_compute_sim_state": 0.010347225164112291, "sim_compute_performance-ego0": 0.002134798388732107}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.920275248347723, "get_ui_image": 0.03368132064739863, "step_physics": 0.11997083636621636, "survival_time": 19.150000000000137, "driven_lanedir": 1.415880416296892, "get_state_dump": 0.0050922830899556475, "get_robot_state": 0.004066485290726026, "sim_render-ego0": 0.004113240788380305, "get_duckie_state": 1.529852549235026e-06, "in-drivable-lane": 11.950000000000085, "deviation-heading": 2.4147872591609283, "agent_compute-ego0": 0.012938876325885454, "complete-iteration": 0.1965149181584517, "set_robot_commands": 0.0024047748496135077, "deviation-center-line": 0.4321886490573389, "driven_lanedir_consec": 1.415880416296892, "sim_compute_sim_state": 0.011940534537037214, "sim_compute_performance-ego0": 0.002203855042656263}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.561144657298295, "get_ui_image": 0.02572372937813783, "step_physics": 0.09812885217177562, "survival_time": 7.7499999999999805, "driven_lanedir": 0.7106435886905049, "get_state_dump": 0.004904473439241067, "get_robot_state": 0.0036274607364947978, "sim_render-ego0": 0.0038222701121599246, "get_duckie_state": 1.439681419959435e-06, "in-drivable-lane": 4.299999999999985, "deviation-heading": 0.8392193005626126, "agent_compute-ego0": 0.011593245542966405, "complete-iteration": 0.15725530416537553, "set_robot_commands": 0.002211560041476519, "deviation-center-line": 0.18099929214577248, "driven_lanedir_consec": 0.7106435886905049, "sim_compute_sim_state": 0.005150368580451379, "sim_compute_performance-ego0": 0.002003020200973902}}
set_robot_commands_max0.0024047748496135077
set_robot_commands_mean0.002227356115010933
set_robot_commands_median0.0021988970507816107
set_robot_commands_min0.002106855508867004
sim_compute_performance-ego0_max0.002203855042656263
sim_compute_performance-ego0_mean0.0020655389147353764
sim_compute_performance-ego0_median0.0020689092948530047
sim_compute_performance-ego0_min0.0019204820265792343
sim_compute_sim_state_max0.011940534537037214
sim_compute_sim_state_mean0.008967761900072018
sim_compute_sim_state_median0.00939007224139974
sim_compute_sim_state_min0.005150368580451379
sim_render-ego0_max0.004113240788380305
sim_render-ego0_mean0.003879213253877803
sim_render-ego0_median0.0038481715076204574
sim_render-ego0_min0.003707269211889992
simulation-passed1
step_physics_max0.14589926914164894
step_physics_mean0.11555215549379444
step_physics_median0.10909025033087658
step_physics_min0.09812885217177562
survival_time_max53.199999999999115
survival_time_mean21.91249999999981
survival_time_min7.549999999999981
No reset possible
5464111399Charlie Gauthier 🇨🇦Baseline solution using reinforcement learningaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:10:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.077656272933691
survival_time_median14.750000000000076
deviation-center-line_median0.46252570745783944
in-drivable-lane_median7.600000000000074


other stats
agent_compute-ego0_max0.02284077891039923
agent_compute-ego0_mean0.018334653520491632
agent_compute-ego0_median0.01748582701862833
agent_compute-ego0_min0.015526181134310636
complete-iteration_max0.20272856759436336
complete-iteration_mean0.18765965729734296
complete-iteration_median0.1897737115216392
complete-iteration_min0.1683626385517301
deviation-center-line_max0.606221524069435
deviation-center-line_mean0.4228870422849925
deviation-center-line_min0.1602752301548561
deviation-heading_max3.3122218941792245
deviation-heading_mean2.450592072456928
deviation-heading_median2.8097493428696865
deviation-heading_min0.8706477099091133
driven_any_max5.259322298273094
driven_any_mean3.467898536937759
driven_any_median3.700061989142467
driven_any_min1.2121478711930087
driven_lanedir_consec_max2.0465666276830836
driven_lanedir_consec_mean1.151906917478624
driven_lanedir_consec_min0.4057484963640312
driven_lanedir_max2.0465666276830836
driven_lanedir_mean1.151906917478624
driven_lanedir_median1.077656272933691
driven_lanedir_min0.4057484963640312
get_duckie_state_max1.4885051830394847e-06
get_duckie_state_mean1.4059802180510185e-06
get_duckie_state_median1.4069800157472074e-06
get_duckie_state_min1.3214556576701743e-06
get_robot_state_max0.0038745209977433487
get_robot_state_mean0.0037441811408587865
get_robot_state_median0.003734191258748372
get_robot_state_min0.0036338210481950536
get_state_dump_max0.00507410582121428
get_state_dump_mean0.004901434581634624
get_state_dump_median0.004955534956404389
get_state_dump_min0.004620562592515434
get_ui_image_max0.03520678944057888
get_ui_image_mean0.0298731468944224
get_ui_image_median0.02929691918847211
get_ui_image_min0.025691959760166493
in-drivable-lane_max14.850000000000112
in-drivable-lane_mean8.325000000000063
in-drivable-lane_min3.2499999999999885
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 1.2121478711930087, "get_ui_image": 0.027325683885866456, "step_physics": 0.11780170921806817, "survival_time": 5.4999999999999885, "driven_lanedir": 0.4057484963640312, "get_state_dump": 0.00507410582121428, "get_robot_state": 0.0038745209977433487, "sim_render-ego0": 0.0038804964976267777, "get_duckie_state": 1.4885051830394847e-06, "in-drivable-lane": 3.2499999999999885, "deviation-heading": 0.8706477099091133, "agent_compute-ego0": 0.016738481349773234, "complete-iteration": 0.1875234573811024, "set_robot_commands": 0.0024607761486156567, "deviation-center-line": 0.1602752301548561, "driven_lanedir_consec": 0.4057484963640312, "sim_compute_sim_state": 0.008200293188696509, "sim_compute_performance-ego0": 0.00207760527327254}, "LF-norm-zigzag-000-ego0": {"driven_any": 5.259322298273094, "get_ui_image": 0.03520678944057888, "step_physics": 0.1206057948830687, "survival_time": 20.200000000000152, "driven_lanedir": 1.0974886516681472, "get_state_dump": 0.0049572173460030265, "get_robot_state": 0.003734890619913737, "sim_render-ego0": 0.003945336518464265, "get_duckie_state": 1.4805499418282217e-06, "in-drivable-lane": 14.850000000000112, "deviation-heading": 3.3122218941792245, "agent_compute-ego0": 0.018233172687483423, "complete-iteration": 0.20272856759436336, "set_robot_commands": 0.002406790815753702, "deviation-center-line": 0.5229964177224931, "driven_lanedir_consec": 1.0974886516681472, "sim_compute_sim_state": 0.01145482239899812, "sim_compute_performance-ego0": 0.002089408591941551}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.464745992803318, "get_ui_image": 0.03126815449107777, "step_physics": 0.11799702297557484, "survival_time": 13.70000000000006, "driven_lanedir": 1.0578238941992344, "get_state_dump": 0.004953852566805752, "get_robot_state": 0.003733491897583008, "sim_render-ego0": 0.003831729021939364, "get_duckie_state": 1.3334100896661932e-06, "in-drivable-lane": 8.350000000000058, "deviation-heading": 2.571240302434447, "agent_compute-ego0": 0.015526181134310636, "complete-iteration": 0.19202396566217597, "set_robot_commands": 0.0024097182533957743, "deviation-center-line": 0.4020549971931858, "driven_lanedir_consec": 1.0578238941992344, "sim_compute_sim_state": 0.010126915845003996, "sim_compute_performance-ego0": 0.0020880352367054333}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.935377985481616, "get_ui_image": 0.025691959760166493, "step_physics": 0.09741093533272246, "survival_time": 15.80000000000009, "driven_lanedir": 2.0465666276830836, "get_state_dump": 0.004620562592515434, "get_robot_state": 0.0036338210481950536, "sim_render-ego0": 0.0037214590547964776, "get_duckie_state": 1.3214556576701743e-06, "in-drivable-lane": 6.850000000000091, "deviation-heading": 3.048258383304926, "agent_compute-ego0": 0.02284077891039923, "complete-iteration": 0.1683626385517301, "set_robot_commands": 0.0023752725462808217, "deviation-center-line": 0.606221524069435, "driven_lanedir_consec": 2.0465666276830836, "sim_compute_sim_state": 0.006021923071202419, "sim_compute_performance-ego0": 0.0019638583487140643}}
set_robot_commands_max0.0024607761486156567
set_robot_commands_mean0.0024131394410114885
set_robot_commands_median0.002408254534574738
set_robot_commands_min0.0023752725462808217
sim_compute_performance-ego0_max0.002089408591941551
sim_compute_performance-ego0_mean0.0020547268626583967
sim_compute_performance-ego0_median0.0020828202549889865
sim_compute_performance-ego0_min0.0019638583487140643
sim_compute_sim_state_max0.01145482239899812
sim_compute_sim_state_mean0.008950988625975261
sim_compute_sim_state_median0.009163604516850252
sim_compute_sim_state_min0.006021923071202419
sim_render-ego0_max0.003945336518464265
sim_render-ego0_mean0.003844755273206721
sim_render-ego0_median0.0038561127597830703
sim_render-ego0_min0.0037214590547964776
simulation-passed1
step_physics_max0.1206057948830687
step_physics_mean0.11345386560235854
step_physics_median0.1178993660968215
step_physics_min0.09741093533272246
survival_time_max20.200000000000152
survival_time_mean13.800000000000074
survival_time_min5.4999999999999885
No reset possible
5462511449Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5461911449Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5460711449Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5460111435Charlie Gauthier 🇨🇦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simfailedyesnogpu-prod-020:01:14
The container "solut [...]
The container "solution-ego0" exited with code 127.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5448711490Melisande Tengexercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:38:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median11.389485639410506
survival_time_median59.99999999999873
deviation-center-line_median2.8185891218312875
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.011964229421750593
agent_compute-ego0_mean0.01183648321848924
agent_compute-ego0_median0.01188387164069056
agent_compute-ego0_min0.011613960170825256
complete-iteration_max0.24502490839295143
complete-iteration_mean0.20964940675192337
complete-iteration_median0.2099457577205915
complete-iteration_min0.17368120317355878
deviation-center-line_max2.9409575686294183
deviation-center-line_mean2.6666477817623115
deviation-center-line_min2.0884553147572533
deviation-heading_max12.221579631720452
deviation-heading_mean10.802540786242956
deviation-heading_median10.921540460132716
deviation-heading_min9.14550259298594
driven_any_max11.67312243334937
driven_any_mean11.670511513252071
driven_any_median11.67291360710145
driven_any_min11.663096405456017
driven_lanedir_consec_max11.4800613735661
driven_lanedir_consec_mean11.393800782656218
driven_lanedir_consec_min11.316170478237762
driven_lanedir_max11.4800613735661
driven_lanedir_mean11.393800782656218
driven_lanedir_median11.389485639410506
driven_lanedir_min11.316170478237762
get_duckie_state_max1.3644054072981175e-06
get_duckie_state_mean1.3062400087329568e-06
get_duckie_state_median1.294130488895159e-06
get_duckie_state_min1.2722936498433922e-06
get_robot_state_max0.0037508558770401297
get_robot_state_mean0.003669942249167869
get_robot_state_median0.003660765317556364
get_robot_state_min0.003607382484518618
get_state_dump_max0.004778883240800615
get_state_dump_mean0.004686461350602175
get_state_dump_median0.004676328511361179
get_state_dump_min0.004614305138885727
get_ui_image_max0.03698132854020169
get_ui_image_mean0.03172004014427318
get_ui_image_median0.0318539790765729
get_ui_image_min0.026190873883745257
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.67312243334937, "get_ui_image": 0.02922933603901351, "step_physics": 0.1211485791265915, "survival_time": 59.99999999999873, "driven_lanedir": 11.40502503975861, "get_state_dump": 0.004718005309791788, "get_robot_state": 0.003662626312535371, "sim_render-ego0": 0.003734454425745066, "get_duckie_state": 1.3644054072981175e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.777750959185372, "agent_compute-ego0": 0.011613960170825256, "complete-iteration": 0.1877545037932638, "set_robot_commands": 0.0021892937891290747, "deviation-center-line": 2.7397894669509015, "driven_lanedir_consec": 11.40502503975861, "sim_compute_sim_state": 0.0093997322848, "sim_compute_performance-ego0": 0.0019668460785598183}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.67287588449254, "get_ui_image": 0.03698132854020169, "step_physics": 0.16580125533174614, "survival_time": 59.99999999999873, "driven_lanedir": 11.373946239062402, "get_state_dump": 0.004614305138885727, "get_robot_state": 0.003658904322577357, "sim_render-ego0": 0.0038228686107981713, "get_duckie_state": 1.2840061362438853e-06, "in-drivable-lane": 0.0, "deviation-heading": 11.06532996108006, "agent_compute-ego0": 0.011964229421750593, "complete-iteration": 0.24502490839295143, "set_robot_commands": 0.0021647160297428737, "deviation-center-line": 2.9409575686294183, "driven_lanedir_consec": 11.373946239062402, "sim_compute_sim_state": 0.013891294338026211, "sim_compute_performance-ego0": 0.0020339324214277816}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.663096405456017, "get_ui_image": 0.03447862211413229, "step_physics": 0.1561437663587305, "survival_time": 59.99999999999873, "driven_lanedir": 11.316170478237762, "get_state_dump": 0.004778883240800615, "get_robot_state": 0.0037508558770401297, "sim_render-ego0": 0.003800356020836111, "get_duckie_state": 1.2722936498433922e-06, "in-drivable-lane": 0.0, "deviation-heading": 12.221579631720452, "agent_compute-ego0": 0.011914409269004144, "complete-iteration": 0.2321370116479192, "set_robot_commands": 0.0022233195547061, "deviation-center-line": 2.897388776711673, "driven_lanedir_consec": 11.316170478237762, "sim_compute_sim_state": 0.01289283405434182, "sim_compute_performance-ego0": 0.002060121143985847}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.672951329710358, "get_ui_image": 0.026190873883745257, "step_physics": 0.11334665133296004, "survival_time": 59.99999999999873, "driven_lanedir": 11.4800613735661, "get_state_dump": 0.00463465171293057, "get_robot_state": 0.003607382484518618, "sim_render-ego0": 0.003725101112029038, "get_duckie_state": 1.3042548415464327e-06, "in-drivable-lane": 0.0, "deviation-heading": 9.14550259298594, "agent_compute-ego0": 0.011853334012376976, "complete-iteration": 0.17368120317355878, "set_robot_commands": 0.002191019097930883, "deviation-center-line": 2.0884553147572533, "driven_lanedir_consec": 11.4800613735661, "sim_compute_sim_state": 0.006140721826926556, "sim_compute_performance-ego0": 0.001903766398624417}}
set_robot_commands_max0.0022233195547061
set_robot_commands_mean0.002192087117877233
set_robot_commands_median0.0021901564435299788
set_robot_commands_min0.0021647160297428737
sim_compute_performance-ego0_max0.002060121143985847
sim_compute_performance-ego0_mean0.0019911665106494662
sim_compute_performance-ego0_median0.0020003892499938
sim_compute_performance-ego0_min0.001903766398624417
sim_compute_sim_state_max0.013891294338026211
sim_compute_sim_state_mean0.010581145626023646
sim_compute_sim_state_median0.01114628316957091
sim_compute_sim_state_min0.006140721826926556
sim_render-ego0_max0.0038228686107981713
sim_render-ego0_mean0.0037706950423520968
sim_render-ego0_median0.003767405223290589
sim_render-ego0_min0.003725101112029038
simulation-passed1
step_physics_max0.16580125533174614
step_physics_mean0.13911006303750706
step_physics_median0.138646172742661
step_physics_min0.11334665133296004
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5437311562Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:43:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.012437095848547231
agent_compute-ego0_mean0.012036547771996996
agent_compute-ego0_median0.012001923974805031
agent_compute-ego0_min0.0117052472898307
complete-iteration_max0.358445781355198
complete-iteration_mean0.305238757502725
complete-iteration_median0.3056761181225487
complete-iteration_min0.25115701241060456
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max1.6530487162187435e-06
get_duckie_state_mean1.615529056393435e-06
get_duckie_state_median1.6340903695874369e-06
get_duckie_state_min1.5408867701801233e-06
get_robot_state_max0.0038133546970567536
get_robot_state_mean0.003752121321863179
get_robot_state_median0.0037441454163995216
get_robot_state_min0.003706839757596921
get_state_dump_max0.004790956035045462
get_state_dump_mean0.004700115181623549
get_state_dump_median0.0047123342826900436
get_state_dump_min0.004584836126068649
get_ui_image_max0.035928613637309585
get_ui_image_mean0.031195233505433243
get_ui_image_median0.030939412355224453
get_ui_image_min0.026973495673974487
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.029594305849988498, "step_physics": 0.22820432160319537, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004744186687231262, "get_robot_state": 0.003761221626021284, "sim_render-ego0": 0.003865308904528717, "get_duckie_state": 1.6415347465369028e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.012184109715597516, "complete-iteration": 0.29562265708186447, "set_robot_commands": 0.0022718279089756948, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008881770998710995, "sim_compute_performance-ego0": 0.0020253394267441926}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.035928613637309585, "step_physics": 0.28153509859439235, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.004790956035045462, "get_robot_state": 0.0037270692067777585, "sim_render-ego0": 0.0039051136903024337, "get_duckie_state": 1.626645992637971e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.012437095848547231, "complete-iteration": 0.358445781355198, "set_robot_commands": 0.0022204706412767192, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.011751949936027432, "sim_compute_performance-ego0": 0.002059858903400507}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.032284518860460416, "step_physics": 0.24646271019553664, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004584836126068649, "get_robot_state": 0.003706839757596921, "sim_render-ego0": 0.0037522218705811766, "get_duckie_state": 1.5408867701801233e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.0117052472898307, "complete-iteration": 0.3157295791632329, "set_robot_commands": 0.002205423272519584, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008959897848886813, "sim_compute_performance-ego0": 0.0019784344920111534}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.026973495673974487, "step_physics": 0.18920830187452128, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004680481878148825, "get_robot_state": 0.0038133546970567536, "sim_render-ego0": 0.0038959753701927062, "get_duckie_state": 1.6530487162187435e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.01181973823401255, "complete-iteration": 0.25115701241060456, "set_robot_commands": 0.0023030807930265835, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.006308968716318859, "sim_compute_performance-ego0": 0.0020630643528565876}}
set_robot_commands_max0.0023030807930265835
set_robot_commands_mean0.0022502006539496452
set_robot_commands_median0.0022461492751262068
set_robot_commands_min0.002205423272519584
sim_compute_performance-ego0_max0.0020630643528565876
sim_compute_performance-ego0_mean0.00203167429375311
sim_compute_performance-ego0_median0.00204259916507235
sim_compute_performance-ego0_min0.0019784344920111534
sim_compute_sim_state_max0.011751949936027432
sim_compute_sim_state_mean0.008975646874986025
sim_compute_sim_state_median0.008920834423798905
sim_compute_sim_state_min0.006308968716318859
sim_render-ego0_max0.0039051136903024337
sim_render-ego0_mean0.003854654958901258
sim_render-ego0_median0.003880642137360711
sim_render-ego0_min0.0037522218705811766
simulation-passed1
step_physics_max0.28153509859439235
step_physics_mean0.2363526080669114
step_physics_median0.23733351589936597
step_physics_min0.18920830187452128
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5436211624Anthony Courchesne 🇨🇦TNetR1aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 35, in init
              ||     self.check_tensorflow_gpu()
              ||   File "solution.py", line 51, in check_tensorflow_gpu
              ||     raise Exception(msg)
              || Exception: Could not find gpu device.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 35, in init
              || |     self.check_tensorflow_gpu()
              || |   File "solution.py", line 51, in check_tensorflow_gpu
              || |     raise Exception(msg)
              || | Exception: Could not find gpu device.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5423411626Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:36:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.867069096272901
survival_time_median59.99999999999873
deviation-center-line_median1.1122265325895313
in-drivable-lane_median41.99999999999941


other stats
agent_compute-ego0_max0.01299392114968026
agent_compute-ego0_mean0.012648709024020596
agent_compute-ego0_median0.012656154801703456
agent_compute-ego0_min0.012288605342995217
complete-iteration_max0.22156341227492848
complete-iteration_mean0.1952306486256224
complete-iteration_median0.19187828086992784
complete-iteration_min0.17560262048770545
deviation-center-line_max1.7166602940422655
deviation-center-line_mean1.0258489516299802
deviation-center-line_min0.16228244729859262
deviation-heading_max9.15428285026691
deviation-heading_mean4.576821619881662
deviation-heading_median4.0689829478457336
deviation-heading_min1.0150377335682732
driven_any_max12.838676863191624
driven_any_mean10.821236768460174
driven_any_median11.903425745966214
driven_any_min6.639418718716655
driven_lanedir_consec_max5.773654484919978
driven_lanedir_consec_mean3.0468603290455714
driven_lanedir_consec_min0.679648638716505
driven_lanedir_max6.298016488494054
driven_lanedir_mean3.1779508299390904
driven_lanedir_median2.867069096272901
driven_lanedir_min0.679648638716505
get_duckie_state_max1.5307624175288496e-06
get_duckie_state_mean1.428342779863952e-06
get_duckie_state_median1.4303129578907229e-06
get_duckie_state_min1.3219827861455135e-06
get_robot_state_max0.004075448578541523
get_robot_state_mean0.003934191732904904
get_robot_state_median0.003908095510674952
get_robot_state_min0.003845127331728191
get_state_dump_max0.005083762438072948
get_state_dump_mean0.004924085019921976
get_state_dump_median0.004923679153279594
get_state_dump_min0.004765219335055768
get_ui_image_max0.036610711861208
get_ui_image_mean0.03156917278305959
get_ui_image_median0.031231894580450385
get_ui_image_min0.02720219011012958
in-drivable-lane_max56.69999999999873
in-drivable-lane_mean38.649999999999395
in-drivable-lane_min13.900000000000016
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.838676863191624, "get_ui_image": 0.02859357572614303, "step_physics": 0.1106881063049977, "survival_time": 59.99999999999873, "driven_lanedir": 1.965476644599613, "get_state_dump": 0.004765219335055768, "get_robot_state": 0.0038651213062295903, "sim_render-ego0": 0.0038259065121437094, "get_duckie_state": 1.3493181366805332e-06, "in-drivable-lane": 51.79999999999875, "deviation-heading": 1.7859763142710374, "agent_compute-ego0": 0.012288605342995217, "complete-iteration": 0.1777717999673505, "set_robot_commands": 0.002228557616844463, "deviation-center-line": 0.5631766614230058, "driven_lanedir_consec": 1.965476644599613, "sim_compute_sim_state": 0.009399227258267749, "sim_compute_performance-ego0": 0.0020276459924982154}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.639418718716655, "get_ui_image": 0.036610711861208, "step_physics": 0.14390894546674166, "survival_time": 34.550000000000175, "driven_lanedir": 3.768661547946189, "get_state_dump": 0.004830342152215152, "get_robot_state": 0.003845127331728191, "sim_render-ego0": 0.003855857890465356, "get_duckie_state": 1.3219827861455135e-06, "in-drivable-lane": 13.900000000000016, "deviation-heading": 6.35198958142043, "agent_compute-ego0": 0.012385392809189813, "complete-iteration": 0.22156341227492848, "set_robot_commands": 0.0022643208503723145, "deviation-center-line": 1.7166602940422655, "driven_lanedir_consec": 3.768661547946189, "sim_compute_sim_state": 0.011685133324882198, "sim_compute_performance-ego0": 0.002083461408670238}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.553079263146737, "get_ui_image": 0.03387021343475774, "step_physics": 0.12928072181371328, "survival_time": 59.99999999999873, "driven_lanedir": 6.298016488494054, "get_state_dump": 0.005083762438072948, "get_robot_state": 0.003951069715120314, "sim_render-ego0": 0.004051490985384393, "get_duckie_state": 1.5307624175288496e-06, "in-drivable-lane": 32.200000000000074, "deviation-heading": 9.15428285026691, "agent_compute-ego0": 0.01299392114968026, "complete-iteration": 0.20598476177250516, "set_robot_commands": 0.0023532531144319227, "deviation-center-line": 1.6612764037560566, "driven_lanedir_consec": 5.773654484919978, "sim_compute_sim_state": 0.012129042368943646, "sim_compute_performance-ego0": 0.0021647392561989563}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.253772228785689, "get_ui_image": 0.02720219011012958, "step_physics": 0.11083127199660532, "survival_time": 59.99999999999873, "driven_lanedir": 0.679648638716505, "get_state_dump": 0.005017016154344036, "get_robot_state": 0.004075448578541523, "sim_render-ego0": 0.0040993464181663395, "get_duckie_state": 1.511307779100912e-06, "in-drivable-lane": 56.69999999999873, "deviation-heading": 1.0150377335682732, "agent_compute-ego0": 0.0129269167942171, "complete-iteration": 0.17560262048770545, "set_robot_commands": 0.002399793770986235, "deviation-center-line": 0.16228244729859262, "driven_lanedir_consec": 0.679648638716505, "sim_compute_sim_state": 0.00672058618435951, "sim_compute_performance-ego0": 0.0022306648718129585}}
set_robot_commands_max0.002399793770986235
set_robot_commands_mean0.0023114813381587334
set_robot_commands_median0.0023087869824021186
set_robot_commands_min0.002228557616844463
sim_compute_performance-ego0_max0.0022306648718129585
sim_compute_performance-ego0_mean0.0021266278822950923
sim_compute_performance-ego0_median0.002124100332434597
sim_compute_performance-ego0_min0.0020276459924982154
sim_compute_sim_state_max0.012129042368943646
sim_compute_sim_state_mean0.009983497284113275
sim_compute_sim_state_median0.010542180291574974
sim_compute_sim_state_min0.00672058618435951
sim_render-ego0_max0.0040993464181663395
sim_render-ego0_mean0.003958150451539949
sim_render-ego0_median0.003953674437924874
sim_render-ego0_min0.0038259065121437094
simulation-passed1
step_physics_max0.14390894546674166
step_physics_mean0.12367726139551448
step_physics_median0.12005599690515928
step_physics_min0.1106881063049977
survival_time_max59.99999999999873
survival_time_mean53.637499999999086
survival_time_min34.550000000000175
No reset possible
5420011634Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:10:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6964635943185153
survival_time_median14.800000000000075
deviation-center-line_median0.2874299912216557
in-drivable-lane_median9.925000000000091


other stats
agent_compute-ego0_max0.013441165924072266
agent_compute-ego0_mean0.012916425008940286
agent_compute-ego0_median0.012885245931995732
agent_compute-ego0_min0.012454042247697418
complete-iteration_max0.21763691902160645
complete-iteration_mean0.19601725500331124
complete-iteration_median0.19539925033913444
complete-iteration_min0.17563360031336955
deviation-center-line_max0.8994002776444183
deviation-center-line_mean0.38970546115225185
deviation-center-line_min0.08456158452127774
deviation-heading_max2.8134534256824124
deviation-heading_mean1.6787629321605797
deviation-heading_median1.6793934677320057
deviation-heading_min0.5428113674958953
driven_any_max5.927809274553928
driven_any_mean3.268454461353926
driven_any_median2.8907343871091777
driven_any_min1.3645397966434205
driven_lanedir_consec_max3.84369670814104
driven_lanedir_consec_mean1.466548237485842
driven_lanedir_consec_min0.629569053165298
driven_lanedir_max3.84369670814104
driven_lanedir_mean1.466548237485842
driven_lanedir_median0.6964635943185153
driven_lanedir_min0.629569053165298
get_duckie_state_max1.3980865478515625e-06
get_duckie_state_mean1.30026808282127e-06
get_duckie_state_median1.2937592587127766e-06
get_duckie_state_min1.2154672660079656e-06
get_robot_state_max0.003879271189371745
get_robot_state_mean0.0037857777783360607
get_robot_state_median0.0037755269971825847
get_robot_state_min0.003712785929607328
get_state_dump_max0.004818625646094753
get_state_dump_mean0.0047010524726161015
get_state_dump_median0.004696289048935059
get_state_dump_min0.004593006146499534
get_ui_image_max0.0346970329284668
get_ui_image_mean0.03103519428349489
get_ui_image_median0.031063696917365578
get_ui_image_min0.027316350370781606
in-drivable-lane_max15.10000000000012
in-drivable-lane_mean9.587500000000071
in-drivable-lane_min3.399999999999988
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.927809274553928, "get_ui_image": 0.02829861640930176, "step_physics": 0.11433431478510982, "survival_time": 26.900000000000247, "driven_lanedir": 3.84369670814104, "get_state_dump": 0.004618172300547527, "get_robot_state": 0.003712785929607328, "sim_render-ego0": 0.003802330905298576, "get_duckie_state": 1.358410865344895e-06, "in-drivable-lane": 11.700000000000166, "deviation-heading": 2.8134534256824124, "agent_compute-ego0": 0.012813321734624804, "complete-iteration": 0.18286955953750184, "set_robot_commands": 0.002327086089493392, "deviation-center-line": 0.8994002776444183, "driven_lanedir_consec": 3.84369670814104, "sim_compute_sim_state": 0.010797443549133187, "sim_compute_performance-ego0": 0.0020754027676272704}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.3645397966434205, "get_ui_image": 0.0338287774254294, "step_physics": 0.1355058243072111, "survival_time": 7.599999999999981, "driven_lanedir": 0.629569053165298, "get_state_dump": 0.004593006146499534, "get_robot_state": 0.0037378220776327297, "sim_render-ego0": 0.003678647521274542, "get_duckie_state": 1.2154672660079656e-06, "in-drivable-lane": 3.399999999999988, "deviation-heading": 1.6784643872497065, "agent_compute-ego0": 0.012454042247697418, "complete-iteration": 0.20792894114076704, "set_robot_commands": 0.002114436205695657, "deviation-center-line": 0.3775908359043549, "driven_lanedir_consec": 0.629569053165298, "sim_compute_sim_state": 0.009975043776767703, "sim_compute_performance-ego0": 0.001959046507193372}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.743996438089528, "get_ui_image": 0.0346970329284668, "step_physics": 0.1412517255147298, "survival_time": 18.70000000000013, "driven_lanedir": 0.7244387694584, "get_state_dump": 0.004774405797322591, "get_robot_state": 0.003879271189371745, "sim_render-ego0": 0.004134560902913411, "get_duckie_state": 1.3980865478515625e-06, "in-drivable-lane": 15.10000000000012, "deviation-heading": 1.6803225482143052, "agent_compute-ego0": 0.013441165924072266, "complete-iteration": 0.21763691902160645, "set_robot_commands": 0.0024242515563964845, "deviation-center-line": 0.1972691465389565, "driven_lanedir_consec": 0.7244387694584, "sim_compute_sim_state": 0.01076708730061849, "sim_compute_performance-ego0": 0.002174226760864258}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.0374723361288267, "get_ui_image": 0.027316350370781606, "step_physics": 0.11316247944418154, "survival_time": 10.90000000000002, "driven_lanedir": 0.6684884191786307, "get_state_dump": 0.004818625646094753, "get_robot_state": 0.00381323191673244, "sim_render-ego0": 0.003840491107609718, "get_duckie_state": 1.229107652080658e-06, "in-drivable-lane": 8.150000000000022, "deviation-heading": 0.5428113674958953, "agent_compute-ego0": 0.012957170129366662, "complete-iteration": 0.17563360031336955, "set_robot_commands": 0.002318530322209885, "deviation-center-line": 0.08456158452127774, "driven_lanedir_consec": 0.6684884191786307, "sim_compute_sim_state": 0.005303504804497984, "sim_compute_performance-ego0": 0.0020158661010602837}}
set_robot_commands_max0.0024242515563964845
set_robot_commands_mean0.0022960760434488545
set_robot_commands_median0.0023228082058516387
set_robot_commands_min0.002114436205695657
sim_compute_performance-ego0_max0.002174226760864258
sim_compute_performance-ego0_mean0.002056135534186296
sim_compute_performance-ego0_median0.0020456344343437773
sim_compute_performance-ego0_min0.001959046507193372
sim_compute_sim_state_max0.010797443549133187
sim_compute_sim_state_mean0.00921076985775434
sim_compute_sim_state_median0.010371065538693098
sim_compute_sim_state_min0.005303504804497984
sim_render-ego0_max0.004134560902913411
sim_render-ego0_mean0.003864007609274062
sim_render-ego0_median0.0038214110064541474
sim_render-ego0_min0.003678647521274542
simulation-passed1
step_physics_max0.1412517255147298
step_physics_mean0.1260635860128081
step_physics_median0.12492006954616044
step_physics_min0.11316247944418154
survival_time_max26.900000000000247
survival_time_mean16.025000000000095
survival_time_min7.599999999999981
No reset possible
5413911652Philippe Reddy 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:26:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.0268585350066735
survival_time_median46.84999999999947
deviation-center-line_median2.395840201682114
in-drivable-lane_median12.224999999999833


other stats
agent_compute-ego0_max0.013143268453389976
agent_compute-ego0_mean0.01293854752032288
agent_compute-ego0_median0.013039588410755743
agent_compute-ego0_min0.01253174480639006
complete-iteration_max0.2944679812381142
complete-iteration_mean0.25560204945129555
complete-iteration_median0.2518980533852367
complete-iteration_min0.2241441097965947
deviation-center-line_max3.633486966984053
deviation-center-line_mean2.1460322141173105
deviation-center-line_min0.1589614861209626
deviation-heading_max17.171638689732752
deviation-heading_mean10.53477137201356
deviation-heading_median11.998679937492712
deviation-heading_min0.970086923336064
driven_any_max15.83891765573174
driven_any_mean10.327426218347831
driven_any_median12.249506576209516
driven_any_min0.9717740652405528
driven_lanedir_consec_max10.100815072796594
driven_lanedir_consec_mean5.717413390136701
driven_lanedir_consec_min0.715121417736863
driven_lanedir_max10.100815072796594
driven_lanedir_mean6.291260582857346
driven_lanedir_median7.174552920447962
driven_lanedir_min0.715121417736863
get_duckie_state_max1.4815302713030482e-06
get_duckie_state_mean1.4448079908340489e-06
get_duckie_state_median1.4434850645857286e-06
get_duckie_state_min1.4107315628616898e-06
get_robot_state_max0.004120305813321662
get_robot_state_mean0.003948131449250082
get_robot_state_median0.003951309482436707
get_robot_state_min0.003769601018805253
get_state_dump_max0.005189699596828885
get_state_dump_mean0.005084019792825176
get_state_dump_median0.005115641344596107
get_state_dump_min0.004915096885279606
get_ui_image_max0.03691940558584113
get_ui_image_mean0.03236721907758543
get_ui_image_median0.03216507640552406
get_ui_image_min0.028219317913452453
in-drivable-lane_max23.74999999999949
in-drivable-lane_mean12.424999999999786
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 8.671831859427954, "get_ui_image": 0.029719970491197373, "step_physics": 0.15321073178891784, "survival_time": 33.70000000000022, "driven_lanedir": 6.361430045610415, "get_state_dump": 0.005189699596828885, "get_robot_state": 0.003943239141393591, "sim_render-ego0": 0.004010403244583695, "get_duckie_state": 1.4107315628616898e-06, "in-drivable-lane": 7.900000000000013, "deviation-heading": 7.003648873564713, "agent_compute-ego0": 0.01301378885904948, "complete-iteration": 0.2241441097965947, "set_robot_commands": 0.0023457103305392795, "deviation-center-line": 1.730614390682045, "driven_lanedir_consec": 4.066041274727839, "sim_compute_sim_state": 0.01043363995022244, "sim_compute_performance-ego0": 0.0021853499942355685}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.9717740652405528, "get_ui_image": 0.03691940558584113, "step_physics": 0.2180635878914281, "survival_time": 4.699999999999991, "driven_lanedir": 0.715121417736863, "get_state_dump": 0.004915096885279606, "get_robot_state": 0.003769601018805253, "sim_render-ego0": 0.003886441180580541, "get_duckie_state": 1.460627505653783e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.970086923336064, "agent_compute-ego0": 0.01253174480639006, "complete-iteration": 0.2944679812381142, "set_robot_commands": 0.002278925243176912, "deviation-center-line": 0.1589614861209626, "driven_lanedir_consec": 0.715121417736863, "sim_compute_sim_state": 0.009951378169812654, "sim_compute_performance-ego0": 0.0020653448606792247}, "LF-norm-techtrack-000-ego0": {"driven_any": 15.827181292991078, "get_ui_image": 0.034610182319850746, "step_physics": 0.19056853823221095, "survival_time": 59.99999999999873, "driven_lanedir": 10.100815072796594, "get_state_dump": 0.005066122639486931, "get_robot_state": 0.003959379823479823, "sim_render-ego0": 0.0040926053859510585, "get_duckie_state": 1.4815302713030482e-06, "in-drivable-lane": 16.54999999999965, "deviation-heading": 16.99371100142071, "agent_compute-ego0": 0.013065387962462007, "complete-iteration": 0.2694529305886071, "set_robot_commands": 0.0023723260052098917, "deviation-center-line": 3.633486966984053, "driven_lanedir_consec": 10.100815072796594, "sim_compute_sim_state": 0.013405100094289406, "sim_compute_performance-ego0": 0.0022174285710800895}, "LF-norm-small_loop-000-ego0": {"driven_any": 15.83891765573174, "get_ui_image": 0.028219317913452453, "step_physics": 0.1679211536315359, "survival_time": 59.99999999999873, "driven_lanedir": 7.987675795285508, "get_state_dump": 0.005165160049705283, "get_robot_state": 0.004120305813321662, "sim_render-ego0": 0.004171053039144219, "get_duckie_state": 1.426342623517674e-06, "in-drivable-lane": 23.74999999999949, "deviation-heading": 17.171638689732752, "agent_compute-ego0": 0.013143268453389976, "complete-iteration": 0.23434317618186623, "set_robot_commands": 0.002481745641297047, "deviation-center-line": 3.0610660126821827, "driven_lanedir_consec": 7.987675795285508, "sim_compute_sim_state": 0.006781825018762847, "sim_compute_performance-ego0": 0.002245247909965166}}
set_robot_commands_max0.002481745641297047
set_robot_commands_mean0.0023696768050557827
set_robot_commands_median0.002359018167874586
set_robot_commands_min0.002278925243176912
sim_compute_performance-ego0_max0.002245247909965166
sim_compute_performance-ego0_mean0.002178342833990012
sim_compute_performance-ego0_median0.002201389282657829
sim_compute_performance-ego0_min0.0020653448606792247
sim_compute_sim_state_max0.013405100094289406
sim_compute_sim_state_mean0.010142985808271837
sim_compute_sim_state_median0.010192509060017544
sim_compute_sim_state_min0.006781825018762847
sim_render-ego0_max0.004171053039144219
sim_render-ego0_mean0.004040125712564879
sim_render-ego0_median0.004051504315267377
sim_render-ego0_min0.003886441180580541
simulation-passed1
step_physics_max0.2180635878914281
step_physics_mean0.1824410028860232
step_physics_median0.17924484593187345
step_physics_min0.15321073178891784
survival_time_max59.99999999999873
survival_time_mean39.59999999999942
survival_time_min4.699999999999991
No reset possible
5411111659Bhavya Patwa 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:09:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.0091759228432915
survival_time_median12.075000000000037
deviation-center-line_median0.4129251658869017
in-drivable-lane_median5.975000000000049


other stats
agent_compute-ego0_max0.01293998989251142
agent_compute-ego0_mean0.01249734553097082
agent_compute-ego0_median0.012494696701658797
agent_compute-ego0_min0.012059998828054265
complete-iteration_max0.2390575526190586
complete-iteration_mean0.2063041034851144
complete-iteration_median0.2051493027810578
complete-iteration_min0.1758602557592834
deviation-center-line_max0.7930538342582206
deviation-center-line_mean0.4620812702167717
deviation-center-line_min0.22942091483506277
deviation-heading_max4.610429256324448
deviation-heading_mean3.104714284194132
deviation-heading_median3.097853335762342
deviation-heading_min1.6127212089273963
driven_any_max5.139089140270729
driven_any_mean2.6905100565437015
driven_any_median2.2084669552773457
driven_any_min1.2060171753493856
driven_lanedir_consec_max1.932515463802208
driven_lanedir_consec_mean1.118158603075052
driven_lanedir_consec_min0.5217671028114168
driven_lanedir_max1.939616904169282
driven_lanedir_mean1.1299026228779674
driven_lanedir_median1.01202592337151
driven_lanedir_min0.5559417405995672
get_duckie_state_max1.4318143083749574e-06
get_duckie_state_mean1.3434035620254717e-06
get_duckie_state_median1.3390176007108573e-06
get_duckie_state_min1.2637647383052146e-06
get_robot_state_max0.004115888981219849
get_robot_state_mean0.003884076907787367
get_robot_state_median0.003823052986976629
get_robot_state_min0.0037743126759763623
get_state_dump_max0.004882258795649627
get_state_dump_mean0.004779669059692597
get_state_dump_median0.004776831696941385
get_state_dump_min0.00468275404923799
get_ui_image_max0.03744892995865619
get_ui_image_mean0.03198217845947671
get_ui_image_median0.031618537994430033
get_ui_image_min0.02724270789039056
in-drivable-lane_max14.150000000000148
in-drivable-lane_mean7.150000000000059
in-drivable-lane_min2.499999999999991
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 1.433099721303008, "get_ui_image": 0.03040313460136372, "step_physics": 0.13931812484407685, "survival_time": 9.099999999999994, "driven_lanedir": 0.5559417405995672, "get_state_dump": 0.004882258795649627, "get_robot_state": 0.004115888981219849, "sim_render-ego0": 0.004010014195259803, "get_duckie_state": 1.4318143083749574e-06, "in-drivable-lane": 3.600000000000006, "deviation-heading": 4.169744601399693, "agent_compute-ego0": 0.01293998989251142, "complete-iteration": 0.209407252692134, "set_robot_commands": 0.002525023423908838, "deviation-center-line": 0.54514664450511, "driven_lanedir_consec": 0.5217671028114168, "sim_compute_sim_state": 0.008970281465457437, "sim_compute_performance-ego0": 0.002151011117820531}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.2060171753493856, "get_ui_image": 0.03744892995865619, "step_physics": 0.16155129377959204, "survival_time": 6.0499999999999865, "driven_lanedir": 0.558113486906108, "get_state_dump": 0.004793319545808385, "get_robot_state": 0.0037743126759763623, "sim_render-ego0": 0.003992881931242396, "get_duckie_state": 1.3738382058065446e-06, "in-drivable-lane": 2.499999999999991, "deviation-heading": 2.025962070124991, "agent_compute-ego0": 0.012565163315319624, "complete-iteration": 0.2390575526190586, "set_robot_commands": 0.002288394286984303, "deviation-center-line": 0.22942091483506277, "driven_lanedir_consec": 0.5524134858496712, "sim_compute_sim_state": 0.010487046398100306, "sim_compute_performance-ego0": 0.0020662858837940654}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.139089140270729, "get_ui_image": 0.03283394138749635, "step_physics": 0.1256495604485822, "survival_time": 24.40000000000021, "driven_lanedir": 1.939616904169282, "get_state_dump": 0.004760343848074384, "get_robot_state": 0.003830290035723665, "sim_render-ego0": 0.003946071265909082, "get_duckie_state": 1.2637647383052146e-06, "in-drivable-lane": 14.150000000000148, "deviation-heading": 4.610429256324448, "agent_compute-ego0": 0.012424230087997966, "complete-iteration": 0.20089135286998164, "set_robot_commands": 0.002305391132222119, "deviation-center-line": 0.7930538342582206, "driven_lanedir_consec": 1.932515463802208, "sim_compute_sim_state": 0.012929631157155418, "sim_compute_performance-ego0": 0.0021230521133828503}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.983834189251683, "get_ui_image": 0.02724270789039056, "step_physics": 0.11353412211336047, "survival_time": 15.05000000000008, "driven_lanedir": 1.4659383598369122, "get_state_dump": 0.00468275404923799, "get_robot_state": 0.003815815938229592, "sim_render-ego0": 0.003886729676202433, "get_duckie_state": 1.3041969956151697e-06, "in-drivable-lane": 8.350000000000094, "deviation-heading": 1.6127212089273963, "agent_compute-ego0": 0.012059998828054265, "complete-iteration": 0.1758602557592834, "set_robot_commands": 0.0022890623042125576, "deviation-center-line": 0.28070368726869344, "driven_lanedir_consec": 1.4659383598369122, "sim_compute_sim_state": 0.006181940337680034, "sim_compute_performance-ego0": 0.002080595256477002}}
set_robot_commands_max0.002525023423908838
set_robot_commands_mean0.0023519677868319546
set_robot_commands_median0.0022972267182173383
set_robot_commands_min0.002288394286984303
sim_compute_performance-ego0_max0.002151011117820531
sim_compute_performance-ego0_mean0.002105236092868612
sim_compute_performance-ego0_median0.002101823684929926
sim_compute_performance-ego0_min0.0020662858837940654
sim_compute_sim_state_max0.012929631157155418
sim_compute_sim_state_mean0.009642224839598298
sim_compute_sim_state_median0.009728663931778871
sim_compute_sim_state_min0.006181940337680034
sim_render-ego0_max0.004010014195259803
sim_render-ego0_mean0.003958924267153429
sim_render-ego0_median0.003969476598575739
sim_render-ego0_min0.003886729676202433
simulation-passed1
step_physics_max0.16155129377959204
step_physics_mean0.1350132752964029
step_physics_median0.1324838426463295
step_physics_min0.11353412211336047
survival_time_max24.40000000000021
survival_time_mean13.65000000000007
survival_time_min6.0499999999999865
No reset possible
5406311676Daniil Lisusreal-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:15:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.2075446514751933
survival_time_median21.225000000000165
deviation-center-line_median0.8483366285295805
in-drivable-lane_median10.775000000000103


other stats
agent_compute-ego0_max0.012681272026862568
agent_compute-ego0_mean0.012366075331045783
agent_compute-ego0_median0.012395047138714596
agent_compute-ego0_min0.011992935019891382
complete-iteration_max0.28125148790853993
complete-iteration_mean0.2351218137382869
complete-iteration_median0.24040247015834643
complete-iteration_min0.17843082672791463
deviation-center-line_max1.8061529827843392
deviation-center-line_mean0.8914770207711334
deviation-center-line_min0.0630818432410336
deviation-heading_max5.9412667956120355
deviation-heading_mean3.389590033982288
deviation-heading_median3.5137163607212565
deviation-heading_min0.5896606188746042
driven_any_max12.81473067600621
driven_any_mean6.987224553474558
driven_any_median6.8308066684394015
driven_any_min1.4725542010132158
driven_lanedir_consec_max5.261540617242175
driven_lanedir_consec_mean3.00287418307445
driven_lanedir_consec_min0.3348668121052385
driven_lanedir_max5.263185912519156
driven_lanedir_mean3.003285506893695
driven_lanedir_median3.2075446514751933
driven_lanedir_min0.3348668121052385
get_duckie_state_max1.4103065102787342e-06
get_duckie_state_mean1.301034299710295e-06
get_duckie_state_median1.289264452411754e-06
get_duckie_state_min1.215301783738938e-06
get_robot_state_max0.003847539349542707
get_robot_state_mean0.003705830405414738
get_robot_state_median0.003694439443191941
get_robot_state_min0.0035869033857323655
get_state_dump_max0.005521960264512169
get_state_dump_mean0.004853769726318977
get_state_dump_median0.004697343832301245
get_state_dump_min0.00449843097616125
get_ui_image_max0.03552932871712579
get_ui_image_mean0.03152269962788533
get_ui_image_median0.03226135034497196
get_ui_image_min0.026038769104471608
in-drivable-lane_max22.249999999999904
in-drivable-lane_mean11.925000000000027
in-drivable-lane_min3.899999999999989
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 9.62855583034733, "get_ui_image": 0.03078441700692904, "step_physics": 0.1691907991797237, "survival_time": 29.450000000000284, "driven_lanedir": 4.9520289289239905, "get_state_dump": 0.004887265674138473, "get_robot_state": 0.003794034456802627, "sim_render-ego0": 0.003847690760079077, "get_duckie_state": 1.4103065102787342e-06, "in-drivable-lane": 13.500000000000153, "deviation-heading": 5.9412667956120355, "agent_compute-ego0": 0.012283313476433189, "complete-iteration": 0.238954181994422, "set_robot_commands": 0.002278690418954623, "deviation-center-line": 1.3852572788032178, "driven_lanedir_consec": 4.9520289289239905, "sim_compute_sim_state": 0.009770647145934024, "sim_compute_performance-ego0": 0.0020291094052589547}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.4725542010132158, "get_ui_image": 0.03552932871712579, "step_physics": 0.20767494704988268, "survival_time": 5.349999999999989, "driven_lanedir": 0.3348668121052385, "get_state_dump": 0.00449843097616125, "get_robot_state": 0.0035948444295812536, "sim_render-ego0": 0.0035819035989266856, "get_duckie_state": 1.3179249233669705e-06, "in-drivable-lane": 3.899999999999989, "deviation-heading": 0.5896606188746042, "agent_compute-ego0": 0.012506780800996002, "complete-iteration": 0.28125148790853993, "set_robot_commands": 0.0022129350238376195, "deviation-center-line": 0.0630818432410336, "driven_lanedir_consec": 0.3348668121052385, "sim_compute_sim_state": 0.009676509433322482, "sim_compute_performance-ego0": 0.0018891427252027723}, "LF-norm-techtrack-000-ego0": {"driven_any": 12.81473067600621, "get_ui_image": 0.03373828368301488, "step_physics": 0.16464261429951857, "survival_time": 39.49999999999989, "driven_lanedir": 5.263185912519156, "get_state_dump": 0.005521960264512169, "get_robot_state": 0.003847539349542707, "sim_render-ego0": 0.004018573182271193, "get_duckie_state": 1.215301783738938e-06, "in-drivable-lane": 22.249999999999904, "deviation-heading": 5.748805421718319, "agent_compute-ego0": 0.012681272026862568, "complete-iteration": 0.24185075832227088, "set_robot_commands": 0.00239480702222976, "deviation-center-line": 1.8061529827843392, "driven_lanedir_consec": 5.261540617242175, "sim_compute_sim_state": 0.012817443095326574, "sim_compute_performance-ego0": 0.002100666313798328}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.033057506531472, "get_ui_image": 0.026038769104471608, "step_physics": 0.11849663997518606, "survival_time": 13.00000000000005, "driven_lanedir": 1.4630603740263963, "get_state_dump": 0.0045074219904640165, "get_robot_state": 0.0035869033857323655, "sim_render-ego0": 0.0036043974631590862, "get_duckie_state": 1.2606039814565374e-06, "in-drivable-lane": 8.05000000000006, "deviation-heading": 1.2786272997241943, "agent_compute-ego0": 0.011992935019891382, "complete-iteration": 0.17843082672791463, "set_robot_commands": 0.002089763509816137, "deviation-center-line": 0.31141597825594314, "driven_lanedir_consec": 1.4630603740263963, "sim_compute_sim_state": 0.006143206380792961, "sim_compute_performance-ego0": 0.0018855584535562215}}
set_robot_commands_max0.00239480702222976
set_robot_commands_mean0.002244048993709535
set_robot_commands_median0.0022458127213961215
set_robot_commands_min0.002089763509816137
sim_compute_performance-ego0_max0.002100666313798328
sim_compute_performance-ego0_mean0.001976119224454069
sim_compute_performance-ego0_median0.0019591260652308635
sim_compute_performance-ego0_min0.0018855584535562215
sim_compute_sim_state_max0.012817443095326574
sim_compute_sim_state_mean0.00960195151384401
sim_compute_sim_state_median0.009723578289628252
sim_compute_sim_state_min0.006143206380792961
sim_render-ego0_max0.004018573182271193
sim_render-ego0_mean0.00376314125110901
sim_render-ego0_median0.0037260441116190816
sim_render-ego0_min0.0035819035989266856
simulation-passed1
step_physics_max0.20767494704988268
step_physics_mean0.16500125012607775
step_physics_median0.16691670673962114
step_physics_min0.11849663997518606
survival_time_max39.49999999999989
survival_time_mean21.825000000000053
survival_time_min5.349999999999989
No reset possible
5396011315Anastasiya Nikolskaya 🇷🇺JetBrains Researchaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-020:46:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.9575220411172665
survival_time_median59.99999999999873
deviation-center-line_median1.237202305385947
in-drivable-lane_median46.07499999999864


other stats
agent_compute-ego0_max0.06961360958394758
agent_compute-ego0_mean0.06648667468119422
agent_compute-ego0_median0.06711044190428239
agent_compute-ego0_min0.06211220533226452
complete-iteration_max0.4086545035801363
complete-iteration_mean0.35585954733236347
complete-iteration_median0.36220660426038986
complete-iteration_min0.29037047722853787
deviation-center-line_max10.52062463261962
deviation-center-line_mean3.280549762518244
deviation-center-line_min0.1271698066814605
deviation-heading_max18.41106986958364
deviation-heading_mean7.103972532016823
deviation-heading_median4.801527175563365
deviation-heading_min0.4017659073569224
driven_any_max7.622248692683846
driven_any_mean3.8743140847390785
driven_any_median3.5058278144877546
driven_any_min0.8633520172969583
driven_lanedir_consec_max7.005153690969786
driven_lanedir_consec_mean2.935459463580595
driven_lanedir_consec_min0.8216400811180619
driven_lanedir_max7.005153690969786
driven_lanedir_mean2.935459463580595
driven_lanedir_median1.9575220411172665
driven_lanedir_min0.8216400811180619
get_duckie_state_max1.4223722891446257e-06
get_duckie_state_mean1.3989473163436391e-06
get_duckie_state_median1.405796143137148e-06
get_duckie_state_min1.3618246899556358e-06
get_robot_state_max0.0040015149970138
get_robot_state_mean0.003850591073524545
get_robot_state_median0.003810863709271103
get_robot_state_min0.0037791218785421738
get_state_dump_max0.004965120707820794
get_state_dump_mean0.0047385277200201765
get_state_dump_median0.004679254647794115
get_state_dump_min0.0046304808766716825
get_ui_image_max0.03615121301465189
get_ui_image_mean0.031876246002095626
get_ui_image_median0.03247002747334807
get_ui_image_min0.02641371604703447
in-drivable-lane_max57.64999999999873
in-drivable-lane_mean37.737499999999
in-drivable-lane_min1.149999999999996
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.622248692683846, "get_ui_image": 0.028895825569476813, "step_physics": 0.19683906299486248, "survival_time": 59.99999999999873, "driven_lanedir": 7.005153690969786, "get_state_dump": 0.004636181681281224, "get_robot_state": 0.003803644450280589, "sim_render-ego0": 0.003908716967262694, "get_duckie_state": 1.4223722891446257e-06, "in-drivable-lane": 43.449999999998624, "deviation-heading": 3.567473275888339, "agent_compute-ego0": 0.06961360958394758, "complete-iteration": 0.32077967812874036, "set_robot_commands": 0.0022621164710992184, "deviation-center-line": 1.3287891977930597, "driven_lanedir_consec": 7.005153690969786, "sim_compute_sim_state": 0.008618526911358353, "sim_compute_performance-ego0": 0.002105473479462305}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.6357525086111688, "get_ui_image": 0.03604422937721932, "step_physics": 0.27857091603529244, "survival_time": 59.99999999999873, "driven_lanedir": 1.161891071053795, "get_state_dump": 0.004722327614307006, "get_robot_state": 0.003818082968261617, "sim_render-ego0": 0.003906329803720898, "get_duckie_state": 1.4158212374290956e-06, "in-drivable-lane": 1.149999999999996, "deviation-heading": 18.41106986958364, "agent_compute-ego0": 0.0664573212051074, "complete-iteration": 0.4086545035801363, "set_robot_commands": 0.002310487054766068, "deviation-center-line": 10.52062463261962, "driven_lanedir_consec": 1.161891071053795, "sim_compute_sim_state": 0.01064573358635819, "sim_compute_performance-ego0": 0.002079737573539486}, "LF-norm-techtrack-000-ego0": {"driven_any": 0.8633520172969583, "get_ui_image": 0.03615121301465189, "step_physics": 0.2765175742372486, "survival_time": 59.99999999999873, "driven_lanedir": 0.8216400811180619, "get_state_dump": 0.004965120707820794, "get_robot_state": 0.0040015149970138, "sim_render-ego0": 0.004245800142184979, "get_duckie_state": 1.3957710488452009e-06, "in-drivable-lane": 57.64999999999873, "deviation-heading": 0.4017659073569224, "agent_compute-ego0": 0.06211220533226452, "complete-iteration": 0.4036335303920393, "set_robot_commands": 0.0024501701675783485, "deviation-center-line": 0.1271698066814605, "driven_lanedir_consec": 0.8216400811180619, "sim_compute_sim_state": 0.010856313570452967, "sim_compute_performance-ego0": 0.0022296169020551925}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.37590312036434, "get_ui_image": 0.02641371604703447, "step_physics": 0.17306195905464675, "survival_time": 59.99999999999873, "driven_lanedir": 2.7531530111807374, "get_state_dump": 0.0046304808766716825, "get_robot_state": 0.0037791218785421738, "sim_render-ego0": 0.003917962883433931, "get_duckie_state": 1.3618246899556358e-06, "in-drivable-lane": 48.69999999999865, "deviation-heading": 6.03558107523839, "agent_compute-ego0": 0.06776356260345738, "complete-iteration": 0.29037047722853787, "set_robot_commands": 0.002294766118782545, "deviation-center-line": 1.1456154129788352, "driven_lanedir_consec": 2.7531530111807374, "sim_compute_sim_state": 0.0063544872102888296, "sim_compute_performance-ego0": 0.0020587307328884053}}
set_robot_commands_max0.0024501701675783485
set_robot_commands_mean0.002329384953056545
set_robot_commands_median0.0023026265867743066
set_robot_commands_min0.0022621164710992184
sim_compute_performance-ego0_max0.0022296169020551925
sim_compute_performance-ego0_mean0.0021183896719863477
sim_compute_performance-ego0_median0.002092605526500896
sim_compute_performance-ego0_min0.0020587307328884053
sim_compute_sim_state_max0.010856313570452967
sim_compute_sim_state_mean0.009118765319614584
sim_compute_sim_state_median0.009632130248858272
sim_compute_sim_state_min0.0063544872102888296
sim_render-ego0_max0.004245800142184979
sim_render-ego0_mean0.003994702449150626
sim_render-ego0_median0.003913339925348312
sim_render-ego0_min0.003906329803720898
simulation-passed1
step_physics_max0.27857091603529244
step_physics_mean0.23124737808051257
step_physics_median0.2366783186160555
step_physics_min0.17306195905464675
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5394611705brian wongbaseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:02:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5392311710Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-020:03:21
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible