Duckietown Challenges Home Challenges Submissions

Evaluator 4882

ID4882
evaluatornogpu-prod-08
ownerI don't have one 😀
machinenogpu-prod_289a2648438c
processnogpu-prod-08_289a2648438c
version6.2.5
first heard
last heard
statusinactive
# evaluating
# success23 53916
# timeout
# failed6 54748
# error
# aborted3 57173
# host-error35 53981
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5758011012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-08----No reset possible
5756911012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-080:01:38
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5740510940Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:43:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.01299798994834576
agent_compute-ego0_mean0.012526460730165962
agent_compute-ego0_median0.0124935419930705
agent_compute-ego0_min0.012120768986177088
complete-iteration_max0.33851697264265557
complete-iteration_mean0.29295185210802077
complete-iteration_median0.2926759146135316
complete-iteration_min0.2479386065623643
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max2.321255991202806e-06
get_duckie_state_mean2.1444768532428217e-06
get_duckie_state_median2.0963365489696085e-06
get_duckie_state_min2.0639783238292633e-06
get_robot_state_max0.0038233607337437104
get_robot_state_mean0.003727445048555347
get_robot_state_median0.003705513467399603
get_robot_state_min0.0036753925256784712
get_state_dump_max0.004879817279748972
get_state_dump_mean0.0047134114344054515
get_state_dump_median0.004677398119441277
get_state_dump_min0.0046190322189902785
get_ui_image_max0.03453837524941323
get_ui_image_mean0.030477642913742126
get_ui_image_median0.030138586681153157
get_ui_image_min0.027095023043248973
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.02771352332001622, "step_physics": 0.20563348246851532, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.00471238788220408, "get_robot_state": 0.003704246533701163, "sim_render-ego0": 0.0039163155917025525, "get_duckie_state": 2.0997113331867e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.012368305239649637, "complete-iteration": 0.27082473253032546, "set_robot_commands": 0.0022250458560915017, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008437719472143473, "sim_compute_performance-ego0": 0.0020298362274551074}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.03453837524941323, "step_physics": 0.2629000243298914, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.0046190322189902785, "get_robot_state": 0.0036753925256784712, "sim_render-ego0": 0.0040018215068273995, "get_duckie_state": 2.0639783238292633e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.01299798994834576, "complete-iteration": 0.33851697264265557, "set_robot_commands": 0.0022524360018308515, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.011400418714321622, "sim_compute_performance-ego0": 0.002046104474031955}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.032563650042290095, "step_physics": 0.24355238800144116, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004879817279748972, "get_robot_state": 0.0038233607337437104, "sim_render-ego0": 0.004018668429639119, "get_duckie_state": 2.321255991202806e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.012120768986177088, "complete-iteration": 0.31452709669673773, "set_robot_commands": 0.002272857218161908, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.009093922242633903, "sim_compute_performance-ego0": 0.002105926891647707}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.027095023043248973, "step_physics": 0.18542496886082632, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004642408356678476, "get_robot_state": 0.003706780401098043, "sim_render-ego0": 0.003936084879129554, "get_duckie_state": 2.0929617647525175e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.012618778746491368, "complete-iteration": 0.2479386065623643, "set_robot_commands": 0.0022901269815843567, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.0061225926846290605, "sim_compute_performance-ego0": 0.002017672711069836}}
set_robot_commands_max0.0022901269815843567
set_robot_commands_mean0.0022601165144171547
set_robot_commands_median0.00226264660999638
set_robot_commands_min0.0022250458560915017
sim_compute_performance-ego0_max0.002105926891647707
sim_compute_performance-ego0_mean0.0020498850760511515
sim_compute_performance-ego0_median0.0020379703507435315
sim_compute_performance-ego0_min0.002017672711069836
sim_compute_sim_state_max0.011400418714321622
sim_compute_sim_state_mean0.008763663278432015
sim_compute_sim_state_median0.008765820857388688
sim_compute_sim_state_min0.0061225926846290605
sim_render-ego0_max0.004018668429639119
sim_render-ego0_mean0.003968222601824656
sim_render-ego0_median0.003968953192978477
sim_render-ego0_min0.0039163155917025525
simulation-passed1
step_physics_max0.2629000243298914
step_physics_mean0.22437771591516853
step_physics_median0.22459293523497825
step_physics_min0.18542496886082632
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5726810976Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:29:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.664724719975088
survival_time_median54.64999999999903
deviation-center-line_median1.704653281626158
in-drivable-lane_median21.099999999999596


other stats
agent_compute-ego0_max0.013766312777847177
agent_compute-ego0_mean0.012588950474231884
agent_compute-ego0_median0.01252510139931334
agent_compute-ego0_min0.01153928632045368
complete-iteration_max0.19087072414525488
complete-iteration_mean0.16720313580384308
complete-iteration_median0.17038310174393884
complete-iteration_min0.1371756155822398
deviation-center-line_max3.0994073678989618
deviation-center-line_mean1.8904888905421424
deviation-center-line_min1.0532416310172912
deviation-heading_max10.582205674039344
deviation-heading_mean7.310388692475323
deviation-heading_median7.506609150920779
deviation-heading_min3.646130794020392
driven_any_max7.693952999706301
driven_any_mean6.7249701181582
driven_any_median6.944310412270058
driven_any_min5.317306648386382
driven_lanedir_consec_max7.545622058952931
driven_lanedir_consec_mean4.414560762398376
driven_lanedir_consec_min2.783171550690398
driven_lanedir_max7.545622058952931
driven_lanedir_mean4.414560762398376
driven_lanedir_median3.664724719975088
driven_lanedir_min2.783171550690398
get_duckie_state_max1.4801406542724813e-06
get_duckie_state_mean1.2737471848827026e-06
get_duckie_state_median1.2460187622801065e-06
get_duckie_state_min1.122810560698116e-06
get_robot_state_max0.003978797140764654
get_robot_state_mean0.003611105705359939
get_robot_state_median0.003537447432187052
get_robot_state_min0.003390730816301
get_state_dump_max0.005012248637730133
get_state_dump_mean0.004638570055875233
get_state_dump_median0.004621953203441708
get_state_dump_min0.004298125178887386
get_ui_image_max0.0348947574299298
get_ui_image_mean0.029354874846847383
get_ui_image_median0.02924910117790369
get_ui_image_min0.02402653960165235
in-drivable-lane_max38.09999999999901
in-drivable-lane_mean20.07499999999955
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 6.6423011865159705, "get_ui_image": 0.026119054148866293, "step_physics": 0.0866430329938307, "survival_time": 49.299999999999336, "driven_lanedir": 3.986962541999499, "get_state_dump": 0.004326945259457543, "get_robot_state": 0.003390730816301, "sim_render-ego0": 0.003621208873080145, "get_duckie_state": 1.232191663624185e-06, "in-drivable-lane": 24.049999999999816, "deviation-heading": 3.646130794020392, "agent_compute-ego0": 0.011989942799947906, "complete-iteration": 0.1499849295785241, "set_robot_commands": 0.00198953492300851, "deviation-center-line": 1.0532416310172912, "driven_lanedir_consec": 3.986962541999499, "sim_compute_sim_state": 0.010029330442016973, "sim_compute_performance-ego0": 0.0017990582861315758}, "LF-norm-zigzag-000-ego0": {"driven_any": 5.317306648386382, "get_ui_image": 0.0348947574299298, "step_physics": 0.1141648795555944, "survival_time": 46.3999999999995, "driven_lanedir": 3.342486897950677, "get_state_dump": 0.005012248637730133, "get_robot_state": 0.003643706489045602, "sim_render-ego0": 0.00397673849140738, "get_duckie_state": 1.2598458609360285e-06, "in-drivable-lane": 18.14999999999938, "deviation-heading": 9.173096816849045, "agent_compute-ego0": 0.013060259998678776, "complete-iteration": 0.19087072414525488, "set_robot_commands": 0.0021795135781377457, "deviation-center-line": 1.858321672000113, "driven_lanedir_consec": 3.342486897950677, "sim_compute_sim_state": 0.0118055584869549, "sim_compute_performance-ego0": 0.00204657481740699}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.246319638024145, "get_ui_image": 0.03237914820694109, "step_physics": 0.11488704935497884, "survival_time": 59.99999999999873, "driven_lanedir": 2.783171550690398, "get_state_dump": 0.004916961147425872, "get_robot_state": 0.003978797140764654, "sim_render-ego0": 0.004148238108219651, "get_duckie_state": 1.4801406542724813e-06, "in-drivable-lane": 38.09999999999901, "deviation-heading": 5.840121484992513, "agent_compute-ego0": 0.013766312777847177, "complete-iteration": 0.1907812739093536, "set_robot_commands": 0.0023052476030106747, "deviation-center-line": 1.5509848912522028, "driven_lanedir_consec": 2.783171550690398, "sim_compute_sim_state": 0.01210518562227959, "sim_compute_performance-ego0": 0.0021957261278468506}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.693952999706301, "get_ui_image": 0.02402653960165235, "step_physics": 0.08067849811963694, "survival_time": 59.99999999999873, "driven_lanedir": 7.545622058952931, "get_state_dump": 0.004298125178887386, "get_robot_state": 0.003431188375328502, "sim_render-ego0": 0.003568615742666735, "get_duckie_state": 1.122810560698116e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.582205674039344, "agent_compute-ego0": 0.01153928632045368, "complete-iteration": 0.1371756155822398, "set_robot_commands": 0.0019737717313234454, "deviation-center-line": 3.0994073678989618, "driven_lanedir_consec": 7.545622058952931, "sim_compute_sim_state": 0.005764458002793998, "sim_compute_performance-ego0": 0.001820540448013293}}
set_robot_commands_max0.0023052476030106747
set_robot_commands_mean0.002112016958870094
set_robot_commands_median0.002084524250573128
set_robot_commands_min0.0019737717313234454
sim_compute_performance-ego0_max0.0021957261278468506
sim_compute_performance-ego0_mean0.001965474919849678
sim_compute_performance-ego0_median0.0019335576327101417
sim_compute_performance-ego0_min0.0017990582861315758
sim_compute_sim_state_max0.01210518562227959
sim_compute_sim_state_mean0.009926133138511366
sim_compute_sim_state_median0.010917444464485938
sim_compute_sim_state_min0.005764458002793998
sim_render-ego0_max0.004148238108219651
sim_render-ego0_mean0.0038287003038434777
sim_render-ego0_median0.003798973682243762
sim_render-ego0_min0.003568615742666735
simulation-passed1
step_physics_max0.11488704935497884
step_physics_mean0.09909336500601022
step_physics_median0.10040395627471256
step_physics_min0.08067849811963694
survival_time_max59.99999999999873
survival_time_mean53.92499999999907
survival_time_min46.3999999999995
No reset possible
5718010996Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:24:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.8485165924856366
survival_time_median41.0499999999997
deviation-center-line_median0.9059883620947816
in-drivable-lane_median10.674999999999692


other stats
agent_compute-ego0_max0.013479751870949014
agent_compute-ego0_mean0.01298622475271005
agent_compute-ego0_median0.012921987728434631
agent_compute-ego0_min0.012621171683021924
complete-iteration_max0.2017460049323316
complete-iteration_mean0.17480898961624575
complete-iteration_median0.1698418744232065
complete-iteration_min0.15780620468623846
deviation-center-line_max2.93372342163456
deviation-center-line_mean1.3530409983773033
deviation-center-line_min0.6664638476850905
deviation-heading_max7.1996933852617815
deviation-heading_mean3.5634264165499485
deviation-heading_median2.7134701704277457
deviation-heading_min1.6270719400825209
driven_any_max10.672664580804806
driven_any_mean6.621417786986354
driven_any_median6.418366324183636
driven_any_min2.9762739187733382
driven_lanedir_consec_max8.208488096617563
driven_lanedir_consec_mean3.879740001537144
driven_lanedir_consec_min1.6134387245597384
driven_lanedir_max8.208488096617563
driven_lanedir_mean3.879740001537144
driven_lanedir_median2.8485165924856366
driven_lanedir_min1.6134387245597384
get_duckie_state_max1.3821985674398785e-06
get_duckie_state_mean1.3445331130242475e-06
get_duckie_state_median1.3432588042128583e-06
get_duckie_state_min1.3094162762313955e-06
get_robot_state_max0.0039034599857887607
get_robot_state_mean0.0038056106745098864
get_robot_state_median0.003791136429533516
get_robot_state_min0.0037367098531837528
get_state_dump_max0.004962672836706393
get_state_dump_mean0.004696569972400877
get_state_dump_median0.004653951160244844
get_state_dump_min0.004515704732407428
get_ui_image_max0.036024687425145564
get_ui_image_mean0.03036874500140936
get_ui_image_median0.029472589618836552
get_ui_image_min0.02650511334281877
in-drivable-lane_max42.39999999999911
in-drivable-lane_mean17.487499999999645
in-drivable-lane_min6.200000000000088
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.771744758870993, "get_ui_image": 0.02674467649508913, "step_physics": 0.09165932342895718, "survival_time": 29.10000000000028, "driven_lanedir": 3.9877001245328376, "get_state_dump": 0.004515704732407428, "get_robot_state": 0.0037367098531837528, "sim_render-ego0": 0.003841964090258957, "get_duckie_state": 1.320912539447765e-06, "in-drivable-lane": 6.200000000000088, "deviation-heading": 2.468367098502538, "agent_compute-ego0": 0.012621171683021924, "complete-iteration": 0.15780620468623846, "set_robot_commands": 0.0022190005660670496, "deviation-center-line": 0.9908902376690276, "driven_lanedir_consec": 3.9877001245328376, "sim_compute_sim_state": 0.01035312845612881, "sim_compute_performance-ego0": 0.002028855001537918}, "LF-norm-zigzag-000-ego0": {"driven_any": 2.9762739187733382, "get_ui_image": 0.036024687425145564, "step_physics": 0.12387194080815482, "survival_time": 18.500000000000128, "driven_lanedir": 1.7093330604384358, "get_state_dump": 0.0045912143676428785, "get_robot_state": 0.0037666890177765304, "sim_render-ego0": 0.004015256773750737, "get_duckie_state": 1.3656050689779523e-06, "in-drivable-lane": 8.40000000000012, "deviation-heading": 1.6270719400825209, "agent_compute-ego0": 0.013144728951055729, "complete-iteration": 0.2017460049323316, "set_robot_commands": 0.002300091509549123, "deviation-center-line": 0.8210864865205353, "driven_lanedir_consec": 1.7093330604384358, "sim_compute_sim_state": 0.01185368591884397, "sim_compute_performance-ego0": 0.0020901552750415236}, "LF-norm-techtrack-000-ego0": {"driven_any": 8.06498788949628, "get_ui_image": 0.03220050274258397, "step_physics": 0.10426600287254974, "survival_time": 52.999999999999126, "driven_lanedir": 1.6134387245597384, "get_state_dump": 0.004962672836706393, "get_robot_state": 0.0039034599857887607, "sim_render-ego0": 0.004192932049807909, "get_duckie_state": 1.3821985674398785e-06, "in-drivable-lane": 42.39999999999911, "deviation-heading": 2.958573242352954, "agent_compute-ego0": 0.013479751870949014, "complete-iteration": 0.17892562275669915, "set_robot_commands": 0.0023207765609784357, "deviation-center-line": 0.6664638476850905, "driven_lanedir_consec": 1.6134387245597384, "sim_compute_sim_state": 0.011349471979383925, "sim_compute_performance-ego0": 0.0021508137759569117}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.672664580804806, "get_ui_image": 0.02650511334281877, "step_physics": 0.09814525027755495, "survival_time": 59.99999999999873, "driven_lanedir": 8.208488096617563, "get_state_dump": 0.00471668795284681, "get_robot_state": 0.0038155838412905017, "sim_render-ego0": 0.003943339077062551, "get_duckie_state": 1.3094162762313955e-06, "in-drivable-lane": 12.949999999999264, "deviation-heading": 7.1996933852617815, "agent_compute-ego0": 0.012699246505813534, "complete-iteration": 0.16075812608971385, "set_robot_commands": 0.002329474781871735, "deviation-center-line": 2.93372342163456, "driven_lanedir_consec": 8.208488096617563, "sim_compute_sim_state": 0.006456089853545609, "sim_compute_performance-ego0": 0.002055690052308806}}
set_robot_commands_max0.002329474781871735
set_robot_commands_mean0.0022923358546165858
set_robot_commands_median0.0023104340352637794
set_robot_commands_min0.0022190005660670496
sim_compute_performance-ego0_max0.0021508137759569117
sim_compute_performance-ego0_mean0.00208137852621129
sim_compute_performance-ego0_median0.002072922663675165
sim_compute_performance-ego0_min0.002028855001537918
sim_compute_sim_state_max0.01185368591884397
sim_compute_sim_state_mean0.010003094051975575
sim_compute_sim_state_median0.010851300217756366
sim_compute_sim_state_min0.006456089853545609
sim_render-ego0_max0.004192932049807909
sim_render-ego0_mean0.003998372997720039
sim_render-ego0_median0.003979297925406644
sim_render-ego0_min0.003841964090258957
simulation-passed1
step_physics_max0.12387194080815482
step_physics_mean0.10448562934680417
step_physics_median0.10120562657505236
step_physics_min0.09165932342895718
survival_time_max59.99999999999873
survival_time_mean40.149999999999565
survival_time_min18.500000000000128
No reset possible
5717311010Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-080:01:20
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5716511312Moustafa Elarabitemplate-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-080:00:53
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5698811048Bhavya Patwa 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:30:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.173242640267555
survival_time_median59.99999999999873
deviation-center-line_median3.647554008962704
in-drivable-lane_median10.84999999999986


other stats
agent_compute-ego0_max0.012541059252622224
agent_compute-ego0_mean0.01226988266837626
agent_compute-ego0_median0.012403150283724542
agent_compute-ego0_min0.01173217085343373
complete-iteration_max0.1953904380607764
complete-iteration_mean0.17155570916212343
complete-iteration_median0.1689548526179483
complete-iteration_min0.1529226933518207
deviation-center-line_max5.140050691293736
deviation-center-line_mean3.512387013641407
deviation-center-line_min1.614389345346485
deviation-heading_max15.83415721593997
deviation-heading_mean9.7021826785993
deviation-heading_median9.374361111835306
deviation-heading_min4.225851274786618
driven_any_max8.76311785652741
driven_any_mean7.373047768103304
driven_any_median7.9054482241217165
driven_any_min4.918176767642372
driven_lanedir_consec_max8.16615706174564
driven_lanedir_consec_mean6.069637596600347
driven_lanedir_consec_min3.765908044120641
driven_lanedir_max8.16615706174564
driven_lanedir_mean6.069637596600347
driven_lanedir_median6.173242640267555
driven_lanedir_min3.765908044120641
get_duckie_state_max1.3143791941977064e-06
get_duckie_state_mean1.246567586273385e-06
get_duckie_state_median1.2326895644722337e-06
get_duckie_state_min1.206512021951367e-06
get_robot_state_max0.003673204077372047
get_robot_state_mean0.0035483947895686837
get_robot_state_median0.0035898454183345037
get_robot_state_min0.00334068424423368
get_state_dump_max0.004664181471863555
get_state_dump_mean0.004451371470871171
get_state_dump_median0.0044365445342687245
get_state_dump_min0.004268215343083678
get_ui_image_max0.03487822812959415
get_ui_image_mean0.02969026244651297
get_ui_image_median0.028923063452892955
get_ui_image_min0.026036694750671815
in-drivable-lane_max17.700000000000212
in-drivable-lane_mean10.062499999999982
in-drivable-lane_min0.849999999999997
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.918176767642372, "get_ui_image": 0.026036694750671815, "step_physics": 0.09053862882901176, "survival_time": 35.50000000000012, "driven_lanedir": 3.765908044120641, "get_state_dump": 0.004268215343083678, "get_robot_state": 0.00334068424423368, "sim_render-ego0": 0.0035568101831964633, "get_duckie_state": 1.206512021951367e-06, "in-drivable-lane": 11.049999999999908, "deviation-heading": 4.225851274786618, "agent_compute-ego0": 0.01173217085343373, "complete-iteration": 0.1529226933518207, "set_robot_commands": 0.001991677720167801, "deviation-center-line": 1.614389345346485, "driven_lanedir_consec": 3.765908044120641, "sim_compute_sim_state": 0.009589512472246604, "sim_compute_performance-ego0": 0.0017895802499205084}, "LF-norm-zigzag-000-ego0": {"driven_any": 7.425932995360117, "get_ui_image": 0.03487822812959415, "step_physics": 0.12031988855404818, "survival_time": 59.99999999999873, "driven_lanedir": 5.994190749773972, "get_state_dump": 0.004473718576486859, "get_robot_state": 0.003610318546787488, "sim_render-ego0": 0.0038731328454442463, "get_duckie_state": 1.254427145164674e-06, "in-drivable-lane": 10.64999999999981, "deviation-heading": 15.83415721593997, "agent_compute-ego0": 0.012541059252622224, "complete-iteration": 0.1953904380607764, "set_robot_commands": 0.0022032481248333096, "deviation-center-line": 4.620539475279912, "driven_lanedir_consec": 5.994190749773972, "sim_compute_sim_state": 0.011378353184009968, "sim_compute_performance-ego0": 0.0020281637241004605}, "LF-norm-techtrack-000-ego0": {"driven_any": 8.76311785652741, "get_ui_image": 0.031087477737223475, "step_physics": 0.10509311726051604, "survival_time": 59.99999999999873, "driven_lanedir": 6.352294530761137, "get_state_dump": 0.00439937049205059, "get_robot_state": 0.0035693722898815197, "sim_render-ego0": 0.003791587140339797, "get_duckie_state": 1.2109519837797928e-06, "in-drivable-lane": 17.700000000000212, "deviation-heading": 8.567162529529975, "agent_compute-ego0": 0.01229676338754824, "complete-iteration": 0.1761190686793649, "set_robot_commands": 0.00213977041887701, "deviation-center-line": 2.6745685426454964, "driven_lanedir_consec": 6.352294530761137, "sim_compute_sim_state": 0.01167346118987351, "sim_compute_performance-ego0": 0.001984607766411088}, "LF-norm-small_loop-000-ego0": {"driven_any": 8.384963452883316, "get_ui_image": 0.02675864916856243, "step_physics": 0.09997260362083568, "survival_time": 59.99999999999873, "driven_lanedir": 8.16615706174564, "get_state_dump": 0.004664181471863555, "get_robot_state": 0.003673204077372047, "sim_render-ego0": 0.003833032269759738, "get_duckie_state": 1.3143791941977064e-06, "in-drivable-lane": 0.849999999999997, "deviation-heading": 10.181559694140637, "agent_compute-ego0": 0.012509537179900844, "complete-iteration": 0.16179063655653167, "set_robot_commands": 0.0021937338140584546, "deviation-center-line": 5.140050691293736, "driven_lanedir_consec": 8.16615706174564, "sim_compute_sim_state": 0.006155946868146885, "sim_compute_performance-ego0": 0.0019422548200367493}}
set_robot_commands_max0.0022032481248333096
set_robot_commands_mean0.002132107519484144
set_robot_commands_median0.002166752116467732
set_robot_commands_min0.001991677720167801
sim_compute_performance-ego0_max0.0020281637241004605
sim_compute_performance-ego0_mean0.0019361516401172015
sim_compute_performance-ego0_median0.0019634312932239185
sim_compute_performance-ego0_min0.0017895802499205084
sim_compute_sim_state_max0.01167346118987351
sim_compute_sim_state_mean0.00969931842856924
sim_compute_sim_state_median0.010483932828128286
sim_compute_sim_state_min0.006155946868146885
sim_render-ego0_max0.0038731328454442463
sim_render-ego0_mean0.0037636406096850615
sim_render-ego0_median0.003812309705049767
sim_render-ego0_min0.0035568101831964633
simulation-passed1
step_physics_max0.12031988855404818
step_physics_mean0.10398105956610292
step_physics_median0.10253286044067586
step_physics_min0.09053862882901176
survival_time_max59.99999999999873
survival_time_mean53.87499999999907
survival_time_min35.50000000000012
No reset possible
5696911313Moustafa Elarabitemplate-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-080:02:48
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5689711066Fernanda Custodio Pereira do Carmo 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:15:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.353184771484048
survival_time_median26.17500000000024
deviation-center-line_median0.895893184101168
in-drivable-lane_median11.7500000000001


other stats
agent_compute-ego0_max0.0133798703517349
agent_compute-ego0_mean0.012814201526065478
agent_compute-ego0_median0.012712700388878275
agent_compute-ego0_min0.012451534974770468
complete-iteration_max0.19757708710255548
complete-iteration_mean0.17092368273262362
complete-iteration_median0.16524448894411636
complete-iteration_min0.15562866593970628
deviation-center-line_max1.173066371629531
deviation-center-line_mean0.825171079149799
deviation-center-line_min0.335831576767329
deviation-heading_max3.073143358824702
deviation-heading_mean2.2345852315024426
deviation-heading_median2.2537645042293124
deviation-heading_min1.3576685587264448
driven_any_max5.317952111129667
driven_any_mean4.2031296335159585
driven_any_median4.294875675817687
driven_any_min2.904815071298793
driven_lanedir_consec_max3.981858153837907
driven_lanedir_consec_mean2.5862265917274945
driven_lanedir_consec_min1.6566786701039755
driven_lanedir_max3.981858153837907
driven_lanedir_mean2.5862265917274945
driven_lanedir_median2.353184771484048
driven_lanedir_min1.6566786701039755
get_duckie_state_max1.451089268638974e-06
get_duckie_state_mean1.3485554482761008e-06
get_duckie_state_median1.330888317183226e-06
get_duckie_state_min1.2813558900989773e-06
get_robot_state_max0.0038552468892925143
get_robot_state_mean0.003696612907555183
get_robot_state_median0.0036958134832743273
get_robot_state_min0.0035395777743795643
get_state_dump_max0.004699771659520873
get_state_dump_mean0.00458551563659925
get_state_dump_median0.004615083094293674
get_state_dump_min0.004412124698288775
get_ui_image_max0.03643911520277992
get_ui_image_mean0.030106793085913915
get_ui_image_median0.029114236177206865
get_ui_image_min0.025759584786462
in-drivable-lane_max14.400000000000205
in-drivable-lane_mean12.000000000000137
in-drivable-lane_min10.100000000000144
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.281345279700124, "get_ui_image": 0.028417503549939112, "step_physics": 0.09416332663524718, "survival_time": 33.55000000000023, "driven_lanedir": 3.981858153837907, "get_state_dump": 0.004624078316347939, "get_robot_state": 0.003685760356131054, "sim_render-ego0": 0.0039581383268038435, "get_duckie_state": 1.451089268638974e-06, "in-drivable-lane": 11.800000000000058, "deviation-heading": 2.0710219508100054, "agent_compute-ego0": 0.01265444393668856, "complete-iteration": 0.16266119373696192, "set_robot_commands": 0.0022228888812519257, "deviation-center-line": 0.9462030994447496, "driven_lanedir_consec": 3.981858153837907, "sim_compute_sim_state": 0.010827080479690008, "sim_compute_performance-ego0": 0.002020866388366336}, "LF-norm-zigzag-000-ego0": {"driven_any": 3.3084060719352504, "get_ui_image": 0.03643911520277992, "step_physics": 0.11849489624788116, "survival_time": 21.900000000000176, "driven_lanedir": 1.6566786701039755, "get_state_dump": 0.004699771659520873, "get_robot_state": 0.0038552468892925143, "sim_render-ego0": 0.004175932369362521, "get_duckie_state": 1.336012994509895e-06, "in-drivable-lane": 11.70000000000014, "deviation-heading": 3.073143358824702, "agent_compute-ego0": 0.0133798703517349, "complete-iteration": 0.19757708710255548, "set_robot_commands": 0.002387398196244294, "deviation-center-line": 0.8455832687575866, "driven_lanedir_consec": 1.6566786701039755, "sim_compute_sim_state": 0.01190680414777682, "sim_compute_performance-ego0": 0.00215076802803335}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.904815071298793, "get_ui_image": 0.029810968804474616, "step_physics": 0.09888665745223778, "survival_time": 20.65000000000016, "driven_lanedir": 1.8096128206911717, "get_state_dump": 0.004412124698288775, "get_robot_state": 0.0035395777743795643, "sim_render-ego0": 0.003823491686208237, "get_duckie_state": 1.2813558900989773e-06, "in-drivable-lane": 10.100000000000144, "deviation-heading": 1.3576685587264448, "agent_compute-ego0": 0.012770956841067991, "complete-iteration": 0.16782778415127078, "set_robot_commands": 0.002107707198691253, "deviation-center-line": 0.335831576767329, "driven_lanedir_consec": 1.8096128206911717, "sim_compute_sim_state": 0.010481009160838842, "sim_compute_performance-ego0": 0.0019144977348438208}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.317952111129667, "get_ui_image": 0.025759584786462, "step_physics": 0.09490153906775305, "survival_time": 30.450000000000298, "driven_lanedir": 2.8967567222769244, "get_state_dump": 0.00460608787223941, "get_robot_state": 0.003705866610417601, "sim_render-ego0": 0.003871301353954878, "get_duckie_state": 1.3257636398565574e-06, "in-drivable-lane": 14.400000000000205, "deviation-heading": 2.4365070576486194, "agent_compute-ego0": 0.012451534974770468, "complete-iteration": 0.15562866593970628, "set_robot_commands": 0.002213138048766089, "deviation-center-line": 1.173066371629531, "driven_lanedir_consec": 2.8967567222769244, "sim_compute_sim_state": 0.006031897810638928, "sim_compute_performance-ego0": 0.002004937656590196}}
set_robot_commands_max0.002387398196244294
set_robot_commands_mean0.0022327830812383905
set_robot_commands_median0.002218013465009007
set_robot_commands_min0.002107707198691253
sim_compute_performance-ego0_max0.00215076802803335
sim_compute_performance-ego0_mean0.0020227674519584255
sim_compute_performance-ego0_median0.0020129020224782662
sim_compute_performance-ego0_min0.0019144977348438208
sim_compute_sim_state_max0.01190680414777682
sim_compute_sim_state_mean0.009811697899736148
sim_compute_sim_state_median0.010654044820264424
sim_compute_sim_state_min0.006031897810638928
sim_render-ego0_max0.004175932369362521
sim_render-ego0_mean0.0039572159340823705
sim_render-ego0_median0.00391471984037936
sim_render-ego0_min0.003823491686208237
simulation-passed1
step_physics_max0.11849489624788116
step_physics_mean0.10161160485077976
step_physics_median0.0968940982599954
step_physics_min0.09416332663524718
survival_time_max33.55000000000023
survival_time_mean26.637500000000216
survival_time_min20.65000000000016
No reset possible
5687211068Étienne Boucher 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:04:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6700312233000871
survival_time_median5.524999999999988
deviation-center-line_median0.122437737984433
in-drivable-lane_median3.0500000000000025


other stats
agent_compute-ego0_max0.012281858353387742
agent_compute-ego0_mean0.011647940990004298
agent_compute-ego0_median0.011636526571155268
agent_compute-ego0_min0.011036852464318911
complete-iteration_max0.19255169729391733
complete-iteration_mean0.17115466670011567
complete-iteration_median0.1699466312329278
complete-iteration_min0.15217370704068975
deviation-center-line_max0.421681577226755
deviation-center-line_mean0.1911256316825894
deviation-center-line_min0.09794547353473658
deviation-heading_max1.2791461955432477
deviation-heading_mean0.8313292588533552
deviation-heading_median0.8321638686015694
deviation-heading_min0.3818431026670343
driven_any_max2.886066831271636
driven_any_mean1.8247567482492997
driven_any_median1.5583383821951786
driven_any_min1.2962833973352053
driven_lanedir_consec_max1.9680640786625685
driven_lanedir_consec_mean0.9161194738653808
driven_lanedir_consec_min0.35635137019878105
driven_lanedir_max1.9680640786625685
driven_lanedir_mean0.9161194738653808
driven_lanedir_median0.6700312233000871
driven_lanedir_min0.35635137019878105
get_duckie_state_max1.7335859395689884e-06
get_duckie_state_mean1.5981932572584825e-06
get_duckie_state_median1.5907443022041911e-06
get_duckie_state_min1.477698485056559e-06
get_robot_state_max0.0034950567504106944
get_robot_state_mean0.003422966568641813
get_robot_state_median0.0034323332477896942
get_robot_state_min0.003332143028577169
get_state_dump_max0.004540195919218518
get_state_dump_mean0.004397461204142437
get_state_dump_median0.004383907206146057
get_state_dump_min0.004281834485059116
get_ui_image_max0.03338239093621572
get_ui_image_mean0.029167456831502084
get_ui_image_median0.028853464588003055
get_ui_image_min0.025580507213786494
in-drivable-lane_max3.4999999999999876
in-drivable-lane_mean3.1124999999999954
in-drivable-lane_min2.84999999999999
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.886066831271636, "get_ui_image": 0.02683858055481936, "step_physics": 0.09603565771949482, "survival_time": 9.299999999999995, "driven_lanedir": 1.9680640786625685, "get_state_dump": 0.004281834485059116, "get_robot_state": 0.0034075675801159863, "sim_render-ego0": 0.0035632166632993973, "get_duckie_state": 1.5784074915921624e-06, "in-drivable-lane": 3.150000000000011, "deviation-heading": 1.2745259472235355, "agent_compute-ego0": 0.011036852464318911, "complete-iteration": 0.1580273977575455, "set_robot_commands": 0.002063044889725466, "deviation-center-line": 0.421681577226755, "driven_lanedir_consec": 1.9680640786625685, "sim_compute_sim_state": 0.008868815427157967, "sim_compute_performance-ego0": 0.0018543368354838163}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.2962833973352053, "get_ui_image": 0.03338239093621572, "step_physics": 0.1238737478852272, "survival_time": 4.749999999999991, "driven_lanedir": 0.35635137019878105, "get_state_dump": 0.004283609489599864, "get_robot_state": 0.003332143028577169, "sim_render-ego0": 0.003458954393863678, "get_duckie_state": 1.477698485056559e-06, "in-drivable-lane": 2.9499999999999935, "deviation-heading": 1.2791461955432477, "agent_compute-ego0": 0.01170496145884196, "complete-iteration": 0.19255169729391733, "set_robot_commands": 0.0019825945297876992, "deviation-center-line": 0.1341830980567997, "driven_lanedir_consec": 0.35635137019878105, "sim_compute_sim_state": 0.008713973065217337, "sim_compute_performance-ego0": 0.0017428969343503316}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.4360499039041343, "get_ui_image": 0.030868348621186755, "step_physics": 0.11538815271286736, "survival_time": 5.1999999999999895, "driven_lanedir": 0.6622415048578161, "get_state_dump": 0.004540195919218518, "get_robot_state": 0.003457098915463402, "sim_render-ego0": 0.0037086191631498793, "get_duckie_state": 1.6030811128162205e-06, "in-drivable-lane": 2.84999999999999, "deviation-heading": 0.3898017899796031, "agent_compute-ego0": 0.012281858353387742, "complete-iteration": 0.1818658647083101, "set_robot_commands": 0.0020562580653599332, "deviation-center-line": 0.09794547353473658, "driven_lanedir_consec": 0.6622415048578161, "sim_compute_sim_state": 0.007611485889979772, "sim_compute_performance-ego0": 0.0018724895658947171}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6806268604862231, "get_ui_image": 0.025580507213786494, "step_physics": 0.09462255946660446, "survival_time": 5.849999999999987, "driven_lanedir": 0.6778209417423582, "get_state_dump": 0.00448420492269225, "get_robot_state": 0.0034950567504106944, "sim_render-ego0": 0.0036543991606114274, "get_duckie_state": 1.7335859395689884e-06, "in-drivable-lane": 3.4999999999999876, "deviation-heading": 0.3818431026670343, "agent_compute-ego0": 0.011568091683468576, "complete-iteration": 0.15217370704068975, "set_robot_commands": 0.002134888859118446, "deviation-center-line": 0.11069237791206632, "driven_lanedir_consec": 0.6778209417423582, "sim_compute_sim_state": 0.0047173742520607125, "sim_compute_performance-ego0": 0.0018358291205713305}}
set_robot_commands_max0.002134888859118446
set_robot_commands_mean0.0020591965859978857
set_robot_commands_median0.0020596514775427
set_robot_commands_min0.0019825945297876992
sim_compute_performance-ego0_max0.0018724895658947171
sim_compute_performance-ego0_mean0.0018263881140750489
sim_compute_performance-ego0_median0.0018450829780275737
sim_compute_performance-ego0_min0.0017428969343503316
sim_compute_sim_state_max0.008868815427157967
sim_compute_sim_state_mean0.007477912158603947
sim_compute_sim_state_median0.008162729477598555
sim_compute_sim_state_min0.0047173742520607125
sim_render-ego0_max0.0037086191631498793
sim_render-ego0_mean0.0035962973452310955
sim_render-ego0_median0.003608807911955413
sim_render-ego0_min0.003458954393863678
simulation-passed1
step_physics_max0.1238737478852272
step_physics_mean0.10748002944604848
step_physics_median0.1057119052161811
step_physics_min0.09462255946660446
survival_time_max9.299999999999995
survival_time_mean6.2749999999999915
survival_time_min4.749999999999991
No reset possible
5680411425Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-080:05:25
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5653711107Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-080:34:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.981948558062888
survival_time_median59.99999999999873
deviation-center-line_median1.842218950247119
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.012825676245455142
agent_compute-ego0_mean0.012259598427469985
agent_compute-ego0_median0.012170991532312244
agent_compute-ego0_min0.011870734399800296
complete-iteration_max0.17306864410514736
complete-iteration_mean0.1708872619516943
complete-iteration_median0.17086980573144384
complete-iteration_min0.16874079223874208
deviation-center-line_max2.9145613140927384
deviation-center-line_mean2.0605622534696018
deviation-center-line_min1.6432497992914303
deviation-heading_max5.8529201469616545
deviation-heading_mean4.254307056179646
deviation-heading_median4.36391055930602
deviation-heading_min2.436486959144886
driven_any_max4.002342130898504
driven_any_mean4.001496682733472
driven_any_median4.002333742167718
driven_any_min3.9989771156999465
driven_lanedir_consec_max3.996989243274156
driven_lanedir_consec_mean3.982317411386577
driven_lanedir_consec_min3.968383286146375
driven_lanedir_max3.996989243274156
driven_lanedir_mean3.982317411386577
driven_lanedir_median3.981948558062888
driven_lanedir_min3.968383286146375
get_duckie_state_max1.9500297273227716e-06
get_duckie_state_mean1.7860549177158682e-06
get_duckie_state_median1.7894297019329594e-06
get_duckie_state_min1.6153305396747828e-06
get_robot_state_max0.0038469040225090137
get_robot_state_mean0.0035884882091582566
get_robot_state_median0.0035711336294677633
get_robot_state_min0.003364781555188486
get_state_dump_max0.004751406144738495
get_state_dump_mean0.004550329652257407
get_state_dump_median0.0045885798536073556
get_state_dump_min0.004272752757076419
get_ui_image_max0.03175951658339425
get_ui_image_mean0.02861295319119659
get_ui_image_median0.02827402614336228
get_ui_image_min0.02614424389466755
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.002342130898504, "get_ui_image": 0.02614424389466755, "step_physics": 0.1047605078583653, "survival_time": 59.99999999999873, "driven_lanedir": 3.9859726017023807, "get_state_dump": 0.004548653476343464, "get_robot_state": 0.0035177567519315773, "sim_render-ego0": 0.0037719998927437993, "get_duckie_state": 1.9085397331244144e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.728574422385836, "agent_compute-ego0": 0.011870734399800296, "complete-iteration": 0.16874079223874208, "set_robot_commands": 0.0022728939437548583, "deviation-center-line": 2.9145613140927384, "driven_lanedir_consec": 3.9859726017023807, "sim_compute_sim_state": 0.009816664243915694, "sim_compute_performance-ego0": 0.0019488654267678749}, "LF-norm-zigzag-000-ego0": {"driven_any": 4.002333622348765, "get_ui_image": 0.03175951658339425, "step_physics": 0.10410507731791042, "survival_time": 59.99999999999873, "driven_lanedir": 3.9779245144233952, "get_state_dump": 0.004272752757076419, "get_robot_state": 0.003364781555188486, "sim_render-ego0": 0.0035888559117503807, "get_duckie_state": 1.6153305396747828e-06, "in-drivable-lane": 0.0, "deviation-heading": 4.999246696226204, "agent_compute-ego0": 0.011933409105629646, "complete-iteration": 0.17306864410514736, "set_robot_commands": 0.0019809743149095927, "deviation-center-line": 1.8357936332029765, "driven_lanedir_consec": 3.9779245144233952, "sim_compute_sim_state": 0.010192956058905585, "sim_compute_performance-ego0": 0.001787131672397839}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.9989771156999465, "get_ui_image": 0.030032935686452895, "step_physics": 0.10290983217542714, "survival_time": 59.99999999999873, "driven_lanedir": 3.968383286146375, "get_state_dump": 0.004628506230871247, "get_robot_state": 0.0036245105070039497, "sim_render-ego0": 0.003863682854086235, "get_duckie_state": 1.6703196707415044e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.8529201469616545, "agent_compute-ego0": 0.012825676245455142, "complete-iteration": 0.17291640540543046, "set_robot_commands": 0.0021721923678840425, "deviation-center-line": 1.6432497992914303, "driven_lanedir_consec": 3.968383286146375, "sim_compute_sim_state": 0.01077545969611302, "sim_compute_performance-ego0": 0.0019933377376305473}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.0023338619866715, "get_ui_image": 0.026515116600271664, "step_physics": 0.10636354247100348, "survival_time": 59.99999999999873, "driven_lanedir": 3.996989243274156, "get_state_dump": 0.004751406144738495, "get_robot_state": 0.0038469040225090137, "sim_render-ego0": 0.004033842849096192, "get_duckie_state": 1.9500297273227716e-06, "in-drivable-lane": 0.0, "deviation-heading": 2.436486959144886, "agent_compute-ego0": 0.012408573958994844, "complete-iteration": 0.1688232060574572, "set_robot_commands": 0.0023159452719454165, "deviation-center-line": 1.8486442672912613, "driven_lanedir_consec": 3.996989243274156, "sim_compute_sim_state": 0.006395451135182758, "sim_compute_performance-ego0": 0.002102274779574658}}
set_robot_commands_max0.0023159452719454165
set_robot_commands_mean0.002185501474623478
set_robot_commands_median0.0022225431558194507
set_robot_commands_min0.0019809743149095927
sim_compute_performance-ego0_max0.002102274779574658
sim_compute_performance-ego0_mean0.00195790240409273
sim_compute_performance-ego0_median0.001971101582199211
sim_compute_performance-ego0_min0.001787131672397839
sim_compute_sim_state_max0.01077545969611302
sim_compute_sim_state_mean0.009295132783529263
sim_compute_sim_state_median0.01000481015141064
sim_compute_sim_state_min0.006395451135182758
sim_render-ego0_max0.004033842849096192
sim_render-ego0_mean0.003814595376919152
sim_render-ego0_median0.0038178413734150174
sim_render-ego0_min0.0035888559117503807
simulation-passed1
step_physics_max0.10636354247100348
step_physics_mean0.10453473995567658
step_physics_median0.10443279258813788
step_physics_min0.10290983217542714
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5648811117Daniil Lisussim-exercise-1aido5-LF-sim-validationLFv-simfailedyesnogpu-prod-080:04:19
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5645011278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:24
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5641011282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:09
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5637011430Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFV-sim-validationLFVv-simfailedyesnogpu-prod-080:01:37
The container "solut [...]
The container "solution-ego0" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5634711281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:39
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5632511427Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-080:00:52
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5630111283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5622811287Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5616211289Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:22
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5615611289Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5611511285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:11
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5604611299Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5597611290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:03:57
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5594911302Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:53
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5591911301Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:09
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5587911284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:02:13
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5572111160Ayman Shams 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simfailedyesnogpu-prod-080:07:24
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5568911311Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-080:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||