Duckietown Challenges Home Challenges Submissions

Evaluator 4891

ID4891
evaluatornogpu-prod-07
ownerI don't have one 😀
machinenogpu-prod_cab7f1cb43e6
processnogpu-prod-07_cab7f1cb43e6
version6.2.5
first heard
last heard
statusinactive
# evaluating
# success17 53944
# timeout1 57581
# failed
# error1 54027
# aborted1 57567
# host-error33 53925
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5758110889Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simtimeoutyesnogpu-prod-07----No reset possible
5756711012Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simabortedyesnogpu-prod-070:02:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 45, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 45, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5741110940Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:42:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.0
survival_time_median59.99999999999873
deviation-center-line_median1.2422730096440104
in-drivable-lane_median0.0


other stats
agent_compute-ego0_max0.013444335335597309
agent_compute-ego0_mean0.012219898011861095
agent_compute-ego0_median0.011976053970838764
agent_compute-ego0_min0.011483148770169552
complete-iteration_max0.34377487196116324
complete-iteration_mean0.2899975616370113
complete-iteration_median0.2888150427561815
complete-iteration_min0.238585289074519
deviation-center-line_max4.053503393024394
deviation-center-line_mean1.731069028434027
deviation-center-line_min0.386226701423694
deviation-heading_max27.859809596736422
deviation-heading_mean14.925250437835436
deviation-heading_median14.309269950178932
deviation-heading_min3.22265225424745
driven_any_max2.6645352591003757e-13
driven_any_mean1.9984014443252818e-13
driven_any_median2.6645352591003757e-13
driven_any_min0.0
driven_lanedir_consec_max0.000286102294921875
driven_lanedir_consec_mean7.152557373046875e-05
driven_lanedir_consec_min0.0
driven_lanedir_max0.000286102294921875
driven_lanedir_mean7.152557373046875e-05
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max1.3939843983773289e-06
get_duckie_state_mean1.3102599722856685e-06
get_duckie_state_median1.3135851273230966e-06
get_duckie_state_min1.219885236119152e-06
get_robot_state_max0.003733983742605141
get_robot_state_mean0.003693517945787491
get_robot_state_median0.0036885417768302113
get_robot_state_min0.003663004486884404
get_state_dump_max0.004810454545668222
get_state_dump_mean0.004686209681032103
get_state_dump_median0.0046719294801342
get_state_dump_min0.00459052521819179
get_ui_image_max0.03504963600069756
get_ui_image_mean0.030222128273346936
get_ui_image_median0.029871438861786573
get_ui_image_min0.026095999369117044
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.027726205957620764, "step_physics": 0.20519530961753724, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004810454545668222, "get_robot_state": 0.003733983742605141, "sim_render-ego0": 0.003944597673058808, "get_duckie_state": 1.3939843983773289e-06, "in-drivable-lane": 0.0, "deviation-heading": 22.66279310353771, "agent_compute-ego0": 0.011697693728685976, "complete-iteration": 0.2699235492105984, "set_robot_commands": 0.0022461904673453273, "deviation-center-line": 4.053503393024394, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.00844441544106362, "sim_compute_performance-ego0": 0.00203983849232441}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.0, "get_ui_image": 0.03504963600069756, "step_physics": 0.26736145769924446, "survival_time": 59.99999999999873, "driven_lanedir": 0.000286102294921875, "get_state_dump": 0.004728774841778681, "get_robot_state": 0.003663004486884404, "sim_render-ego0": 0.003881406426727523, "get_duckie_state": 1.3493181366805332e-06, "in-drivable-lane": 0.0, "deviation-heading": 27.859809596736422, "agent_compute-ego0": 0.013444335335597309, "complete-iteration": 0.34377487196116324, "set_robot_commands": 0.0021506163798005852, "deviation-center-line": 1.0457540566888746, "driven_lanedir_consec": 0.000286102294921875, "sim_compute_sim_state": 0.01145280965857462, "sim_compute_performance-ego0": 0.001961562159059446}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.03201667176595238, "step_physics": 0.23908648522668435, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.004615084118489719, "get_robot_state": 0.0036725936384621903, "sim_render-ego0": 0.003859848106632026, "get_duckie_state": 1.219885236119152e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.22265225424745, "agent_compute-ego0": 0.011483148770169552, "complete-iteration": 0.30770653630176453, "set_robot_commands": 0.002174953933957217, "deviation-center-line": 1.4387919625991463, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.008775836522136501, "sim_compute_performance-ego0": 0.0019426738888298244}, "LF-norm-small_loop-000-ego0": {"driven_any": 2.6645352591003757e-13, "get_ui_image": 0.026095999369117044, "step_physics": 0.17786056949733, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "get_state_dump": 0.00459052521819179, "get_robot_state": 0.003704489915198231, "sim_render-ego0": 0.0038212169516989823, "get_duckie_state": 1.27785211796566e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.955746796820156, "agent_compute-ego0": 0.012254414212991557, "complete-iteration": 0.238585289074519, "set_robot_commands": 0.0021691288578817984, "deviation-center-line": 0.386226701423694, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.006032371203369344, "sim_compute_performance-ego0": 0.001977682113647461}}
set_robot_commands_max0.0022461904673453273
set_robot_commands_mean0.002185222409746232
set_robot_commands_median0.0021720413959195077
set_robot_commands_min0.0021506163798005852
sim_compute_performance-ego0_max0.00203983849232441
sim_compute_performance-ego0_mean0.0019804391634652856
sim_compute_performance-ego0_median0.0019696221363534536
sim_compute_performance-ego0_min0.0019426738888298244
sim_compute_sim_state_max0.01145280965857462
sim_compute_sim_state_mean0.008676358206286022
sim_compute_sim_state_median0.008610125981600061
sim_compute_sim_state_min0.006032371203369344
sim_render-ego0_max0.003944597673058808
sim_render-ego0_mean0.003876767289529335
sim_render-ego0_median0.003870627266679775
sim_render-ego0_min0.0038212169516989823
simulation-passed1
step_physics_max0.26736145769924446
step_physics_mean0.222375955510199
step_physics_median0.2221408974221108
step_physics_min0.17786056949733
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5733410957Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:10:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.9678598788319768
survival_time_median17.725000000000115
deviation-center-line_median0.7009870215928671
in-drivable-lane_median4.925000000000045


other stats
agent_compute-ego0_max0.03220234224506096
agent_compute-ego0_mean0.01753283384935216
agent_compute-ego0_median0.01268964821626896
agent_compute-ego0_min0.012549696719809754
complete-iteration_max0.21289318351335423
complete-iteration_mean0.17600308152177715
complete-iteration_median0.1686254051363269
complete-iteration_min0.15386833230110064
deviation-center-line_max1.1016798121984852
deviation-center-line_mean0.6544526696958486
deviation-center-line_min0.11415682339917506
deviation-heading_max4.6042144100930615
deviation-heading_mean2.35312693494661
deviation-heading_median1.9694778160722928
deviation-heading_min0.8693376975487915
driven_any_max4.94374098159553
driven_any_mean2.8886111801587915
driven_any_median2.9869197413475357
driven_any_min0.636864256344564
driven_lanedir_consec_max2.364252145580529
driven_lanedir_consec_mean1.6298013362454498
driven_lanedir_consec_min0.21923344173731651
driven_lanedir_max2.364252145580529
driven_lanedir_mean1.641192487141415
driven_lanedir_median1.990642180623907
driven_lanedir_min0.21923344173731651
get_duckie_state_max1.2331111456758232e-06
get_duckie_state_mean1.2050317602141832e-06
get_duckie_state_median1.2028671716777536e-06
get_duckie_state_min1.1812815518254028e-06
get_robot_state_max0.0035228871502569013
get_robot_state_mean0.0035015253850637024
get_robot_state_median0.003508463342817324
get_robot_state_min0.003466287704363261
get_state_dump_max0.005071427201883321
get_state_dump_mean0.004548578183734277
get_state_dump_median0.004399315304374962
get_state_dump_min0.004324254924303865
get_ui_image_max0.03522393524005849
get_ui_image_mean0.029591082628079676
get_ui_image_median0.02873453141891793
get_ui_image_min0.025671332434424352
in-drivable-lane_max16.15000000000023
in-drivable-lane_mean7.012500000000087
in-drivable-lane_min2.050000000000029
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.419106763175214, "get_ui_image": 0.02663466456818254, "step_physics": 0.09033985578850524, "survival_time": 14.550000000000072, "driven_lanedir": 2.148505553432445, "get_state_dump": 0.004324254924303865, "get_robot_state": 0.003466287704363261, "sim_render-ego0": 0.003654763306656929, "get_duckie_state": 1.192909397491037e-06, "in-drivable-lane": 2.050000000000029, "deviation-heading": 1.4868612198961164, "agent_compute-ego0": 0.012549696719809754, "complete-iteration": 0.15386833230110064, "set_robot_commands": 0.002062980442830961, "deviation-center-line": 0.5561771565487942, "driven_lanedir_consec": 2.148505553432445, "sim_compute_sim_state": 0.008863987171486632, "sim_compute_performance-ego0": 0.0018952456239151629}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.636864256344564, "get_ui_image": 0.03522393524005849, "step_physics": 0.13927760944571546, "survival_time": 4.599999999999992, "driven_lanedir": 0.21923344173731651, "get_state_dump": 0.004395269578503024, "get_robot_state": 0.003496213625836116, "sim_render-ego0": 0.003797149145474998, "get_duckie_state": 1.2331111456758232e-06, "in-drivable-lane": 2.8499999999999908, "deviation-heading": 0.8693376975487915, "agent_compute-ego0": 0.012780366405364006, "complete-iteration": 0.21289318351335423, "set_robot_commands": 0.0022178311501779863, "deviation-center-line": 0.11415682339917506, "driven_lanedir_consec": 0.21923344173731651, "sim_compute_sim_state": 0.009733025745678973, "sim_compute_performance-ego0": 0.0018888160746584656}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.94374098159553, "get_ui_image": 0.03083439826965332, "step_physics": 0.09861911566361135, "survival_time": 28.700000000000273, "driven_lanedir": 1.832778807815369, "get_state_dump": 0.0044033610302469, "get_robot_state": 0.003520713059798531, "sim_render-ego0": 0.00381724855174189, "get_duckie_state": 1.2128249458644702e-06, "in-drivable-lane": 16.15000000000023, "deviation-heading": 4.6042144100930615, "agent_compute-ego0": 0.012598930027173914, "complete-iteration": 0.16966465950012208, "set_robot_commands": 0.0021353908207105555, "deviation-center-line": 1.1016798121984852, "driven_lanedir_consec": 1.787214204231509, "sim_compute_sim_state": 0.011737725631050442, "sim_compute_performance-ego0": 0.001913822837497877}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.5547327195198575, "get_ui_image": 0.025671332434424352, "step_physics": 0.08709861327469491, "survival_time": 20.90000000000016, "driven_lanedir": 2.364252145580529, "get_state_dump": 0.005071427201883321, "get_robot_state": 0.0035228871502569013, "sim_render-ego0": 0.003828064638561167, "get_duckie_state": 1.1812815518254028e-06, "in-drivable-lane": 7.0000000000000995, "deviation-heading": 2.45209441224847, "agent_compute-ego0": 0.03220234224506096, "complete-iteration": 0.16758615077253172, "set_robot_commands": 0.002188733767780313, "deviation-center-line": 0.8457968866369401, "driven_lanedir_consec": 2.364252145580529, "sim_compute_sim_state": 0.006078662052928406, "sim_compute_performance-ego0": 0.0018416873595026057}}
set_robot_commands_max0.0022178311501779863
set_robot_commands_mean0.002151234045374954
set_robot_commands_median0.002162062294245434
set_robot_commands_min0.002062980442830961
sim_compute_performance-ego0_max0.001913822837497877
sim_compute_performance-ego0_mean0.0018848929738935277
sim_compute_performance-ego0_median0.0018920308492868143
sim_compute_performance-ego0_min0.0018416873595026057
sim_compute_sim_state_max0.011737725631050442
sim_compute_sim_state_mean0.009103350150286111
sim_compute_sim_state_median0.009298506458582802
sim_compute_sim_state_min0.006078662052928406
sim_render-ego0_max0.003828064638561167
sim_render-ego0_mean0.003774306410608746
sim_render-ego0_median0.003807198848608444
sim_render-ego0_min0.003654763306656929
simulation-passed1
step_physics_max0.13927760944571546
step_physics_mean0.10383379854313174
step_physics_median0.0944794857260583
step_physics_min0.08709861327469491
survival_time_max28.700000000000273
survival_time_mean17.187500000000124
survival_time_min4.599999999999992
No reset possible
5725810984Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:17:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.3616896385636652
survival_time_median19.225000000000136
deviation-center-line_median0.46800357365456213
in-drivable-lane_median11.57500000000013


other stats
agent_compute-ego0_max0.05203619368566661
agent_compute-ego0_mean0.04863648427020994
agent_compute-ego0_median0.05038118292102789
agent_compute-ego0_min0.04174737755311739
complete-iteration_max0.26203241042040903
complete-iteration_mean0.2186601374901772
complete-iteration_median0.2129836594975572
complete-iteration_min0.18664082054518535
deviation-center-line_max0.7596176106329084
deviation-center-line_mean0.46076080773401423
deviation-center-line_min0.14741847299402425
deviation-heading_max3.995360636994562
deviation-heading_mean1.7515397213551251
deviation-heading_median1.1635142134843277
deviation-heading_min0.6837698214572832
driven_any_max6.430261724383912
driven_any_mean3.0169744909329865
driven_any_median2.404467763189473
driven_any_min0.8287007129690873
driven_lanedir_consec_max2.8312849229314736
driven_lanedir_consec_mean1.5339801818778396
driven_lanedir_consec_min0.5812565274525534
driven_lanedir_max3.13839567649551
driven_lanedir_mean1.6107578702688483
driven_lanedir_median1.3616896385636652
driven_lanedir_min0.5812565274525534
get_duckie_state_max1.463224721509357e-06
get_duckie_state_mean1.2738365753639904e-06
get_duckie_state_median1.2811574915643435e-06
get_duckie_state_min1.0698065968179186e-06
get_robot_state_max0.003960666545601778
get_robot_state_mean0.003721769084831376
get_robot_state_median0.003706865418159374
get_robot_state_min0.003512678957404978
get_state_dump_max0.00486078705898551
get_state_dump_mean0.004609990064901559
get_state_dump_median0.0046188872766673075
get_state_dump_min0.004341398647286116
get_ui_image_max0.03652358273847388
get_ui_image_mean0.03041123774865299
get_ui_image_median0.029898112232212386
get_ui_image_min0.025325143791713325
in-drivable-lane_max41.99999999999861
in-drivable-lane_mean16.57499999999972
in-drivable-lane_min1.149999999999996
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.84327945492039, "get_ui_image": 0.029867165033207382, "step_physics": 0.1058808165927266, "survival_time": 21.45000000000017, "driven_lanedir": 2.009779013441402, "get_state_dump": 0.00486078705898551, "get_robot_state": 0.003960666545601778, "sim_render-ego0": 0.004139500995014989, "get_duckie_state": 1.463224721509357e-06, "in-drivable-lane": 10.400000000000148, "deviation-heading": 1.126327751844945, "agent_compute-ego0": 0.04874738149864729, "complete-iteration": 0.21332215098447577, "set_robot_commands": 0.0023810142694517624, "deviation-center-line": 0.4411724927479856, "driven_lanedir_consec": 2.009779013441402, "sim_compute_sim_state": 0.011184263229370116, "sim_compute_performance-ego0": 0.0022009766379068063}, "LF-norm-zigzag-000-ego0": {"driven_any": 0.8287007129690873, "get_ui_image": 0.03652358273847388, "step_physics": 0.155649427973896, "survival_time": 5.399999999999989, "driven_lanedir": 0.5812565274525534, "get_state_dump": 0.004833763892497491, "get_robot_state": 0.00387996708581207, "sim_render-ego0": 0.004089276724999104, "get_duckie_state": 1.4436354330920298e-06, "in-drivable-lane": 1.149999999999996, "deviation-heading": 1.2007006751237104, "agent_compute-ego0": 0.04174737755311739, "complete-iteration": 0.26203241042040903, "set_robot_commands": 0.0023987074510766824, "deviation-center-line": 0.49483465456113873, "driven_lanedir_consec": 0.5812565274525534, "sim_compute_sim_state": 0.010660287437088993, "sim_compute_performance-ego0": 0.00215181954410098}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.430261724383912, "get_ui_image": 0.029929059431217393, "step_physics": 0.10060897397558258, "survival_time": 59.99999999999873, "driven_lanedir": 3.13839567649551, "get_state_dump": 0.004341398647286116, "get_robot_state": 0.003533763750506678, "sim_render-ego0": 0.003811867608317328, "get_duckie_state": 1.0698065968179186e-06, "in-drivable-lane": 41.99999999999861, "deviation-heading": 3.995360636994562, "agent_compute-ego0": 0.05203619368566661, "complete-iteration": 0.21264516801063865, "set_robot_commands": 0.002135058029009639, "deviation-center-line": 0.7596176106329084, "driven_lanedir_consec": 2.8312849229314736, "sim_compute_sim_state": 0.014281526989583468, "sim_compute_performance-ego0": 0.001889138297177076}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.965656071458556, "get_ui_image": 0.025325143791713325, "step_physics": 0.08862734749869518, "survival_time": 17.000000000000107, "driven_lanedir": 0.7136002636859284, "get_state_dump": 0.004404010660837123, "get_robot_state": 0.003512678957404978, "sim_render-ego0": 0.0038033293838724714, "get_duckie_state": 1.118679550036657e-06, "in-drivable-lane": 12.750000000000114, "deviation-heading": 0.6837698214572832, "agent_compute-ego0": 0.05201498434340849, "complete-iteration": 0.18664082054518535, "set_robot_commands": 0.0021619580008766866, "deviation-center-line": 0.14741847299402425, "driven_lanedir_consec": 0.7136002636859284, "sim_compute_sim_state": 0.004853920503096147, "sim_compute_performance-ego0": 0.001855799879146811}}
set_robot_commands_max0.0023987074510766824
set_robot_commands_mean0.0022691844376036925
set_robot_commands_median0.0022714861351642245
set_robot_commands_min0.002135058029009639
sim_compute_performance-ego0_max0.0022009766379068063
sim_compute_performance-ego0_mean0.0020244335895829185
sim_compute_performance-ego0_median0.0020204789206390277
sim_compute_performance-ego0_min0.001855799879146811
sim_compute_sim_state_max0.014281526989583468
sim_compute_sim_state_mean0.010244999539784682
sim_compute_sim_state_median0.010922275333229554
sim_compute_sim_state_min0.004853920503096147
sim_render-ego0_max0.004139500995014989
sim_render-ego0_mean0.0039609936780509725
sim_render-ego0_median0.003950572166658216
sim_render-ego0_min0.0038033293838724714
simulation-passed1
step_physics_max0.155649427973896
step_physics_mean0.1126916415102251
step_physics_median0.1032448952841546
step_physics_min0.08862734749869518
survival_time_max59.99999999999873
survival_time_mean25.96249999999975
survival_time_min5.399999999999989
No reset possible
5710911017Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:38:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.3023424510147015
survival_time_median59.99999999999873
deviation-center-line_median2.607629819906061
in-drivable-lane_median3.849999999999918


other stats
agent_compute-ego0_max0.03440073685880307
agent_compute-ego0_mean0.025675358968809386
agent_compute-ego0_median0.027617538005088788
agent_compute-ego0_min0.013065623006256893
complete-iteration_max0.23378441216645887
complete-iteration_mean0.20726315524556255
complete-iteration_median0.2049973911290165
complete-iteration_min0.18527342655775847
deviation-center-line_max2.935116314690988
deviation-center-line_mean2.671400584528107
deviation-center-line_min2.5352263836093183
deviation-heading_max18.77454763577295
deviation-heading_mean14.702346165045409
deviation-heading_median13.455301793488822
deviation-heading_min13.124233437431029
driven_any_max9.493989086346993
driven_any_mean9.298123549099998
driven_any_median9.341505781497936
driven_any_min9.01549354705713
driven_lanedir_consec_max6.303205813282484
driven_lanedir_consec_mean4.567387194376099
driven_lanedir_consec_min3.3616580621925083
driven_lanedir_max9.077295890710843
driven_lanedir_mean8.080377513416543
driven_lanedir_median8.299236679614538
driven_lanedir_min6.645740803726257
get_duckie_state_max1.4807362044284385e-06
get_duckie_state_mean1.4210819304733848e-06
get_duckie_state_median1.41413384532055e-06
get_duckie_state_min1.3753238268240009e-06
get_robot_state_max0.00387317512950532
get_robot_state_mean0.00375529431383576
get_robot_state_median0.00375533262756246
get_robot_state_min0.003637336870712801
get_state_dump_max0.005002864095988023
get_state_dump_mean0.004845330210946979
get_state_dump_median0.004829959607342697
get_state_dump_min0.0047185375331144945
get_ui_image_max0.03587950179221529
get_ui_image_mean0.030940671745287588
get_ui_image_median0.03092982667768925
get_ui_image_min0.02602353183355657
in-drivable-lane_max9.799999999999637
in-drivable-lane_mean4.449999999999863
in-drivable-lane_min0.29999999999998295
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 9.493989086346993, "get_ui_image": 0.02885843137221769, "step_physics": 0.11776311590113708, "survival_time": 59.99999999999873, "driven_lanedir": 9.077295890710843, "get_state_dump": 0.004911669088740035, "get_robot_state": 0.003792330188417712, "sim_render-ego0": 0.004001926125138129, "get_duckie_state": 1.4265411402363264e-06, "in-drivable-lane": 0.29999999999998295, "deviation-heading": 13.124233437431029, "agent_compute-ego0": 0.021276964732351945, "complete-iteration": 0.1952234301142252, "set_robot_commands": 0.0022904223744617115, "deviation-center-line": 2.64654614291011, "driven_lanedir_consec": 6.303205813282484, "sim_compute_sim_state": 0.010183147149320247, "sim_compute_performance-ego0": 0.002056863286116042}, "LF-norm-zigzag-000-ego0": {"driven_any": 9.311327075258369, "get_ui_image": 0.03587950179221529, "step_physics": 0.13587329647721696, "survival_time": 59.99999999999873, "driven_lanedir": 8.05276491778584, "get_state_dump": 0.0047185375331144945, "get_robot_state": 0.003637336870712801, "sim_render-ego0": 0.00383944158053815, "get_duckie_state": 1.4017265504047734e-06, "in-drivable-lane": 5.149999999999853, "deviation-heading": 13.767906970790197, "agent_compute-ego0": 0.03440073685880307, "complete-iteration": 0.23378441216645887, "set_robot_commands": 0.002196156512092889, "deviation-center-line": 2.5352263836093183, "driven_lanedir_consec": 4.315881567185935, "sim_compute_sim_state": 0.011167597711135903, "sim_compute_performance-ego0": 0.001984318329035293}, "LF-norm-techtrack-000-ego0": {"driven_any": 9.01549354705713, "get_ui_image": 0.0330012219831608, "step_physics": 0.13897451393610233, "survival_time": 59.99999999999873, "driven_lanedir": 6.645740803726257, "get_state_dump": 0.005002864095988023, "get_robot_state": 0.00387317512950532, "sim_render-ego0": 0.004150001929264879, "get_duckie_state": 1.4807362044284385e-06, "in-drivable-lane": 9.799999999999637, "deviation-heading": 18.77454763577295, "agent_compute-ego0": 0.013065623006256893, "complete-iteration": 0.21477135214380777, "set_robot_commands": 0.0023488333382078452, "deviation-center-line": 2.935116314690988, "driven_lanedir_consec": 4.288803334843468, "sim_compute_sim_state": 0.012067676086806933, "sim_compute_performance-ego0": 0.002191726810032878}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.371684487737502, "get_ui_image": 0.02602353183355657, "step_physics": 0.10237165493929416, "survival_time": 59.99999999999873, "driven_lanedir": 8.545708441443233, "get_state_dump": 0.004748250125945359, "get_robot_state": 0.003718335066707208, "sim_render-ego0": 0.003891626861470625, "get_duckie_state": 1.3753238268240009e-06, "in-drivable-lane": 2.549999999999983, "deviation-heading": 13.142696616187449, "agent_compute-ego0": 0.03395811127782563, "complete-iteration": 0.18527342655775847, "set_robot_commands": 0.0022531042091058356, "deviation-center-line": 2.5687134969020113, "driven_lanedir_consec": 3.3616580621925083, "sim_compute_sim_state": 0.006231582333503615, "sim_compute_performance-ego0": 0.0019835024253216315}}
set_robot_commands_max0.0023488333382078452
set_robot_commands_mean0.0022721291084670703
set_robot_commands_median0.002271763291783774
set_robot_commands_min0.002196156512092889
sim_compute_performance-ego0_max0.002191726810032878
sim_compute_performance-ego0_mean0.0020541027126264612
sim_compute_performance-ego0_median0.002020590807575667
sim_compute_performance-ego0_min0.0019835024253216315
sim_compute_sim_state_max0.012067676086806933
sim_compute_sim_state_mean0.009912500820191674
sim_compute_sim_state_median0.010675372430228076
sim_compute_sim_state_min0.006231582333503615
sim_render-ego0_max0.004150001929264879
sim_render-ego0_mean0.003970749124102946
sim_render-ego0_median0.003946776493304377
sim_render-ego0_min0.00383944158053815
simulation-passed1
step_physics_max0.13897451393610233
step_physics_mean0.12374564531343764
step_physics_median0.12681820618917702
step_physics_min0.10237165493929416
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5699711042Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:17:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.513553557142739
survival_time_median20.70000000000016
deviation-center-line_median0.4864798041665735
in-drivable-lane_median11.82500000000014


other stats
agent_compute-ego0_max0.013452028453995247
agent_compute-ego0_mean0.012823673222016406
agent_compute-ego0_median0.0126461805081835
agent_compute-ego0_min0.01255030341770338
complete-iteration_max0.20765340212479377
complete-iteration_mean0.17720862085132916
complete-iteration_median0.17449768410219946
complete-iteration_min0.152185713076124
deviation-center-line_max1.295294619179903
deviation-center-line_mean0.6097916660053635
deviation-center-line_min0.17091243650840382
deviation-heading_max7.068673410455176
deviation-heading_mean3.326390147838714
deviation-heading_median2.660440931302112
deviation-heading_min0.916005318295458
driven_any_max9.23386500188564
driven_any_mean4.212922717459296
driven_any_median3.106312321859803
driven_any_min1.40520122423194
driven_lanedir_consec_max2.514189916315347
driven_lanedir_consec_mean1.5564583052154128
driven_lanedir_consec_min0.6845361902608262
driven_lanedir_max2.5550100060133234
driven_lanedir_mean1.566663327639907
driven_lanedir_median1.513553557142739
driven_lanedir_min0.6845361902608262
get_duckie_state_max1.2921352012484682e-06
get_duckie_state_mean1.2531184152637888e-06
get_duckie_state_median1.2553880465287306e-06
get_duckie_state_min1.209562366749226e-06
get_robot_state_max0.003683509826660156
get_robot_state_mean0.003623985158134946
get_robot_state_median0.003610889843211527
get_robot_state_min0.0035906511194565716
get_state_dump_max0.004641006293806057
get_state_dump_mean0.004540196309100971
get_state_dump_median0.00453163240876649
get_state_dump_min0.0044565141250648465
get_ui_image_max0.03476418916461537
get_ui_image_mean0.029705884151919063
get_ui_image_median0.029180216130342725
get_ui_image_min0.02569891518237544
in-drivable-lane_max40.99999999999858
in-drivable-lane_mean16.69999999999971
in-drivable-lane_min2.1499999999999924
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.397630919657531, "get_ui_image": 0.027104984366375467, "step_physics": 0.10048769287441088, "survival_time": 28.700000000000273, "driven_lanedir": 2.046730695725299, "get_state_dump": 0.004562727057415506, "get_robot_state": 0.003683509826660156, "sim_render-ego0": 0.003886943485425866, "get_duckie_state": 1.226093458092731e-06, "in-drivable-lane": 15.800000000000225, "deviation-heading": 2.097885773720894, "agent_compute-ego0": 0.01255030341770338, "complete-iteration": 0.16656958953193995, "set_robot_commands": 0.0022400702600893765, "deviation-center-line": 0.42770588369315526, "driven_lanedir_consec": 2.046730695725299, "sim_compute_sim_state": 0.009938393053801163, "sim_compute_performance-ego0": 0.002035515826681386}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.40520122423194, "get_ui_image": 0.03476418916461537, "step_physics": 0.13367045851587092, "survival_time": 10.25000000000001, "driven_lanedir": 0.9803764185601792, "get_state_dump": 0.004641006293806057, "get_robot_state": 0.003627902095757642, "sim_render-ego0": 0.003915748549896536, "get_duckie_state": 1.28468263496473e-06, "in-drivable-lane": 2.1499999999999924, "deviation-heading": 3.22299608888333, "agent_compute-ego0": 0.01271653175354004, "complete-iteration": 0.20765340212479377, "set_robot_commands": 0.0021823320573973425, "deviation-center-line": 0.5452537246399919, "driven_lanedir_consec": 0.9803764185601792, "sim_compute_sim_state": 0.010084115185783903, "sim_compute_performance-ego0": 0.0019685516079652656}, "LF-norm-techtrack-000-ego0": {"driven_any": 9.23386500188564, "get_ui_image": 0.03125544789430998, "step_physics": 0.10859672076299128, "survival_time": 59.99999999999873, "driven_lanedir": 2.5550100060133234, "get_state_dump": 0.0044565141250648465, "get_robot_state": 0.003593877590665412, "sim_render-ego0": 0.003884541204231764, "get_duckie_state": 1.209562366749226e-06, "in-drivable-lane": 40.99999999999858, "deviation-heading": 7.068673410455176, "agent_compute-ego0": 0.013452028453995247, "complete-iteration": 0.18242577867245893, "set_robot_commands": 0.0021655309408729418, "deviation-center-line": 1.295294619179903, "driven_lanedir_consec": 2.514189916315347, "sim_compute_sim_state": 0.012999067497094605, "sim_compute_performance-ego0": 0.0019413414446142296}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.8149937240620757, "get_ui_image": 0.02569891518237544, "step_physics": 0.09302922230140837, "survival_time": 12.700000000000044, "driven_lanedir": 0.6845361902608262, "get_state_dump": 0.004500537760117475, "get_robot_state": 0.0035906511194565716, "sim_render-ego0": 0.0037659822725782206, "get_duckie_state": 1.2921352012484682e-06, "in-drivable-lane": 7.850000000000055, "deviation-heading": 0.916005318295458, "agent_compute-ego0": 0.012575829262826957, "complete-iteration": 0.152185713076124, "set_robot_commands": 0.0021501887078378716, "deviation-center-line": 0.17091243650840382, "driven_lanedir_consec": 0.6845361902608262, "sim_compute_sim_state": 0.004938176098991843, "sim_compute_performance-ego0": 0.0018557903813380824}}
set_robot_commands_max0.0022400702600893765
set_robot_commands_mean0.0021845304915493833
set_robot_commands_median0.002173931499135142
set_robot_commands_min0.0021501887078378716
sim_compute_performance-ego0_max0.002035515826681386
sim_compute_performance-ego0_mean0.0019502998151497408
sim_compute_performance-ego0_median0.0019549465262897474
sim_compute_performance-ego0_min0.0018557903813380824
sim_compute_sim_state_max0.012999067497094605
sim_compute_sim_state_mean0.00948993795891788
sim_compute_sim_state_median0.010011254119792534
sim_compute_sim_state_min0.004938176098991843
sim_render-ego0_max0.003915748549896536
sim_render-ego0_mean0.003863303878033097
sim_render-ego0_median0.003885742344828815
sim_render-ego0_min0.0037659822725782206
simulation-passed1
step_physics_max0.13367045851587092
step_physics_mean0.10894602361367035
step_physics_median0.10454220681870108
step_physics_min0.09302922230140837
survival_time_max59.99999999999873
survival_time_mean27.912499999999763
survival_time_min10.25000000000001
No reset possible
5698411312Moustafa Elarabitemplate-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-070:01:09
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5697011313Moustafa Elarabitemplate-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-070:01:04
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5695511278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:04:39
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5692411278Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:06:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 48, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 48, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5689511283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:03:49
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5687811283Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5685311282Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5684811285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5683311285Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:53
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5681011281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:16
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 49, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 49, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5680511286Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5679411286Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:42
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5678811287Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5675511289Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:56
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5674811427Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-erroryesnogpu-prod-070:00:54
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5670811284Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:03:47
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5668811290Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:01:17
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5666811296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:15
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5663611296Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:04:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5661911425Frank (Chude) Qian 🇨🇦template-pytorchaido5-LFP-sim-validationLFP-simhost-errornonogpu-prod-070:00:43
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5659211311Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:53
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5657311303Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:11
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5653911301Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:17
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5647611305Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:21
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5642511299Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:57
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5502911415Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LFV_multi-sim-validationLFVmultibodyv-simsuccessyesnogpu-prod-071:37:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median13.244289278462052
survival_time_median59.99999999999873
deviation-center-line_median2.9369463298662275
in-drivable-lane_median2.349999999999956


other stats
agent_compute-ego0_max0.04618208037129449
agent_compute-ego0_mean0.04261658026315866
agent_compute-ego0_median0.0421116129742574
agent_compute-ego0_min0.04055528974255158
agent_compute-ego1_max0.04537203448896702
agent_compute-ego1_mean0.041547109979015485
agent_compute-ego1_median0.04091519539203374
agent_compute-ego1_min0.03937782971289235
complete-iteration_max0.776721277800726
complete-iteration_mean0.6517498859736268
complete-iteration_median0.7190106522530739
complete-iteration_min0.31643653798986365
deviation-center-line_max3.3309830572989374
deviation-center-line_mean2.835894500478143
deviation-center-line_min1.6753504117871665
deviation-heading_max14.44818062825616
deviation-heading_mean10.999047094770328
deviation-heading_median11.54486383287232
deviation-heading_min5.458199613473701
driven_any_max14.590320002142294
driven_any_mean13.835537597781002
driven_any_median14.590151130839264
driven_any_min9.307535393842445
driven_lanedir_consec_max14.240157955669602
driven_lanedir_consec_mean12.421074282388194
driven_lanedir_consec_min6.987901081338952
driven_lanedir_max14.240157955669602
driven_lanedir_mean12.518419343317436
driven_lanedir_median13.244289278462052
driven_lanedir_min6.987901081338952
get_duckie_state_max1.5708627946966397e-06
get_duckie_state_mean1.3610641257200978e-06
get_duckie_state_median1.3765149271359157e-06
get_duckie_state_min1.2059890658134822e-06
get_robot_state_max0.014901315639854768
get_robot_state_mean0.01297143095020194
get_robot_state_median0.014072340195820194
get_robot_state_min0.00677477658784557
get_state_dump_max0.009787753658628186
get_state_dump_mean0.008761000269951615
get_state_dump_median0.009401263642767685
get_state_dump_min0.005844010856111997
get_ui_image_max0.045169535921971864
get_ui_image_mean0.04019471440131966
get_ui_image_median0.04269639320119434
get_ui_image_min0.027430567248114223
in-drivable-lane_max9.499999999999968
in-drivable-lane_mean3.324999999999949
in-drivable-lane_min0.0
per-episodes
details{"LFV_multi-norm-loop-000-ego0": {"driven_any": 14.5902730432591, "get_ui_image": 0.03910028765739549, "step_physics": 0.3401384824122319, "survival_time": 59.99999999999873, "driven_lanedir": 14.141441985241428, "get_state_dump": 0.009401263642767685, "get_robot_state": 0.014072340195820194, "sim_render-ego0": 0.0037330161721184298, "sim_render-ego1": 0.003702545245422313, "sim_render-ego2": 0.0036123888776463135, "sim_render-ego3": 0.003638402706975246, "get_duckie_state": 1.3765149271359157e-06, "in-drivable-lane": 0.5500000000000078, "deviation-heading": 8.910538068363719, "agent_compute-ego0": 0.0421116129742574, "agent_compute-ego1": 0.04091519539203374, "agent_compute-ego2": 0.04075447169072821, "agent_compute-ego3": 0.04086595034222122, "complete-iteration": 0.6271744018589627, "set_robot_commands": 0.002135427468623051, "deviation-center-line": 2.733821751657914, "driven_lanedir_consec": 14.141441985241428, "sim_compute_sim_state": 0.02848804403999068, "sim_compute_performance-ego0": 0.0020372821925383226, "sim_compute_performance-ego1": 0.0019623268454596957, "sim_compute_performance-ego2": 0.001898634741447252, "sim_compute_performance-ego3": 0.0019076781705654631}, "LFV_multi-norm-loop-000-ego1": {"driven_any": 14.590315637034868, "get_ui_image": 0.03910028765739549, "step_physics": 0.3401384824122319, "survival_time": 59.99999999999873, "driven_lanedir": 14.22893005474485, "get_state_dump": 0.009401263642767685, "get_robot_state": 0.014072340195820194, "sim_render-ego0": 0.0037330161721184298, "sim_render-ego1": 0.003702545245422313, "sim_render-ego2": 0.0036123888776463135, "sim_render-ego3": 0.003638402706975246, "get_duckie_state": 1.3765149271359157e-06, "in-drivable-lane": 0.4500000000000064, "deviation-heading": 8.144409722198054, "agent_compute-ego0": 0.0421116129742574, "agent_compute-ego1": 0.04091519539203374, "agent_compute-ego2": 0.04075447169072821, "agent_compute-ego3": 0.04086595034222122, "complete-iteration": 0.6271744018589627, "set_robot_commands": 0.002135427468623051, "deviation-center-line": 2.55476947818271, "driven_lanedir_consec": 14.22893005474485, "sim_compute_sim_state": 0.02848804403999068, "sim_compute_performance-ego0": 0.0020372821925383226, "sim_compute_performance-ego1": 0.0019623268454596957, "sim_compute_performance-ego2": 0.001898634741447252, "sim_compute_performance-ego3": 0.0019076781705654631}, "LFV_multi-norm-loop-000-ego2": {"driven_any": 14.590127640767218, "get_ui_image": 0.03910028765739549, "step_physics": 0.3401384824122319, "survival_time": 59.99999999999873, "driven_lanedir": 12.625761572714728, "get_state_dump": 0.009401263642767685, "get_robot_state": 0.014072340195820194, "sim_render-ego0": 0.0037330161721184298, "sim_render-ego1": 0.003702545245422313, "sim_render-ego2": 0.0036123888776463135, "sim_render-ego3": 0.003638402706975246, "get_duckie_state": 1.3765149271359157e-06, "in-drivable-lane": 5.849999999999839, "deviation-heading": 11.342340466666103, "agent_compute-ego0": 0.0421116129742574, "agent_compute-ego1": 0.04091519539203374, "agent_compute-ego2": 0.04075447169072821, "agent_compute-ego3": 0.04086595034222122, "complete-iteration": 0.6271744018589627, "set_robot_commands": 0.002135427468623051, "deviation-center-line": 2.5598726661459428, "driven_lanedir_consec": 12.625761572714728, "sim_compute_sim_state": 0.02848804403999068, "sim_compute_performance-ego0": 0.0020372821925383226, "sim_compute_performance-ego1": 0.0019623268454596957, "sim_compute_performance-ego2": 0.001898634741447252, "sim_compute_performance-ego3": 0.0019076781705654631}, "LFV_multi-norm-loop-000-ego3": {"driven_any": 14.590320002142294, "get_ui_image": 0.03910028765739549, "step_physics": 0.3401384824122319, "survival_time": 59.99999999999873, "driven_lanedir": 14.240157955669602, "get_state_dump": 0.009401263642767685, "get_robot_state": 0.014072340195820194, "sim_render-ego0": 0.0037330161721184298, "sim_render-ego1": 0.003702545245422313, "sim_render-ego2": 0.0036123888776463135, "sim_render-ego3": 0.003638402706975246, "get_duckie_state": 1.3765149271359157e-06, "in-drivable-lane": 0.4499999999999744, "deviation-heading": 7.700365314146502, "agent_compute-ego0": 0.0421116129742574, "agent_compute-ego1": 0.04091519539203374, "agent_compute-ego2": 0.04075447169072821, "agent_compute-ego3": 0.04086595034222122, "complete-iteration": 0.6271744018589627, "set_robot_commands": 0.002135427468623051, "deviation-center-line": 2.5378176698404205, "driven_lanedir_consec": 14.240157955669602, "sim_compute_sim_state": 0.02848804403999068, "sim_compute_performance-ego0": 0.0020372821925383226, "sim_compute_performance-ego1": 0.0019623268454596957, "sim_compute_performance-ego2": 0.001898634741447252, "sim_compute_performance-ego3": 0.0019076781705654631}, "LFV_multi-norm-zigzag-000-ego0": {"driven_any": 14.590144647585197, "get_ui_image": 0.04269639320119434, "step_physics": 0.4244263602930938, "survival_time": 59.99999999999873, "driven_lanedir": 13.258694451344738, "get_state_dump": 0.008552478215378786, "get_robot_state": 0.013038964196109058, "sim_render-ego0": 0.00346634667878544, "sim_render-ego1": 0.0034625992786874384, "sim_render-ego2": 0.00339973717307568, "sim_render-ego3": 0.003374290704528656, "get_duckie_state": 1.2059890658134822e-06, "in-drivable-lane": 2.7999999999999776, "deviation-heading": 13.120754353129335, "agent_compute-ego0": 0.04055528974255158, "agent_compute-ego1": 0.03937782971289235, "agent_compute-ego2": 0.0397336780776787, "agent_compute-ego3": 0.03998830534834151, "complete-iteration": 0.7190106522530739, "set_robot_commands": 0.0019392171172079296, "deviation-center-line": 3.2048200089630243, "driven_lanedir_consec": 13.258694451344738, "sim_compute_sim_state": 0.041871269775568494, "sim_compute_performance-ego0": 0.0018658905998058463, "sim_compute_performance-ego1": 0.0017644489536078943, "sim_compute_performance-ego2": 0.0017123242202746083, "sim_compute_performance-ego3": 0.0017137978098771653}, "LFV_multi-norm-zigzag-000-ego1": {"driven_any": 14.590111097542447, "get_ui_image": 0.04269639320119434, "step_physics": 0.4244263602930938, "survival_time": 59.99999999999873, "driven_lanedir": 13.229884105579366, "get_state_dump": 0.008552478215378786, "get_robot_state": 0.013038964196109058, "sim_render-ego0": 0.00346634667878544, "sim_render-ego1": 0.0034625992786874384, "sim_render-ego2": 0.00339973717307568, "sim_render-ego3": 0.003374290704528656, "get_duckie_state": 1.2059890658134822e-06, "in-drivable-lane": 2.8999999999999195, "deviation-heading": 13.335821685212204, "agent_compute-ego0": 0.04055528974255158, "agent_compute-ego1": 0.03937782971289235, "agent_compute-ego2": 0.0397336780776787, "agent_compute-ego3": 0.03998830534834151, "complete-iteration": 0.7190106522530739, "set_robot_commands": 0.0019392171172079296, "deviation-center-line": 3.282022719765582, "driven_lanedir_consec": 13.229884105579366, "sim_compute_sim_state": 0.041871269775568494, "sim_compute_performance-ego0": 0.0018658905998058463, "sim_compute_performance-ego1": 0.0017644489536078943, "sim_compute_performance-ego2": 0.0017123242202746083, "sim_compute_performance-ego3": 0.0017137978098771653}, "LFV_multi-norm-zigzag-000-ego2": {"driven_any": 14.590157614093332, "get_ui_image": 0.04269639320119434, "step_physics": 0.4244263602930938, "survival_time": 59.99999999999873, "driven_lanedir": 13.226162819960347, "get_state_dump": 0.008552478215378786, "get_robot_state": 0.013038964196109058, "sim_render-ego0": 0.00346634667878544, "sim_render-ego1": 0.0034625992786874384, "sim_render-ego2": 0.00339973717307568, "sim_render-ego3": 0.003374290704528656, "get_duckie_state": 1.2059890658134822e-06, "in-drivable-lane": 3.0999999999998913, "deviation-heading": 12.503919157748056, "agent_compute-ego0": 0.04055528974255158, "agent_compute-ego1": 0.03937782971289235, "agent_compute-ego2": 0.0397336780776787, "agent_compute-ego3": 0.03998830534834151, "complete-iteration": 0.7190106522530739, "set_robot_commands": 0.0019392171172079296, "deviation-center-line": 3.1046431577056612, "driven_lanedir_consec": 13.226162819960347, "sim_compute_sim_state": 0.041871269775568494, "sim_compute_performance-ego0": 0.0018658905998058463, "sim_compute_performance-ego1": 0.0017644489536078943, "sim_compute_performance-ego2": 0.0017123242202746083, "sim_compute_performance-ego3": 0.0017137978098771653}, "LFV_multi-norm-zigzag-000-ego3": {"driven_any": 14.590136745179713, "get_ui_image": 0.04269639320119434, "step_physics": 0.4244263602930938, "survival_time": 59.99999999999873, "driven_lanedir": 11.589663841250047, "get_state_dump": 0.008552478215378786, "get_robot_state": 0.013038964196109058, "sim_render-ego0": 0.00346634667878544, "sim_render-ego1": 0.0034625992786874384, "sim_render-ego2": 0.00339973717307568, "sim_render-ego3": 0.003374290704528656, "get_duckie_state": 1.2059890658134822e-06, "in-drivable-lane": 9.499999999999968, "deviation-heading": 12.44061494097976, "agent_compute-ego0": 0.04055528974255158, "agent_compute-ego1": 0.03937782971289235, "agent_compute-ego2": 0.0397336780776787, "agent_compute-ego3": 0.03998830534834151, "complete-iteration": 0.7190106522530739, "set_robot_commands": 0.0019392171172079296, "deviation-center-line": 2.917440862836468, "driven_lanedir_consec": 10.226832988240714, "sim_compute_sim_state": 0.041871269775568494, "sim_compute_performance-ego0": 0.0018658905998058463, "sim_compute_performance-ego1": 0.0017644489536078943, "sim_compute_performance-ego2": 0.0017123242202746083, "sim_compute_performance-ego3": 0.0017137978098771653}, "LFV_multi-norm-techtrack-000-ego0": {"driven_any": 14.59019195988844, "get_ui_image": 0.045169535921971864, "step_physics": 0.4488970935592842, "survival_time": 59.99999999999873, "driven_lanedir": 13.577606095123388, "get_state_dump": 0.009787753658628186, "get_robot_state": 0.014901315639854768, "sim_render-ego0": 0.003907093299020835, "sim_render-ego1": 0.003904177882490706, "sim_render-ego2": 0.003832750971569408, "sim_render-ego3": 0.003834068328514385, "get_duckie_state": 1.5708627946966397e-06, "in-drivable-lane": 1.8500000000000263, "deviation-heading": 11.656570340127372, "agent_compute-ego0": 0.04618208037129449, "agent_compute-ego1": 0.04537203448896702, "agent_compute-ego2": 0.045211390591382385, "agent_compute-ego3": 0.045319019606667295, "complete-iteration": 0.776721277800726, "set_robot_commands": 0.0023305408166508987, "deviation-center-line": 2.956451796895987, "driven_lanedir_consec": 13.577606095123388, "sim_compute_sim_state": 0.04244072193110813, "sim_compute_performance-ego0": 0.002260058845310386, "sim_compute_performance-ego1": 0.0020719743787398644, "sim_compute_performance-ego2": 0.0020506380003358204, "sim_compute_performance-ego3": 0.0020078854398068342}, "LFV_multi-norm-techtrack-000-ego1": {"driven_any": 14.590167828058831, "get_ui_image": 0.045169535921971864, "step_physics": 0.4488970935592842, "survival_time": 59.99999999999873, "driven_lanedir": 13.406730166889671, "get_state_dump": 0.009787753658628186, "get_robot_state": 0.014901315639854768, "sim_render-ego0": 0.003907093299020835, "sim_render-ego1": 0.003904177882490706, "sim_render-ego2": 0.003832750971569408, "sim_render-ego3": 0.003834068328514385, "get_duckie_state": 1.5708627946966397e-06, "in-drivable-lane": 1.8999999999999344, "deviation-heading": 13.447641062118793, "agent_compute-ego0": 0.04618208037129449, "agent_compute-ego1": 0.04537203448896702, "agent_compute-ego2": 0.045211390591382385, "agent_compute-ego3": 0.045319019606667295, "complete-iteration": 0.776721277800726, "set_robot_commands": 0.0023305408166508987, "deviation-center-line": 3.3309830572989374, "driven_lanedir_consec": 13.406730166889671, "sim_compute_sim_state": 0.04244072193110813, "sim_compute_performance-ego0": 0.002260058845310386, "sim_compute_performance-ego1": 0.0020719743787398644, "sim_compute_performance-ego2": 0.0020506380003358204, "sim_compute_performance-ego3": 0.0020078854398068342}, "LFV_multi-norm-techtrack-000-ego2": {"driven_any": 14.590071146172187, "get_ui_image": 0.045169535921971864, "step_physics": 0.4488970935592842, "survival_time": 59.99999999999873, "driven_lanedir": 11.906694861243944, "get_state_dump": 0.009787753658628186, "get_robot_state": 0.014901315639854768, "sim_render-ego0": 0.003907093299020835, "sim_render-ego1": 0.003904177882490706, "sim_render-ego2": 0.003832750971569408, "sim_render-ego3": 0.003834068328514385, "get_duckie_state": 1.5708627946966397e-06, "in-drivable-lane": 7.4499999999998, "deviation-heading": 14.44818062825616, "agent_compute-ego0": 0.04618208037129449, "agent_compute-ego1": 0.04537203448896702, "agent_compute-ego2": 0.045211390591382385, "agent_compute-ego3": 0.045319019606667295, "complete-iteration": 0.776721277800726, "set_robot_commands": 0.0023305408166508987, "deviation-center-line": 3.2471807830604957, "driven_lanedir_consec": 11.906694861243944, "sim_compute_sim_state": 0.04244072193110813, "sim_compute_performance-ego0": 0.002260058845310386, "sim_compute_performance-ego1": 0.0020719743787398644, "sim_compute_performance-ego2": 0.0020506380003358204, "sim_compute_performance-ego3": 0.0020078854398068342}, "LFV_multi-norm-techtrack-000-ego3": {"driven_any": 14.590173458304786, "get_ui_image": 0.045169535921971864, "step_physics": 0.4488970935592842, "survival_time": 59.99999999999873, "driven_lanedir": 13.660081841917345, "get_state_dump": 0.009787753658628186, "get_robot_state": 0.014901315639854768, "sim_render-ego0": 0.003907093299020835, "sim_render-ego1": 0.003904177882490706, "sim_render-ego2": 0.003832750971569408, "sim_render-ego3": 0.003834068328514385, "get_duckie_state": 1.5708627946966397e-06, "in-drivable-lane": 1.6500000000000057, "deviation-heading": 11.433157325617266, "agent_compute-ego0": 0.04618208037129449, "agent_compute-ego1": 0.04537203448896702, "agent_compute-ego2": 0.045211390591382385, "agent_compute-ego3": 0.045319019606667295, "complete-iteration": 0.776721277800726, "set_robot_commands": 0.0023305408166508987, "deviation-center-line": 3.0487830917671808, "driven_lanedir_consec": 13.660081841917345, "sim_compute_sim_state": 0.04244072193110813, "sim_compute_performance-ego0": 0.002260058845310386, "sim_compute_performance-ego1": 0.0020719743787398644, "sim_compute_performance-ego2": 0.0020506380003358204, "sim_compute_performance-ego3": 0.0020078854398068342}, "LFV_multi-norm-small_loop-000-ego0": {"driven_any": 9.307800155063164, "get_ui_image": 0.027430567248114223, "step_physics": 0.17178806673978023, "survival_time": 39.09999999999992, "driven_lanedir": 9.178159973425672, "get_state_dump": 0.005844010856111997, "get_robot_state": 0.00677477658784557, "sim_render-ego0": 0.0035976297858422135, "sim_render-ego1": 0.003559782739495561, "get_duckie_state": 1.220715304748613e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.458199613473701, "agent_compute-ego0": 0.040618095665903686, "agent_compute-ego1": 0.03949965066532218, "complete-iteration": 0.31643653798986365, "set_robot_commands": 0.0020398225820841003, "deviation-center-line": 1.6753504117871665, "driven_lanedir_consec": 9.178159973425672, "sim_compute_sim_state": 0.009414134811167516, "sim_compute_performance-ego0": 0.0018239627243763032, "sim_compute_performance-ego1": 0.001829667109639252}, "LFV_multi-norm-small_loop-000-ego1": {"driven_any": 9.307535393842445, "get_ui_image": 0.027430567248114223, "step_physics": 0.17178806673978023, "survival_time": 39.09999999999992, "driven_lanedir": 6.987901081338952, "get_state_dump": 0.005844010856111997, "get_robot_state": 0.00677477658784557, "sim_render-ego0": 0.0035976297858422135, "sim_render-ego1": 0.003559782739495561, "get_duckie_state": 1.220715304748613e-06, "in-drivable-lane": 8.099999999999937, "deviation-heading": 10.044146648747548, "agent_compute-ego0": 0.040618095665903686, "agent_compute-ego1": 0.03949965066532218, "complete-iteration": 0.31643653798986365, "set_robot_commands": 0.0020398225820841003, "deviation-center-line": 2.548565550786516, "driven_lanedir_consec": 6.987901081338952, "sim_compute_sim_state": 0.009414134811167516, "sim_compute_performance-ego0": 0.0018239627243763032, "sim_compute_performance-ego1": 0.001829667109639252}}
set_robot_commands_max0.0023305408166508987
set_robot_commands_mean0.0021214561981496944
set_robot_commands_median0.002135427468623051
set_robot_commands_min0.0019392171172079296
sim_compute_performance-ego0_max0.002260058845310386
sim_compute_performance-ego0_mean0.0020214894285264877
sim_compute_performance-ego0_median0.0020372821925383226
sim_compute_performance-ego0_min0.0018239627243763032
sim_compute_performance-ego1_max0.0020719743787398644
sim_compute_performance-ego1_mean0.0019181667807505944
sim_compute_performance-ego1_median0.0019623268454596957
sim_compute_performance-ego1_min0.0017644489536078943
sim_compute_sim_state_max0.04244072193110813
sim_compute_sim_state_mean0.0335734580435003
sim_compute_sim_state_median0.041871269775568494
sim_compute_sim_state_min0.009414134811167516
sim_render-ego0_max0.003907093299020835
sim_render-ego0_mean0.0036872202979559465
sim_render-ego0_median0.0037330161721184298
sim_render-ego0_min0.00346634667878544
sim_render-ego1_max0.003904177882490706
sim_render-ego1_mean0.0036712039360994953
sim_render-ego1_median0.003702545245422313
sim_render-ego1_min0.0034625992786874384
simulation-passed1
step_physics_max0.4488970935592842
step_physics_mean0.3712445627527142
step_physics_median0.4244263602930938
step_physics_min0.17178806673978023
survival_time_max59.99999999999873
survival_time_mean57.01428571428461
survival_time_min39.09999999999992
No reset possible
5501311307Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:55
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5498411450Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-testingLFt-simhost-erroryesnogpu-prod-070:00:41
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5493711310Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:09
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 50, in init
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 50, in init
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5472911379Martin Cote 🇨🇦baseline-duckietownaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:34:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.569980806905086
survival_time_median59.99999999999873
deviation-center-line_median3.1613669928897816
in-drivable-lane_median3.774999999999963


other stats
agent_compute-ego0_max0.030627305660517787
agent_compute-ego0_mean0.016520843071107762
agent_compute-ego0_median0.011928175311600736
agent_compute-ego0_min0.01159971600071179
complete-iteration_max0.20530106542906496
complete-iteration_mean0.1818056639088481
complete-iteration_median0.18168880619871725
complete-iteration_min0.1585439778088928
deviation-center-line_max3.6432755496573943
deviation-center-line_mean3.1739603138174064
deviation-center-line_min2.729831719832668
deviation-heading_max14.285845212341988
deviation-heading_mean9.77079335711182
deviation-heading_median9.180646068762425
deviation-heading_min6.436036078580436
driven_any_max7.921187500366632
driven_any_mean7.919400990208506
driven_any_median7.921058825806382
driven_any_min7.914298808854628
driven_lanedir_consec_max7.438706423772585
driven_lanedir_consec_mean5.537472571520274
driven_lanedir_consec_min3.57122224849834
driven_lanedir_max7.438706423772585
driven_lanedir_mean7.073357843352568
driven_lanedir_median7.216970831580957
driven_lanedir_min6.420783286475777
get_duckie_state_max1.325496130442242e-06
get_duckie_state_mean1.2071801661253968e-06
get_duckie_state_median1.184450001839694e-06
get_duckie_state_min1.1343245303799569e-06
get_robot_state_max0.0037357447844163063
get_robot_state_mean0.00363868494811999
get_robot_state_median0.003635334730346832
get_robot_state_min0.00354832554736999
get_state_dump_max0.004609756723827962
get_state_dump_mean0.004546897447079445
get_state_dump_median0.004562250779729203
get_state_dump_min0.004453331505031411
get_ui_image_max0.03530319227366324
get_ui_image_mean0.029813113359487824
get_ui_image_median0.029459687891252632
get_ui_image_min0.02502988538178278
in-drivable-lane_max6.949999999999832
in-drivable-lane_mean4.2249999999999055
in-drivable-lane_min2.3999999999998636
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 7.921187500366632, "get_ui_image": 0.026761735110953883, "step_physics": 0.09451536453336006, "survival_time": 59.99999999999873, "driven_lanedir": 7.3963595660433565, "get_state_dump": 0.004453331505031411, "get_robot_state": 0.003576868876727991, "sim_render-ego0": 0.003717772271809828, "get_duckie_state": 1.155367302557114e-06, "in-drivable-lane": 2.80000000000004, "deviation-heading": 6.436036078580436, "agent_compute-ego0": 0.01159971600071179, "complete-iteration": 0.1585439778088928, "set_robot_commands": 0.002150217758229531, "deviation-center-line": 2.729831719832668, "driven_lanedir_consec": 4.102379516691615, "sim_compute_sim_state": 0.00968866066372861, "sim_compute_performance-ego0": 0.001998894419102347}, "LF-norm-zigzag-000-ego0": {"driven_any": 7.9209929682732145, "get_ui_image": 0.03530319227366324, "step_physics": 0.1294028538649128, "survival_time": 59.99999999999873, "driven_lanedir": 7.037582097118556, "get_state_dump": 0.004609756723827962, "get_robot_state": 0.0036938005839656737, "sim_render-ego0": 0.003783269091311542, "get_duckie_state": 1.325496130442242e-06, "in-drivable-lane": 4.749999999999886, "deviation-heading": 10.449931608793351, "agent_compute-ego0": 0.011921200327432524, "complete-iteration": 0.20530106542906496, "set_robot_commands": 0.002151104929445189, "deviation-center-line": 3.6432755496573943, "driven_lanedir_consec": 7.037582097118556, "sim_compute_sim_state": 0.012351089869808098, "sim_compute_performance-ego0": 0.0020005766497762077}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.914298808854628, "get_ui_image": 0.03215764067155138, "step_physics": 0.12663368499844796, "survival_time": 59.99999999999873, "driven_lanedir": 6.420783286475777, "get_state_dump": 0.004568632595942082, "get_robot_state": 0.0037357447844163063, "sim_render-ego0": 0.0037877220198276338, "get_duckie_state": 1.2135327011222743e-06, "in-drivable-lane": 6.949999999999832, "deviation-heading": 14.285845212341988, "agent_compute-ego0": 0.011935150295768948, "complete-iteration": 0.19757067234093303, "set_robot_commands": 0.0022611264682232987, "deviation-center-line": 3.5415713463936744, "driven_lanedir_consec": 3.57122224849834, "sim_compute_sim_state": 0.010368285032235813, "sim_compute_performance-ego0": 0.002039500418352545}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.921124683339549, "get_ui_image": 0.02502988538178278, "step_physics": 0.0884364568422875, "survival_time": 59.99999999999873, "driven_lanedir": 7.438706423772585, "get_state_dump": 0.004555868963516324, "get_robot_state": 0.00354832554736999, "sim_render-ego0": 0.003673473067525821, "get_duckie_state": 1.1343245303799569e-06, "in-drivable-lane": 2.3999999999998636, "deviation-heading": 7.911360528731495, "agent_compute-ego0": 0.030627305660517787, "complete-iteration": 0.16580694005650148, "set_robot_commands": 0.0021866197689288263, "deviation-center-line": 2.781162639385889, "driven_lanedir_consec": 7.438706423772585, "sim_compute_sim_state": 0.005780269065367789, "sim_compute_performance-ego0": 0.001888009531114818}}
set_robot_commands_max0.0022611264682232987
set_robot_commands_mean0.002187267231206711
set_robot_commands_median0.0021688623491870076
set_robot_commands_min0.002150217758229531
sim_compute_performance-ego0_max0.002039500418352545
sim_compute_performance-ego0_mean0.0019817452545864795
sim_compute_performance-ego0_median0.0019997355344392774
sim_compute_performance-ego0_min0.001888009531114818
sim_compute_sim_state_max0.012351089869808098
sim_compute_sim_state_mean0.009547076157785078
sim_compute_sim_state_median0.010028472847982212
sim_compute_sim_state_min0.005780269065367789
sim_render-ego0_max0.0037877220198276338
sim_render-ego0_mean0.003740559112618706
sim_render-ego0_median0.003750520681560685
sim_render-ego0_min0.003673473067525821
simulation-passed1
step_physics_max0.1294028538649128
step_physics_mean0.10974709005975208
step_physics_median0.110574524765904
step_physics_min0.0884364568422875
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5466911424Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:05:33
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5465111398Charlie Gauthier 🇨🇦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:01:31
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5461711439Charlie Gauthier 🇨🇦baseline-RL-sim-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:00:45
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 23, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 43, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 23, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 43, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5458711441Riccardo Tresa 🇮🇹template-randomaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:04:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5040495128145347
survival_time_median4.6499999999999915
deviation-center-line_median0.10620228737371736
in-drivable-lane_median2.674999999999992


other stats
agent_compute-ego0_max0.011288281944062974
agent_compute-ego0_mean0.010657194734056165
agent_compute-ego0_median0.010843160130128666
agent_compute-ego0_min0.009654176731904348
complete-iteration_max0.1897471469381581
complete-iteration_mean0.16620588420785995
complete-iteration_median0.1692033658844174
complete-iteration_min0.13666965812444687
deviation-center-line_max0.16455763165774434
deviation-center-line_mean0.1092777488823554
deviation-center-line_min0.06014878912424249
deviation-heading_max1.819139548495243
deviation-heading_mean0.860079214882594
deviation-heading_median0.6517793617099584
deviation-heading_min0.31761858761521594
driven_any_max3.078535752374254
driven_any_mean1.8526223552312917
driven_any_median1.6219947547238958
driven_any_min1.0879641591031215
driven_lanedir_consec_max0.6643282900394009
driven_lanedir_consec_mean0.4884432809313369
driven_lanedir_consec_min0.28134580805687714
driven_lanedir_max0.6643282900394009
driven_lanedir_mean0.4924956463812869
driven_lanedir_median0.5121542437144349
driven_lanedir_min0.28134580805687714
get_duckie_state_max1.2814998626708984e-06
get_duckie_state_mean1.1646213569327946e-06
get_duckie_state_median1.1533467326989154e-06
get_duckie_state_min1.070292099662449e-06
get_robot_state_max0.003992107179429796
get_robot_state_mean0.003613407767287184
get_robot_state_median0.0035865889287086785
get_robot_state_min0.0032883460323015847
get_state_dump_max0.005659921301735772
get_state_dump_mean0.004676059529363894
get_state_dump_median0.004462136202745538
get_state_dump_min0.004120044410228729
get_ui_image_max0.033161982246067215
get_ui_image_mean0.028868159093680328
get_ui_image_median0.02902331074330374
get_ui_image_min0.02426403264204661
in-drivable-lane_max6.549999999999982
in-drivable-lane_mean3.3499999999999894
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.078535752374254, "get_ui_image": 0.026837937439544293, "step_physics": 0.08995521219470833, "survival_time": 7.84999999999998, "driven_lanedir": 0.28134580805687714, "get_state_dump": 0.004460449460186536, "get_robot_state": 0.003465248059622849, "sim_render-ego0": 0.00355588635311851, "get_duckie_state": 1.2313263325751582e-06, "in-drivable-lane": 6.549999999999982, "deviation-heading": 0.891006057417327, "agent_compute-ego0": 0.010533347914490518, "complete-iteration": 0.15195194226277026, "set_robot_commands": 0.0022538176065758815, "deviation-center-line": 0.13094886067496936, "driven_lanedir_consec": 0.28134580805687714, "sim_compute_sim_state": 0.008960467350633838, "sim_compute_performance-ego0": 0.0018507378010810176}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5806581102909054, "get_ui_image": 0.033161982246067215, "step_physics": 0.1208263609720313, "survival_time": 4.549999999999992, "driven_lanedir": 0.3891269085945981, "get_state_dump": 0.004463822945304539, "get_robot_state": 0.003707929797794508, "sim_render-ego0": 0.0035992523898249087, "get_duckie_state": 1.070292099662449e-06, "in-drivable-lane": 2.449999999999993, "deviation-heading": 1.819139548495243, "agent_compute-ego0": 0.011152972345766812, "complete-iteration": 0.1897471469381581, "set_robot_commands": 0.0020972956781801972, "deviation-center-line": 0.16455763165774434, "driven_lanedir_consec": 0.37291744679479777, "sim_compute_sim_state": 0.00877771170242973, "sim_compute_performance-ego0": 0.00188433605691661}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0879641591031215, "get_ui_image": 0.03120868404706319, "step_physics": 0.1193732884195116, "survival_time": 3.5499999999999954, "driven_lanedir": 0.6643282900394009, "get_state_dump": 0.005659921301735772, "get_robot_state": 0.003992107179429796, "sim_render-ego0": 0.003667102919684516, "get_duckie_state": 1.2814998626708984e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.41255266600258983, "agent_compute-ego0": 0.011288281944062974, "complete-iteration": 0.18645478950606453, "set_robot_commands": 0.002157022555669149, "deviation-center-line": 0.08145571407246535, "driven_lanedir_consec": 0.6643282900394009, "sim_compute_sim_state": 0.007114609082539876, "sim_compute_performance-ego0": 0.001910759343041314}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6633313991568863, "get_ui_image": 0.02426403264204661, "step_physics": 0.08303423225879669, "survival_time": 4.749999999999991, "driven_lanedir": 0.6351815788342716, "get_state_dump": 0.004120044410228729, "get_robot_state": 0.0032883460323015847, "sim_render-ego0": 0.0036108518640200296, "get_duckie_state": 1.0753671328226726e-06, "in-drivable-lane": 2.89999999999999, "deviation-heading": 0.31761858761521594, "agent_compute-ego0": 0.009654176731904348, "complete-iteration": 0.13666965812444687, "set_robot_commands": 0.0022037153442700705, "deviation-center-line": 0.06014878912424249, "driven_lanedir_consec": 0.6351815788342716, "sim_compute_sim_state": 0.00456346074740092, "sim_compute_performance-ego0": 0.0018561060229937236}}
set_robot_commands_max0.0022538176065758815
set_robot_commands_mean0.0021779627961738247
set_robot_commands_median0.0021803689499696097
set_robot_commands_min0.0020972956781801972
sim_compute_performance-ego0_max0.001910759343041314
sim_compute_performance-ego0_mean0.0018754848060081664
sim_compute_performance-ego0_median0.0018702210399551668
sim_compute_performance-ego0_min0.0018507378010810176
sim_compute_sim_state_max0.008960467350633838
sim_compute_sim_state_mean0.007354062220751091
sim_compute_sim_state_median0.007946160392484802
sim_compute_sim_state_min0.00456346074740092
sim_render-ego0_max0.003667102919684516
sim_render-ego0_mean0.003608273381661991
sim_render-ego0_median0.003605052126922469
sim_render-ego0_min0.00355588635311851
simulation-passed1
step_physics_max0.1208263609720313
step_physics_mean0.10329727346126198
step_physics_median0.10466425030710996
step_physics_min0.08303423225879669
survival_time_max7.84999999999998
survival_time_mean5.17499999999999
survival_time_min3.5499999999999954
No reset possible
5457711449Cliff Chandler 🇺🇸template-pytorchaido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:02:22
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego0" aborted with the following error:

error in ego0 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 21, in init
              ||     self.check_gpu_available(context)
              ||   File "solution.py", line 48, in check_gpu_available
              ||     raise Exception(msg)
              || Exception: I need a GPU; bailing.
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 319, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 21, in init
              || |     self.check_gpu_available(context)
              || |   File "solution.py", line 48, in check_gpu_available
              || |     raise Exception(msg)
              || | Exception: I need a GPU; bailing.
              || |
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5455911447Diego Charrez 🇵🇪template-randomaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:04:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5040495128145347
survival_time_median4.6499999999999915
deviation-center-line_median0.10620228737371736
in-drivable-lane_median2.674999999999992


other stats
agent_compute-ego0_max0.01201948653096738
agent_compute-ego0_mean0.011911952542343255
agent_compute-ego0_median0.011982600308126874
agent_compute-ego0_min0.0116631230221519
complete-iteration_max0.2136119422705277
complete-iteration_mean0.18247032843485528
complete-iteration_median0.17808054594529998
complete-iteration_min0.16010827957829343
deviation-center-line_max0.16455763165774434
deviation-center-line_mean0.1092777488823554
deviation-center-line_min0.06014878912424249
deviation-heading_max1.819139548495243
deviation-heading_mean0.860079214882594
deviation-heading_median0.6517793617099584
deviation-heading_min0.31761858761521594
driven_any_max3.078535752374254
driven_any_mean1.8526223552312917
driven_any_median1.6219947547238958
driven_any_min1.0879641591031215
driven_lanedir_consec_max0.6643282900394009
driven_lanedir_consec_mean0.4884432809313369
driven_lanedir_consec_min0.28134580805687714
driven_lanedir_max0.6643282900394009
driven_lanedir_mean0.4924956463812869
driven_lanedir_median0.5121542437144349
driven_lanedir_min0.28134580805687714
get_duckie_state_max1.7260511716206868e-06
get_duckie_state_mean1.4729406699444374e-06
get_duckie_state_median1.4362749335802723e-06
get_duckie_state_min1.2931616409965183e-06
get_robot_state_max0.003879135168051418
get_robot_state_mean0.003740503616794594
get_robot_state_median0.0037323078815487847
get_robot_state_min0.003618263536029392
get_state_dump_max0.0049470166365305586
get_state_dump_mean0.0048076253841555765
get_state_dump_median0.0048006054658427015
get_state_dump_min0.004682273968406345
get_ui_image_max0.03600180149078369
get_ui_image_mean0.029825415703272858
get_ui_image_median0.02838114980194304
get_ui_image_min0.026537561718421646
in-drivable-lane_max6.549999999999982
in-drivable-lane_mean3.3499999999999894
in-drivable-lane_min1.499999999999995
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 3.078535752374254, "get_ui_image": 0.026537561718421646, "step_physics": 0.0958661082424695, "survival_time": 7.84999999999998, "driven_lanedir": 0.28134580805687714, "get_state_dump": 0.0047070089774795725, "get_robot_state": 0.003879135168051418, "sim_render-ego0": 0.0037738809102698216, "get_duckie_state": 1.3460086870797073e-06, "in-drivable-lane": 6.549999999999982, "deviation-heading": 0.891006057417327, "agent_compute-ego0": 0.0116631230221519, "complete-iteration": 0.16010827957829343, "set_robot_commands": 0.002432786965671974, "deviation-center-line": 0.13094886067496936, "driven_lanedir_consec": 0.28134580805687714, "sim_compute_sim_state": 0.00925456421284736, "sim_compute_performance-ego0": 0.0019136787969854811}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.5806581102909054, "get_ui_image": 0.03600180149078369, "step_physics": 0.13955627576164578, "survival_time": 4.549999999999992, "driven_lanedir": 0.3891269085945981, "get_state_dump": 0.004682273968406345, "get_robot_state": 0.0036374511926070504, "sim_render-ego0": 0.003690615944240404, "get_duckie_state": 1.2931616409965183e-06, "in-drivable-lane": 2.449999999999993, "deviation-heading": 1.819139548495243, "agent_compute-ego0": 0.01201948653096738, "complete-iteration": 0.2136119422705277, "set_robot_commands": 0.0022263086360433827, "deviation-center-line": 0.16455763165774434, "driven_lanedir_consec": 0.37291744679479777, "sim_compute_sim_state": 0.009726869023364525, "sim_compute_performance-ego0": 0.0019834974537725034}, "LF-norm-techtrack-000-ego0": {"driven_any": 1.0879641591031215, "get_ui_image": 0.02958134478992886, "step_physics": 0.12468560205565558, "survival_time": 3.5499999999999954, "driven_lanedir": 0.6643282900394009, "get_state_dump": 0.0049470166365305586, "get_robot_state": 0.003618263536029392, "sim_render-ego0": 0.0038001139958699546, "get_duckie_state": 1.5265411800808377e-06, "in-drivable-lane": 1.499999999999995, "deviation-heading": 0.41255266600258983, "agent_compute-ego0": 0.01195254259639316, "complete-iteration": 0.1905313895808326, "set_robot_commands": 0.002169122298558553, "deviation-center-line": 0.08145571407246535, "driven_lanedir_consec": 0.6643282900394009, "sim_compute_sim_state": 0.007598459720611572, "sim_compute_performance-ego0": 0.0020834505558013916}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.6633313991568863, "get_ui_image": 0.027180954813957214, "step_physics": 0.1041676700115204, "survival_time": 4.749999999999991, "driven_lanedir": 0.6351815788342716, "get_state_dump": 0.004894201954205831, "get_robot_state": 0.003827164570490519, "sim_render-ego0": 0.003913961350917816, "get_duckie_state": 1.7260511716206868e-06, "in-drivable-lane": 2.89999999999999, "deviation-heading": 0.31761858761521594, "agent_compute-ego0": 0.012012658019860586, "complete-iteration": 0.1656297023097674, "set_robot_commands": 0.002244025468826294, "deviation-center-line": 0.06014878912424249, "driven_lanedir_consec": 0.6351815788342716, "sim_compute_sim_state": 0.005215734243392944, "sim_compute_performance-ego0": 0.0020792012413342795}}
set_robot_commands_max0.002432786965671974
set_robot_commands_mean0.002268060842275051
set_robot_commands_median0.0022351670524348383
set_robot_commands_min0.002169122298558553
sim_compute_performance-ego0_max0.0020834505558013916
sim_compute_performance-ego0_mean0.002014957011973414
sim_compute_performance-ego0_median0.0020313493475533915
sim_compute_performance-ego0_min0.0019136787969854811
sim_compute_sim_state_max0.009726869023364525
sim_compute_sim_state_mean0.0079489068000541
sim_compute_sim_state_median0.008426511966729465
sim_compute_sim_state_min0.005215734243392944
sim_render-ego0_max0.003913961350917816
sim_render-ego0_mean0.0037946430503244992
sim_render-ego0_median0.003786997453069888
sim_render-ego0_min0.003690615944240404
simulation-passed1
step_physics_max0.13955627576164578
step_physics_mean0.11606891401782282
step_physics_median0.114426636033588
step_physics_min0.0958661082424695
survival_time_max7.84999999999998
survival_time_mean5.17499999999999
survival_time_min3.5499999999999954
No reset possible
5449611485Raphael Jeanreal-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:24:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.029136478381325
survival_time_median41.224999999999454
deviation-center-line_median1.327567471098283
in-drivable-lane_median8.050000000000047


other stats
agent_compute-ego0_max0.013262120031175158
agent_compute-ego0_mean0.012876478715372768
agent_compute-ego0_median0.012890527196768222
agent_compute-ego0_min0.01246274043677947
complete-iteration_max0.30336763177599224
complete-iteration_mean0.2517089474388218
complete-iteration_median0.25113819133541676
complete-iteration_min0.2011917753084613
deviation-center-line_max3.845190662866096
deviation-center-line_mean1.7740031450011835
deviation-center-line_min0.5956869749420718
deviation-heading_max22.77614256288944
deviation-heading_mean10.463668780028964
deviation-heading_median8.005660417307553
deviation-heading_min3.0672117226113045
driven_any_max11.379742025086315
driven_any_mean7.065217388958028
driven_any_median7.794365860130881
driven_any_min1.292395810484036
driven_lanedir_consec_max9.51425967413592
driven_lanedir_consec_mean3.686434905604784
driven_lanedir_consec_min1.1732069915205663
driven_lanedir_max9.51425967413592
driven_lanedir_mean4.061489910255676
driven_lanedir_median2.7792464876831104
driven_lanedir_min1.1732069915205663
get_duckie_state_max1.4404455820719402e-06
get_duckie_state_mean1.3815774067509837e-06
get_duckie_state_median1.3717469967021477e-06
get_duckie_state_min1.3423700515276985e-06
get_robot_state_max0.003905689981248643
get_robot_state_mean0.00382729999947828
get_robot_state_median0.0038508842827179946
get_robot_state_min0.0037017414512284887
get_state_dump_max0.004817773898442586
get_state_dump_mean0.004718856650587036
get_state_dump_median0.004718393293734186
get_state_dump_min0.004620866116437189
get_ui_image_max0.03691586568242028
get_ui_image_mean0.03123435302349577
get_ui_image_median0.031078872450825815
get_ui_image_min0.025863801509911175
in-drivable-lane_max35.89999999999908
in-drivable-lane_mean12.999999999999794
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.379742025086315, "get_ui_image": 0.02835527586003922, "step_physics": 0.16513956099326765, "survival_time": 59.99999999999873, "driven_lanedir": 9.51425967413592, "get_state_dump": 0.004684588792024306, "get_robot_state": 0.0038041016342836454, "sim_render-ego0": 0.0037889746603223303, "get_duckie_state": 1.4009324835301636e-06, "in-drivable-lane": 3.449999999999913, "deviation-heading": 22.77614256288944, "agent_compute-ego0": 0.012665634052044744, "complete-iteration": 0.2322597988042903, "set_robot_commands": 0.0023076859838658823, "deviation-center-line": 3.845190662866096, "driven_lanedir_consec": 9.51425967413592, "sim_compute_sim_state": 0.009353117581509631, "sim_compute_performance-ego0": 0.0020699471260089857}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.292395810484036, "get_ui_image": 0.03691586568242028, "step_physics": 0.2255483525139945, "survival_time": 8.349999999999984, "driven_lanedir": 1.1732069915205663, "get_state_dump": 0.004817773898442586, "get_robot_state": 0.003897666931152344, "sim_render-ego0": 0.003896233581361317, "get_duckie_state": 1.4404455820719402e-06, "in-drivable-lane": 0.0, "deviation-heading": 3.0672117226113045, "agent_compute-ego0": 0.013262120031175158, "complete-iteration": 0.30336763177599224, "set_robot_commands": 0.002334777797971453, "deviation-center-line": 0.6667408826597455, "driven_lanedir_consec": 1.1732069915205663, "sim_compute_sim_state": 0.010519744384856452, "sim_compute_performance-ego0": 0.00207719916389102}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.409122776799236, "get_ui_image": 0.03380246904161242, "step_physics": 0.19464788066016303, "survival_time": 22.450000000000184, "driven_lanedir": 1.928310879832566, "get_state_dump": 0.0047521977954440645, "get_robot_state": 0.003905689981248643, "sim_render-ego0": 0.004032810529073079, "get_duckie_state": 1.342561509874132e-06, "in-drivable-lane": 12.65000000000018, "deviation-heading": 3.1433737602873872, "agent_compute-ego0": 0.0131154203414917, "complete-iteration": 0.27001658386654326, "set_robot_commands": 0.0023790031009250216, "deviation-center-line": 0.5956869749420718, "driven_lanedir_consec": 1.9194777923586197, "sim_compute_sim_state": 0.01110318660736084, "sim_compute_performance-ego0": 0.0021846214930216473}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.179608943462526, "get_ui_image": 0.025863801509911175, "step_physics": 0.14059986818045203, "survival_time": 59.99999999999873, "driven_lanedir": 3.630182095533655, "get_state_dump": 0.004620866116437189, "get_robot_state": 0.0037017414512284887, "sim_render-ego0": 0.00379321894776712, "get_duckie_state": 1.3423700515276985e-06, "in-drivable-lane": 35.89999999999908, "deviation-heading": 12.867947074327716, "agent_compute-ego0": 0.01246274043677947, "complete-iteration": 0.2011917753084613, "set_robot_commands": 0.0022336406374255584, "deviation-center-line": 1.9883940595368204, "driven_lanedir_consec": 2.1387951644040304, "sim_compute_sim_state": 0.005851897470758519, "sim_compute_performance-ego0": 0.0019748733005952477}}
set_robot_commands_max0.0023790031009250216
set_robot_commands_mean0.002313776880046979
set_robot_commands_median0.002321231890918668
set_robot_commands_min0.0022336406374255584
sim_compute_performance-ego0_max0.0021846214930216473
sim_compute_performance-ego0_mean0.0020766602708792254
sim_compute_performance-ego0_median0.002073573144950003
sim_compute_performance-ego0_min0.0019748733005952477
sim_compute_sim_state_max0.01110318660736084
sim_compute_sim_state_mean0.00920698651112136
sim_compute_sim_state_median0.00993643098318304
sim_compute_sim_state_min0.005851897470758519
sim_render-ego0_max0.004032810529073079
sim_render-ego0_mean0.0038778094296309617
sim_render-ego0_median0.0038447262645642183
sim_render-ego0_min0.0037889746603223303
simulation-passed1
step_physics_max0.2255483525139945
step_physics_mean0.1814839155869693
step_physics_median0.17989372082671534
step_physics_min0.14059986818045203
survival_time_max59.99999999999873
survival_time_mean37.699999999999406
survival_time_min8.349999999999984
No reset possible
5445411537Jean-Sébastien Grondin 🇨🇦exercise_state_estimationaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:22:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.654628474877212
survival_time_median29.825000000000127
deviation-center-line_median1.5032097080703255
in-drivable-lane_median9.824999999999823


other stats
agent_compute-ego0_max0.012641368162423543
agent_compute-ego0_mean0.012449666627711136
agent_compute-ego0_median0.012520644720786256
agent_compute-ego0_min0.012116008906848488
complete-iteration_max0.2717552206895433
complete-iteration_mean0.22914554188405936
complete-iteration_median0.2244294999202343
complete-iteration_min0.1959679470062256
deviation-center-line_max4.081755692870062
deviation-center-line_mean1.8085863760785248
deviation-center-line_min0.14617039530338588
deviation-heading_max18.130057362212504
deviation-heading_mean8.04768307282674
deviation-heading_median6.220502199612633
deviation-heading_min1.619670529869187
driven_any_max12.506158020225246
driven_any_mean6.411406033030105
driven_any_median6.028888079532434
driven_any_min1.0816899528303066
driven_lanedir_consec_max7.735578715287257
driven_lanedir_consec_mean3.864212693548672
driven_lanedir_consec_min0.41201510915300465
driven_lanedir_max8.677411395306372
driven_lanedir_mean4.117884317651679
driven_lanedir_median3.671128336324563
driven_lanedir_min0.4518692026512208
get_duckie_state_max1.5700610101094257e-06
get_duckie_state_mean1.474271663567708e-06
get_duckie_state_median1.457793827940078e-06
get_duckie_state_min1.41143798828125e-06
get_robot_state_max0.00395354819635269
get_robot_state_mean0.003808057816527532
get_robot_state_median0.003761493467893757
get_robot_state_min0.003755696133969927
get_state_dump_max0.00483916482758661
get_state_dump_mean0.00476594142377053
get_state_dump_median0.004762446918122755
get_state_dump_min0.00469970703125
get_ui_image_max0.03787660737716586
get_ui_image_mean0.03175331331858906
get_ui_image_median0.031018550686021957
get_ui_image_min0.027099544525146485
in-drivable-lane_max13.949999999999877
in-drivable-lane_mean9.149999999999878
in-drivable-lane_min2.9999999999999893
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.624445710904598, "get_ui_image": 0.028374268220773782, "step_physics": 0.13523580188627635, "survival_time": 23.100000000000193, "driven_lanedir": 3.259666670759892, "get_state_dump": 0.00469970703125, "get_robot_state": 0.003755696133969927, "sim_render-ego0": 0.0038401730374692583, "get_duckie_state": 1.5700610101094257e-06, "in-drivable-lane": 6.600000000000094, "deviation-heading": 3.5772602403770715, "agent_compute-ego0": 0.012116008906848488, "complete-iteration": 0.202736397283896, "set_robot_commands": 0.0022955517531986114, "deviation-center-line": 1.2584886841613188, "driven_lanedir_consec": 3.259666670759892, "sim_compute_sim_state": 0.010270011090305405, "sim_compute_performance-ego0": 0.0020680561436432743}, "LF-norm-zigzag-000-ego0": {"driven_any": 12.506158020225246, "get_ui_image": 0.03787660737716586, "step_physics": 0.18901619684884788, "survival_time": 59.99999999999873, "driven_lanedir": 8.677411395306372, "get_state_dump": 0.00483916482758661, "get_robot_state": 0.00395354819635269, "sim_render-ego0": 0.004024254888618717, "get_duckie_state": 1.474979219587518e-06, "in-drivable-lane": 13.049999999999551, "deviation-heading": 18.130057362212504, "agent_compute-ego0": 0.012641368162423543, "complete-iteration": 0.2717552206895433, "set_robot_commands": 0.002413968063214736, "deviation-center-line": 4.081755692870062, "driven_lanedir_consec": 7.735578715287257, "sim_compute_sim_state": 0.014688214096399668, "sim_compute_performance-ego0": 0.0022149959472097225}, "LF-norm-techtrack-000-ego0": {"driven_any": 7.433330448160269, "get_ui_image": 0.03366283315127013, "step_physics": 0.16950016660117062, "survival_time": 36.55000000000006, "driven_lanedir": 4.082590001889233, "get_state_dump": 0.004703551367983792, "get_robot_state": 0.003762902783565834, "sim_render-ego0": 0.003936775720836035, "get_duckie_state": 1.440608436292638e-06, "in-drivable-lane": 13.949999999999877, "deviation-heading": 8.863744158848194, "agent_compute-ego0": 0.012414433591352783, "complete-iteration": 0.2461226025565726, "set_robot_commands": 0.0022751571702175452, "deviation-center-line": 1.7479307319793325, "driven_lanedir_consec": 4.049590278994532, "sim_compute_sim_state": 0.013678218823313062, "sim_compute_performance-ego0": 0.002102673053741455}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.0816899528303066, "get_ui_image": 0.027099544525146485, "step_physics": 0.13352873992919922, "survival_time": 6.199999999999986, "driven_lanedir": 0.4518692026512208, "get_state_dump": 0.004821342468261719, "get_robot_state": 0.00376008415222168, "sim_render-ego0": 0.0038916263580322263, "get_duckie_state": 1.41143798828125e-06, "in-drivable-lane": 2.9999999999999893, "deviation-heading": 1.619670529869187, "agent_compute-ego0": 0.012626855850219727, "complete-iteration": 0.1959679470062256, "set_robot_commands": 0.0022825145721435546, "deviation-center-line": 0.14617039530338588, "driven_lanedir_consec": 0.41201510915300465, "sim_compute_sim_state": 0.0058761978149414066, "sim_compute_performance-ego0": 0.0019990940093994143}}
set_robot_commands_max0.002413968063214736
set_robot_commands_mean0.002316797889693612
set_robot_commands_median0.002289033162671083
set_robot_commands_min0.0022751571702175452
sim_compute_performance-ego0_max0.0022149959472097225
sim_compute_performance-ego0_mean0.002096204788498467
sim_compute_performance-ego0_median0.002085364598692365
sim_compute_performance-ego0_min0.0019990940093994143
sim_compute_sim_state_max0.014688214096399668
sim_compute_sim_state_mean0.011128160456239884
sim_compute_sim_state_median0.011974114956809234
sim_compute_sim_state_min0.0058761978149414066
sim_render-ego0_max0.004024254888618717
sim_render-ego0_mean0.0039232075012390595
sim_render-ego0_median0.00391420103943413
sim_render-ego0_min0.0038401730374692583
simulation-passed1
step_physics_max0.18901619684884788
step_physics_mean0.15682022631637352
step_physics_median0.1523679842437235
step_physics_min0.13352873992919922
survival_time_max59.99999999999873
survival_time_mean31.462499999999743
survival_time_min6.199999999999986
No reset possible
5438411555Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:20:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9809531250952004
survival_time_median29.400000000000254
deviation-center-line_median0.8720462615221556
in-drivable-lane_median19.97500000000022


other stats
agent_compute-ego0_max0.013414224867004058
agent_compute-ego0_mean0.01325023013250384
agent_compute-ego0_median0.013276512552026417
agent_compute-ego0_min0.013033670558958474
complete-iteration_max0.29783398933832156
complete-iteration_mean0.24963185017437317
complete-iteration_median0.2448138640606768
complete-iteration_min0.21106568323781744
deviation-center-line_max1.2462828584510095
deviation-center-line_mean0.8291607284825531
deviation-center-line_min0.32626753243489137
deviation-heading_max8.724287359625125
deviation-heading_mean5.218471712435031
deviation-heading_median4.742154857458061
deviation-heading_min2.6652897751988753
driven_any_max7.919935175412725
driven_any_mean4.124919169449713
driven_any_median3.7615348092169
driven_any_min1.056671883952328
driven_lanedir_consec_max2.120572199003452
driven_lanedir_consec_mean1.135560719886501
driven_lanedir_consec_min0.45976443035215064
driven_lanedir_max2.1835728363308498
driven_lanedir_mean1.1513108792183502
driven_lanedir_median0.9809531250952004
driven_lanedir_min0.45976443035215064
get_duckie_state_max1.3554293806381646e-06
get_duckie_state_mean1.3217278127265248e-06
get_duckie_state_median1.3161942317032323e-06
get_duckie_state_min1.2990934068614696e-06
get_robot_state_max0.003891033494998573
get_robot_state_mean0.0037896096139179184
get_robot_state_median0.003800401916166008
get_robot_state_min0.003666601128341085
get_state_dump_max0.004901398611332172
get_state_dump_mean0.004798196486325791
get_state_dump_median0.004784286917853225
get_state_dump_min0.00472281349826454
get_ui_image_max0.036297202768905384
get_ui_image_mean0.0315147394578842
get_ui_image_median0.031141917975098212
get_ui_image_min0.027477919112434992
in-drivable-lane_max40.39999999999905
in-drivable-lane_mean21.112499999999873
in-drivable-lane_min4.100000000000002
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.2163774082018834, "get_ui_image": 0.028971883622841567, "step_physics": 0.15146970821479322, "survival_time": 32.800000000000274, "driven_lanedir": 1.152169199661322, "get_state_dump": 0.004758333688094373, "get_robot_state": 0.0037467853482273977, "sim_render-ego0": 0.003797642172199406, "get_duckie_state": 1.3071289526998906e-06, "in-drivable-lane": 22.35000000000026, "deviation-heading": 4.575906682279467, "agent_compute-ego0": 0.013033670558958474, "complete-iteration": 0.2187690426406977, "set_robot_commands": 0.0022126608606161413, "deviation-center-line": 0.9144661167747848, "driven_lanedir_consec": 1.152169199661322, "sim_compute_sim_state": 0.008706462982038385, "sim_compute_performance-ego0": 0.0019890467325846353}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.056671883952328, "get_ui_image": 0.036297202768905384, "step_physics": 0.22024835012235697, "survival_time": 8.999999999999993, "driven_lanedir": 0.45976443035215064, "get_state_dump": 0.004901398611332172, "get_robot_state": 0.003666601128341085, "sim_render-ego0": 0.0037954722978792137, "get_duckie_state": 1.3554293806381646e-06, "in-drivable-lane": 4.100000000000002, "deviation-heading": 2.6652897751988753, "agent_compute-ego0": 0.013414224867004058, "complete-iteration": 0.29783398933832156, "set_robot_commands": 0.002225082882201474, "deviation-center-line": 0.32626753243489137, "driven_lanedir_consec": 0.45976443035215064, "sim_compute_sim_state": 0.011188541328050814, "sim_compute_performance-ego0": 0.0020106181255361653}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.3066922102319163, "get_ui_image": 0.03331195232735486, "step_physics": 0.19791282710553132, "survival_time": 26.00000000000023, "driven_lanedir": 0.8097370505290785, "get_state_dump": 0.00472281349826454, "get_robot_state": 0.003854018484104618, "sim_render-ego0": 0.0038739797478674017, "get_duckie_state": 1.3252595107065738e-06, "in-drivable-lane": 17.600000000000186, "deviation-heading": 4.908403032636656, "agent_compute-ego0": 0.013285402601831475, "complete-iteration": 0.27085868548065595, "set_robot_commands": 0.0022878578208000784, "deviation-center-line": 0.8296264062695263, "driven_lanedir_consec": 0.8097370505290785, "sim_compute_sim_state": 0.009485105177720082, "sim_compute_performance-ego0": 0.002041430024862747}, "LF-norm-small_loop-000-ego0": {"driven_any": 7.919935175412725, "get_ui_image": 0.027477919112434992, "step_physics": 0.14680038066231935, "survival_time": 59.99999999999873, "driven_lanedir": 2.1835728363308498, "get_state_dump": 0.004810240147612077, "get_robot_state": 0.003891033494998573, "sim_render-ego0": 0.003972951220433778, "get_duckie_state": 1.2990934068614696e-06, "in-drivable-lane": 40.39999999999905, "deviation-heading": 8.724287359625125, "agent_compute-ego0": 0.013267622502221356, "complete-iteration": 0.21106568323781744, "set_robot_commands": 0.0023647698633478247, "deviation-center-line": 1.2462828584510095, "driven_lanedir_consec": 2.120572199003452, "sim_compute_sim_state": 0.006269804941029672, "sim_compute_performance-ego0": 0.002124956704297729}}
set_robot_commands_max0.0023647698633478247
set_robot_commands_mean0.0022725928567413796
set_robot_commands_median0.0022564703515007764
set_robot_commands_min0.0022126608606161413
sim_compute_performance-ego0_max0.002124956704297729
sim_compute_performance-ego0_mean0.002041512896820319
sim_compute_performance-ego0_median0.0020260240751994564
sim_compute_performance-ego0_min0.0019890467325846353
sim_compute_sim_state_max0.011188541328050814
sim_compute_sim_state_mean0.008912478607209739
sim_compute_sim_state_median0.009095784079879234
sim_compute_sim_state_min0.006269804941029672
sim_render-ego0_max0.003972951220433778
sim_render-ego0_mean0.0038600113595949497
sim_render-ego0_median0.003835810960033404
sim_render-ego0_min0.0037954722978792137
simulation-passed1
step_physics_max0.22024835012235697
step_physics_mean0.1791078165262502
step_physics_median0.17469126766016227
step_physics_min0.14680038066231935
survival_time_max59.99999999999873
survival_time_mean31.949999999999804
survival_time_min8.999999999999993
No reset possible
5422211630Jean-Sébastien Grondin 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:41:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median7.263562267881615
survival_time_median59.99999999999873
deviation-center-line_median3.952071246945383
in-drivable-lane_median2.9000000000000252


other stats
agent_compute-ego0_max0.05176499840818178
agent_compute-ego0_mean0.03730623757809426
agent_compute-ego0_median0.0427743161747001
agent_compute-ego0_min0.011911319554795036
complete-iteration_max0.32909446969615924
complete-iteration_mean0.26558087896447097
complete-iteration_median0.25968971994893936
complete-iteration_min0.213849606263846
deviation-center-line_max4.045313584464306
deviation-center-line_mean3.931233258472231
deviation-center-line_min3.775476955533853
deviation-heading_max26.81192117804281
deviation-heading_mean25.25800555667081
deviation-heading_median26.346702169662972
deviation-heading_min21.52669670931449
driven_any_max11.361851826987694
driven_any_mean11.118786912690958
driven_any_median11.13402371904689
driven_any_min10.845248385682355
driven_lanedir_consec_max10.185124887636942
driven_lanedir_consec_mean7.431920263047495
driven_lanedir_consec_min5.015431628789807
driven_lanedir_max10.185124887636942
driven_lanedir_mean8.919710515218238
driven_lanedir_median8.971139613701848
driven_lanedir_min7.551437945832314
get_duckie_state_max2.1872572061124193e-06
get_duckie_state_mean2.028493460370142e-06
get_duckie_state_median2.1022920505291812e-06
get_duckie_state_min1.7221325343097871e-06
get_robot_state_max0.00380895358140423
get_robot_state_mean0.0036369938338229697
get_robot_state_median0.003730347313352866
get_robot_state_min0.003278327127181918
get_state_dump_max0.004982337467279363
get_state_dump_mean0.004647164271336412
get_state_dump_median0.004695557932571806
get_state_dump_min0.0042152037529226746
get_ui_image_max0.03615596649748003
get_ui_image_mean0.030101203700088643
get_ui_image_median0.03040536188463883
get_ui_image_min0.02343812453359688
in-drivable-lane_max9.09999999999966
in-drivable-lane_mean3.949999999999915
in-drivable-lane_min0.8999999999999488
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 11.361851826987694, "get_ui_image": 0.02800389372438912, "step_physics": 0.15379394976721517, "survival_time": 59.99999999999873, "driven_lanedir": 10.185124887636942, "get_state_dump": 0.004982337467279363, "get_robot_state": 0.0037184916963982247, "sim_render-ego0": 0.003796459931715839, "get_duckie_state": 2.130084391140521e-06, "in-drivable-lane": 0.8999999999999808, "deviation-heading": 21.52669670931449, "agent_compute-ego0": 0.03937599919022966, "complete-iteration": 0.24757605805980673, "set_robot_commands": 0.002376889904571711, "deviation-center-line": 3.775476955533853, "driven_lanedir_consec": 10.185124887636942, "sim_compute_sim_state": 0.009398738907139863, "sim_compute_performance-ego0": 0.0020416783452728805}, "LF-norm-zigzag-000-ego0": {"driven_any": 10.845248385682355, "get_ui_image": 0.03615596649748003, "step_physics": 0.21270087140485905, "survival_time": 59.99999999999873, "driven_lanedir": 9.234281743158917, "get_state_dump": 0.004697071721809889, "get_robot_state": 0.00380895358140423, "sim_render-ego0": 0.0038794726753711302, "get_duckie_state": 2.1872572061124193e-06, "in-drivable-lane": 0.8999999999999488, "deviation-heading": 26.81192117804281, "agent_compute-ego0": 0.05176499840818178, "complete-iteration": 0.32909446969615924, "set_robot_commands": 0.002357036247539282, "deviation-center-line": 3.949786473965089, "driven_lanedir_consec": 5.015431628789807, "sim_compute_sim_state": 0.011491287360084146, "sim_compute_performance-ego0": 0.00214691662371506}, "LF-norm-techtrack-000-ego0": {"driven_any": 11.278304802050524, "get_ui_image": 0.03280683004488854, "step_physics": 0.19712107663944697, "survival_time": 59.99999999999873, "driven_lanedir": 8.70799748424478, "get_state_dump": 0.004694044143333721, "get_robot_state": 0.0037422029303075073, "sim_render-ego0": 0.003864040382696528, "get_duckie_state": 2.074499709917842e-06, "in-drivable-lane": 4.90000000000007, "deviation-heading": 26.08099204683935, "agent_compute-ego0": 0.011911319554795036, "complete-iteration": 0.27180338183807196, "set_robot_commands": 0.002235714740102039, "deviation-center-line": 3.954356019925677, "driven_lanedir_consec": 8.70799748424478, "sim_compute_sim_state": 0.013253457540278629, "sim_compute_performance-ego0": 0.002087160709200056}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.98974263604326, "get_ui_image": 0.02343812453359688, "step_physics": 0.12396885254897246, "survival_time": 59.99999999999873, "driven_lanedir": 7.551437945832314, "get_state_dump": 0.0042152037529226746, "get_robot_state": 0.003278327127181918, "sim_render-ego0": 0.003412472020577233, "get_duckie_state": 1.7221325343097871e-06, "in-drivable-lane": 9.09999999999966, "deviation-heading": 26.612412292486596, "agent_compute-ego0": 0.04617263315917054, "complete-iteration": 0.213849606263846, "set_robot_commands": 0.001997436909354001, "deviation-center-line": 4.045313584464306, "driven_lanedir_consec": 5.81912705151845, "sim_compute_sim_state": 0.005564360098477506, "sim_compute_performance-ego0": 0.0017262531458388558}}
set_robot_commands_max0.002376889904571711
set_robot_commands_mean0.002241769450391758
set_robot_commands_median0.0022963754938206604
set_robot_commands_min0.001997436909354001
sim_compute_performance-ego0_max0.00214691662371506
sim_compute_performance-ego0_mean0.0020005022060067133
sim_compute_performance-ego0_median0.0020644195272364685
sim_compute_performance-ego0_min0.0017262531458388558
sim_compute_sim_state_max0.013253457540278629
sim_compute_sim_state_mean0.009926960976495036
sim_compute_sim_state_median0.010445013133612005
sim_compute_sim_state_min0.005564360098477506
sim_render-ego0_max0.0038794726753711302
sim_render-ego0_mean0.0037381112525901824
sim_render-ego0_median0.0038302501572061833
sim_render-ego0_min0.003412472020577233
simulation-passed1
step_physics_max0.21270087140485905
step_physics_mean0.17189618759012343
step_physics_median0.17545751320333108
step_physics_min0.12396885254897246
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
5417011645Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:20:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.3364636123589113
survival_time_median28.00000000000024
deviation-center-line_median0.8322795495581929
in-drivable-lane_median16.75000000000018


other stats
agent_compute-ego0_max0.012740789901009209
agent_compute-ego0_mean0.012550192369916842
agent_compute-ego0_median0.012568711513017038
agent_compute-ego0_min0.012322556552624082
complete-iteration_max0.2301350803309699
complete-iteration_mean0.2114528922327136
complete-iteration_median0.2151277371043008
complete-iteration_min0.18542101439128297
deviation-center-line_max2.459355875919366
deviation-center-line_mean1.0942421500950847
deviation-center-line_min0.25305362534458753
deviation-heading_max8.306769296486179
deviation-heading_mean4.38719027774679
deviation-heading_median3.792300135944489
deviation-heading_min1.6573915426120025
driven_any_max12.147898689835362
driven_any_mean6.5177280483613576
driven_any_median5.69162032208438
driven_any_min2.5397728594413067
driven_lanedir_consec_max4.779758834554187
driven_lanedir_consec_mean2.53777395194558
driven_lanedir_consec_min0.6984097485103071
driven_lanedir_max6.681022152571893
driven_lanedir_mean3.013583882568198
driven_lanedir_median2.337451814595297
driven_lanedir_min0.6984097485103071
get_duckie_state_max1.7189078314607496e-06
get_duckie_state_mean1.5347123279156416e-06
get_duckie_state_median1.589826855561852e-06
get_duckie_state_min1.240287769078113e-06
get_robot_state_max0.004060220758087133
get_robot_state_mean0.003910247476056744
get_robot_state_median0.003954886099159133
get_robot_state_min0.003670996947821578
get_state_dump_max0.005320935721798404
get_state_dump_mean0.005096895193059778
get_state_dump_median0.005100276260221409
get_state_dump_min0.004866092529997892
get_ui_image_max0.03505184201332548
get_ui_image_mean0.03143573571637741
get_ui_image_median0.031377603777748536
get_ui_image_min0.02793589329668708
in-drivable-lane_max29.1999999999993
in-drivable-lane_mean17.724999999999934
in-drivable-lane_min8.200000000000065
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.147898689835362, "get_ui_image": 0.03006726577815962, "step_physics": 0.13358933979228177, "survival_time": 59.99999999999873, "driven_lanedir": 6.681022152571893, "get_state_dump": 0.005320935721798404, "get_robot_state": 0.004060220758087133, "sim_render-ego0": 0.004148046738102871, "get_duckie_state": 1.7110155980652517e-06, "in-drivable-lane": 29.1999999999993, "deviation-heading": 8.306769296486179, "agent_compute-ego0": 0.012740789901009209, "complete-iteration": 0.20530490871273807, "set_robot_commands": 0.002415678284745927, "deviation-center-line": 2.459355875919366, "driven_lanedir_consec": 4.779758834554187, "sim_compute_sim_state": 0.010604089146152722, "sim_compute_performance-ego0": 0.0022483020896816334}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.534921977225469, "get_ui_image": 0.03505184201332548, "step_physics": 0.14970170484020004, "survival_time": 32.600000000000286, "driven_lanedir": 3.652891666139507, "get_state_dump": 0.004866092529997892, "get_robot_state": 0.003670996947821578, "sim_render-ego0": 0.003831566566712641, "get_duckie_state": 1.240287769078113e-06, "in-drivable-lane": 14.150000000000158, "deviation-heading": 4.913535222061783, "agent_compute-ego0": 0.012322556552624082, "complete-iteration": 0.22495056549586345, "set_robot_commands": 0.0022489655802845044, "deviation-center-line": 1.2738194406903778, "driven_lanedir_consec": 3.652891666139507, "sim_compute_sim_state": 0.011151740598459522, "sim_compute_performance-ego0": 0.002013798694698223}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.5397728594413067, "get_ui_image": 0.032687941777337456, "step_physics": 0.1574616210976827, "survival_time": 14.500000000000073, "driven_lanedir": 1.0220119630510869, "get_state_dump": 0.005214084874313721, "get_robot_state": 0.003914084221489241, "sim_render-ego0": 0.003977314303421073, "get_duckie_state": 1.7189078314607496e-06, "in-drivable-lane": 8.200000000000065, "deviation-heading": 2.671065049827195, "agent_compute-ego0": 0.012664708075244812, "complete-iteration": 0.2301350803309699, "set_robot_commands": 0.0023259198952376636, "deviation-center-line": 0.3907396584260081, "driven_lanedir_consec": 1.0200355585783156, "sim_compute_sim_state": 0.009631904949437302, "sim_compute_performance-ego0": 0.00215118037875985}, "LF-norm-small_loop-000-ego0": {"driven_any": 4.848318666943291, "get_ui_image": 0.02793589329668708, "step_physics": 0.1207310151952162, "survival_time": 23.4000000000002, "driven_lanedir": 0.6984097485103071, "get_state_dump": 0.004986467646129096, "get_robot_state": 0.003995687976829024, "sim_render-ego0": 0.0040476764443078276, "get_duckie_state": 1.468638113058452e-06, "in-drivable-lane": 19.350000000000204, "deviation-heading": 1.6573915426120025, "agent_compute-ego0": 0.012472714950789266, "complete-iteration": 0.18542101439128297, "set_robot_commands": 0.0024069275683177307, "deviation-center-line": 0.25305362534458753, "driven_lanedir_consec": 0.6984097485103071, "sim_compute_sim_state": 0.006578538463568128, "sim_compute_performance-ego0": 0.0021690651297823454}}
set_robot_commands_max0.002415678284745927
set_robot_commands_mean0.0023493728321464565
set_robot_commands_median0.002366423731777697
set_robot_commands_min0.0022489655802845044
sim_compute_performance-ego0_max0.0022483020896816334
sim_compute_performance-ego0_mean0.002145586573230513
sim_compute_performance-ego0_median0.0021601227542710974
sim_compute_performance-ego0_min0.002013798694698223
sim_compute_sim_state_max0.011151740598459522
sim_compute_sim_state_mean0.009491568289404418
sim_compute_sim_state_median0.010117997047795013
sim_compute_sim_state_min0.006578538463568128
sim_render-ego0_max0.004148046738102871
sim_render-ego0_mean0.004001151013136103
sim_render-ego0_median0.00401249537386445
sim_render-ego0_min0.003831566566712641
simulation-passed1
step_physics_max0.1574616210976827
step_physics_mean0.14037092023134518
step_physics_median0.1416455223162409
step_physics_min0.1207310151952162
survival_time_max59.99999999999873
survival_time_mean32.62499999999982
survival_time_min14.500000000000073
No reset possible
5412511656Mo Kleit 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:19:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median3.039720061844742
survival_time_median22.77500000000019
deviation-center-line_median0.9774675372163906
in-drivable-lane_median12.6500000000001


other stats
agent_compute-ego0_max0.013416762499846227
agent_compute-ego0_mean0.012882089715049738
agent_compute-ego0_median0.012937172369474803
agent_compute-ego0_min0.012237251621403109
complete-iteration_max0.28550716374058127
complete-iteration_mean0.25405810549889
complete-iteration_median0.25988449425369425
complete-iteration_min0.21095626974759035
deviation-center-line_max3.0173482630062765
deviation-center-line_mean1.3001528580525543
deviation-center-line_min0.22832809477115945
deviation-heading_max11.640971093584495
deviation-heading_mean5.530212841997611
deviation-heading_median4.525919131389243
deviation-heading_min1.4280420116274617
driven_any_max21.36120302412145
driven_any_mean10.79271385026377
driven_any_median8.585004914953679
driven_any_min4.639642547026262
driven_lanedir_consec_max12.8259482591177
driven_lanedir_consec_mean4.933712520478678
driven_lanedir_consec_min0.8294616991075259
driven_lanedir_max12.911628894669098
driven_lanedir_mean5.4671689250678694
driven_lanedir_median4.063792553247427
driven_lanedir_min0.8294616991075259
get_duckie_state_max1.5331247484174178e-06
get_duckie_state_mean1.4196201970341691e-06
get_duckie_state_median1.411048345939488e-06
get_duckie_state_min1.323259347840283e-06
get_robot_state_max0.003990984702295111
get_robot_state_mean0.003853059610100304
get_robot_state_median0.003863688360222394
get_robot_state_min0.0036938770176613168
get_state_dump_max0.0051356202872224555
get_state_dump_mean0.004980380246348082
get_state_dump_median0.0049982253545953455
get_state_dump_min0.004789449988979183
get_ui_image_max0.03720493954365739
get_ui_image_mean0.032067599332525475
get_ui_image_median0.03210067889698993
get_ui_image_min0.026864099992464665
in-drivable-lane_max19.09999999999921
in-drivable-lane_mean12.662499999999866
in-drivable-lane_min6.250000000000063
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 21.36120302412145, "get_ui_image": 0.02958407979102651, "step_physics": 0.16504498608444257, "survival_time": 54.89999999999902, "driven_lanedir": 12.911628894669098, "get_state_dump": 0.00501806352006185, "get_robot_state": 0.0039333030242503396, "sim_render-ego0": 0.003984643720951375, "get_duckie_state": 1.5331247484174178e-06, "in-drivable-lane": 19.09999999999921, "deviation-heading": 11.640971093584495, "agent_compute-ego0": 0.013097745272330092, "complete-iteration": 0.2349391400108997, "set_robot_commands": 0.0023155394633104847, "deviation-center-line": 3.0173482630062765, "driven_lanedir_consec": 12.8259482591177, "sim_compute_sim_state": 0.00971515055023832, "sim_compute_performance-ego0": 0.0021454973368345335}, "LF-norm-zigzag-000-ego0": {"driven_any": 6.060854602199331, "get_ui_image": 0.03720493954365739, "step_physics": 0.20729689612577024, "survival_time": 16.400000000000098, "driven_lanedir": 0.8294616991075259, "get_state_dump": 0.004978387189128841, "get_robot_state": 0.0037940736961944487, "sim_render-ego0": 0.00398207145621349, "get_duckie_state": 1.323259347840283e-06, "in-drivable-lane": 13.500000000000073, "deviation-heading": 1.4280420116274617, "agent_compute-ego0": 0.01277659946661952, "complete-iteration": 0.28550716374058127, "set_robot_commands": 0.0022838260627445115, "deviation-center-line": 0.22832809477115945, "driven_lanedir_consec": 0.8294616991075259, "sim_compute_sim_state": 0.010969829414390866, "sim_compute_performance-ego0": 0.002124624049409907}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.639642547026262, "get_ui_image": 0.03461727800295335, "step_physics": 0.20756049211635147, "survival_time": 12.850000000000048, "driven_lanedir": 2.2801989221476417, "get_state_dump": 0.0051356202872224555, "get_robot_state": 0.003990984702295111, "sim_render-ego0": 0.004133203232935233, "get_duckie_state": 1.474868419558503e-06, "in-drivable-lane": 6.250000000000063, "deviation-heading": 1.981328125067261, "agent_compute-ego0": 0.013416762499846227, "complete-iteration": 0.28482984849648885, "set_robot_commands": 0.002440113429875337, "deviation-center-line": 0.4767625295135355, "driven_lanedir_consec": 2.2743895017562195, "sim_compute_sim_state": 0.01115173240040624, "sim_compute_performance-ego0": 0.0022801022196924964}, "LF-norm-small_loop-000-ego0": {"driven_any": 11.109155227708026, "get_ui_image": 0.026864099992464665, "step_physics": 0.1490302902378448, "survival_time": 29.15000000000028, "driven_lanedir": 5.8473861843472115, "get_state_dump": 0.004789449988979183, "get_robot_state": 0.0036938770176613168, "sim_render-ego0": 0.003802133749609124, "get_duckie_state": 1.347228272320473e-06, "in-drivable-lane": 11.800000000000129, "deviation-heading": 7.070510137711223, "agent_compute-ego0": 0.012237251621403109, "complete-iteration": 0.21095626974759035, "set_robot_commands": 0.0021691812227850093, "deviation-center-line": 1.4781725449192458, "driven_lanedir_consec": 3.8050506219332663, "sim_compute_sim_state": 0.006294059426817176, "sim_compute_performance-ego0": 0.0019887747013405577}}
set_robot_commands_max0.002440113429875337
set_robot_commands_mean0.0023021650446788354
set_robot_commands_median0.002299682763027498
set_robot_commands_min0.0021691812227850093
sim_compute_performance-ego0_max0.0022801022196924964
sim_compute_performance-ego0_mean0.0021347495768193735
sim_compute_performance-ego0_median0.0021350606931222205
sim_compute_performance-ego0_min0.0019887747013405577
sim_compute_sim_state_max0.01115173240040624
sim_compute_sim_state_mean0.00953269294796315
sim_compute_sim_state_median0.010342489982314594
sim_compute_sim_state_min0.006294059426817176
sim_render-ego0_max0.004133203232935233
sim_render-ego0_mean0.003975513039927305
sim_render-ego0_median0.003983357588582433
sim_render-ego0_min0.003802133749609124
simulation-passed1
step_physics_max0.20756049211635147
step_physics_mean0.1822331661411023
step_physics_median0.1861709411051064
step_physics_min0.1490302902378448
survival_time_max54.89999999999902
survival_time_mean28.324999999999857
survival_time_min12.850000000000048
No reset possible
5402711687Dishank Bansal 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simerroryesnogpu-prod-070:30:19
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 577, in run_episode
    await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 331, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "step".
                   |
                   || Traceback (most recent call last):
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_euclidean_group.py", line 73, in belongs
                   ||     self.SOn.belongs(R)
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_orthogonal_group.py", line 39, in belongs
                   ||     assert_allclose(det, 1, err_msg="I expect the determinant to be +1.")
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
                   ||     assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 764, in assert_array_compare
                   ||     flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 740, in func_assert_same_pos
                   ||     raise AssertionError(msg)
                   || AssertionError:
                   || Not equal to tolerance rtol=1e-07, atol=0
                   || I expect the determinant to be +1.
                   || x and y nan location mismatch:
                   ||  x: array(nan)
                   ||  y: array(1)
                   ||
                   || During handling of the above exception, another exception occurred:
                   ||
                   || Traceback (most recent call last):
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 355, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 531, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 510, in on_received_step
                   ||     self.update_physics_and_observations(until=data.until, context=context)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 560, in update_physics_and_observations
                   ||     pc.integrate(delta_time)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 217, in integrate
                   ||     self.state = self.state.integrate(dt, self.last_commands.wheels)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/dynamics_delay.py", line 74, in integrate
                   ||     state2 = self.state.integrate(dt, use_commands)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/pwm_dynamics.py", line 199, in integrate
                   ||     s1 = GenericKinematicsSE2.integrate(self, dt, commands_se2)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/generic_kinematics.py", line 48, in integrate
                   ||     return GenericKinematicsSE2(c1, t1)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/generic_kinematics.py", line 26, in __init__
                   ||     geo.SE2.belongs(q0)
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_euclidean_group.py", line 76, in belongs
                   ||     raise ValueError(msg)
                   || ValueError: The rotation is not a rotation:
                   || [[nan nan]
                   ||  [nan nan]]
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
5396811731Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:19:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.8790951756018943
survival_time_median23.6250000000002
deviation-center-line_median0.9850105487640796
in-drivable-lane_median9.375000000000124


other stats
agent_compute-ego0_max0.01260748038104936
agent_compute-ego0_mean0.012396267639297492
agent_compute-ego0_median0.012557791223204676
agent_compute-ego0_min0.011862007729731255
complete-iteration_max0.23725618192570108
complete-iteration_mean0.20573544816693828
complete-iteration_median0.2079825742516494
complete-iteration_min0.16972046223875323
deviation-center-line_max1.3122181463677534
deviation-center-line_mean0.999095682283246
deviation-center-line_min0.7141434852370707
deviation-heading_max5.846409649557609
deviation-heading_mean4.101548718084948
deviation-heading_median3.671262907379508
deviation-heading_min3.217259408023168
driven_any_max13.179265104687545
driven_any_mean6.435138865602953
driven_any_median4.576671748678638
driven_any_min3.407946860366991
driven_lanedir_consec_max3.840525881701143
driven_lanedir_consec_mean2.959276728088859
driven_lanedir_consec_min2.2383906794505064
driven_lanedir_max3.840525881701143
driven_lanedir_mean2.9603726224838223
driven_lanedir_median2.88128696439182
driven_lanedir_min2.2383906794505064
get_duckie_state_max1.9809258797755937e-06
get_duckie_state_mean1.9352498901874345e-06
get_duckie_state_median1.937465350068395e-06
get_duckie_state_min1.8851429808373543e-06
get_robot_state_max0.003978961711124417
get_robot_state_mean0.003897194343040898
get_robot_state_median0.0039094946298695
get_robot_state_min0.003790826401300176
get_state_dump_max0.004979767668356407
get_state_dump_mean0.004855991151872206
get_state_dump_median0.0048820774210401804
get_state_dump_min0.004680042097052056
get_ui_image_max0.03790365919774892
get_ui_image_mean0.03237208346924373
get_ui_image_median0.032415276817301476
get_ui_image_min0.026754121044623037
in-drivable-lane_max43.74999999999864
in-drivable-lane_mean16.98749999999974
in-drivable-lane_min5.4500000000000774
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 13.179265104687545, "get_ui_image": 0.030311685319149326, "step_physics": 0.12056433191704412, "survival_time": 59.99999999999873, "driven_lanedir": 3.840525881701143, "get_state_dump": 0.004979767668356407, "get_robot_state": 0.003978961711124417, "sim_render-ego0": 0.004018687090210672, "get_duckie_state": 1.9202522195249076e-06, "in-drivable-lane": 43.74999999999864, "deviation-heading": 3.801955060530188, "agent_compute-ego0": 0.012529128596347932, "complete-iteration": 0.1910734730497387, "set_robot_commands": 0.00239638702557744, "deviation-center-line": 1.0763723143707675, "driven_lanedir_consec": 3.840525881701143, "sim_compute_sim_state": 0.010019116755032122, "sim_compute_performance-ego0": 0.00218163560967362}, "LF-norm-zigzag-000-ego0": {"driven_any": 5.209773281241665, "get_ui_image": 0.03790365919774892, "step_physics": 0.1581270186047386, "survival_time": 26.900000000000247, "driven_lanedir": 3.4657314293239145, "get_state_dump": 0.0049236236565188265, "get_robot_state": 0.003909757274422442, "sim_render-ego0": 0.003985857919328509, "get_duckie_state": 1.9546784806118826e-06, "in-drivable-lane": 10.050000000000123, "deviation-heading": 5.846409649557609, "agent_compute-ego0": 0.01258645385006142, "complete-iteration": 0.23725618192570108, "set_robot_commands": 0.002340245114186699, "deviation-center-line": 1.3122181463677534, "driven_lanedir_consec": 3.4613478517440623, "sim_compute_sim_state": 0.011295346470622272, "sim_compute_performance-ego0": 0.0020890474761793036}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.943570216115611, "get_ui_image": 0.03451886831545362, "step_physics": 0.14923705014528013, "survival_time": 20.35000000000015, "driven_lanedir": 2.296842499459726, "get_state_dump": 0.004840531185561535, "get_robot_state": 0.003909231985316557, "sim_render-ego0": 0.003968539775586596, "get_duckie_state": 1.8851429808373543e-06, "in-drivable-lane": 8.700000000000124, "deviation-heading": 3.217259408023168, "agent_compute-ego0": 0.01260748038104936, "complete-iteration": 0.22489167545356004, "set_robot_commands": 0.002386374216453702, "deviation-center-line": 0.8936487831573918, "driven_lanedir_consec": 2.296842499459726, "sim_compute_sim_state": 0.01118001575563468, "sim_compute_performance-ego0": 0.0021480666656120153}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.407946860366991, "get_ui_image": 0.026754121044623037, "step_physics": 0.1079804883158879, "survival_time": 16.800000000000104, "driven_lanedir": 2.2383906794505064, "get_state_dump": 0.004680042097052056, "get_robot_state": 0.003790826401300176, "sim_render-ego0": 0.0038770030443321703, "get_duckie_state": 1.9809258797755937e-06, "in-drivable-lane": 5.4500000000000774, "deviation-heading": 3.5405707542288285, "agent_compute-ego0": 0.011862007729731255, "complete-iteration": 0.16972046223875323, "set_robot_commands": 0.002352256803201285, "deviation-center-line": 0.7141434852370707, "driven_lanedir_consec": 2.2383906794505064, "sim_compute_sim_state": 0.006261100401156734, "sim_compute_performance-ego0": 0.0020719317371130697}}
set_robot_commands_max0.00239638702557744
set_robot_commands_mean0.0023688157898547816
set_robot_commands_median0.0023693155098274935
set_robot_commands_min0.002340245114186699
sim_compute_performance-ego0_max0.00218163560967362
sim_compute_performance-ego0_mean0.0021226703721445024
sim_compute_performance-ego0_median0.002118557070895659
sim_compute_performance-ego0_min0.0020719317371130697
sim_compute_sim_state_max0.011295346470622272
sim_compute_sim_state_mean0.009688894845611452
sim_compute_sim_state_median0.0105995662553334
sim_compute_sim_state_min0.006261100401156734
sim_render-ego0_max0.004018687090210672
sim_render-ego0_mean0.003962521957364486
sim_render-ego0_median0.003977198847457552
sim_render-ego0_min0.0038770030443321703
simulation-passed1
step_physics_max0.1581270186047386
step_physics_mean0.1339772222457377
step_physics_median0.13490069103116217
step_physics_min0.1079804883158879
survival_time_max59.99999999999873
survival_time_mean31.01249999999981
survival_time_min16.800000000000104
No reset possible
5394411703Mo Kleit 🇨🇦real-exercise-2aido5-LF-sim-validationLFv-simsuccessyesnogpu-prod-070:18:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.7866672250898774
survival_time_median18.525000000000126
deviation-center-line_median0.8198288837690222
in-drivable-lane_median7.925000000000079


other stats
agent_compute-ego0_max0.013061178285022117
agent_compute-ego0_mean0.012656887854896636
agent_compute-ego0_median0.012624337186191912
agent_compute-ego0_min0.012317698762180604
complete-iteration_max0.3272856488114312
complete-iteration_mean0.26583040006924696
complete-iteration_median0.25171160787133795
complete-iteration_min0.23261273572288088
deviation-center-line_max1.5205162257416307
deviation-center-line_mean0.8863981615703806
deviation-center-line_min0.3854186530018473
deviation-heading_max10.69335099781207
deviation-heading_mean5.175750268007791
deviation-heading_median3.82741186073869
deviation-heading_min2.3548263527417133
driven_any_max15.839861094118952
driven_any_mean6.76318516775188
driven_any_median4.636028376764321
driven_any_min1.9408228233599272
driven_lanedir_consec_max3.4864905006939155
driven_lanedir_consec_mean2.1370091089037
driven_lanedir_consec_min1.4882114847411274
driven_lanedir_max3.799888869736652
driven_lanedir_mean2.592357035066303
driven_lanedir_median2.540663892893717
driven_lanedir_min1.4882114847411274
get_duckie_state_max1.5553973969959075e-06
get_duckie_state_mean1.5446734314630931e-06
get_duckie_state_median1.5476928927957824e-06
get_duckie_state_min1.5279105432649005e-06
get_robot_state_max0.004196279329763618
get_robot_state_mean0.004173664040929074
get_robot_state_median0.004171994140377868
get_robot_state_min0.0041543885531969416
get_state_dump_max0.0054066019846980735
get_state_dump_mean0.0052997681347050785
get_state_dump_median0.00528998031909
get_state_dump_min0.005212509915942238
get_ui_image_max0.04035647000585284
get_ui_image_mean0.03410656751571776
get_ui_image_median0.033145701055094
get_ui_image_min0.029778397946830196
in-drivable-lane_max41.299999999999066
in-drivable-lane_mean14.549999999999804
in-drivable-lane_min1.0500000000000016
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 5.007806439878018, "get_ui_image": 0.03157146711994831, "step_physics": 0.16546208995924258, "survival_time": 19.900000000000148, "driven_lanedir": 3.4864905006939155, "get_state_dump": 0.0054066019846980735, "get_robot_state": 0.004196279329763618, "sim_render-ego0": 0.004216788705429039, "get_duckie_state": 1.5279105432649005e-06, "in-drivable-lane": 5.4500000000000774, "deviation-heading": 4.317383997301968, "agent_compute-ego0": 0.012612203010042808, "complete-iteration": 0.23993960478550808, "set_robot_commands": 0.0025010491373544945, "deviation-center-line": 0.9829317553129155, "driven_lanedir_consec": 3.4864905006939155, "sim_compute_sim_state": 0.011561988290389976, "sim_compute_performance-ego0": 0.002311314556532934}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.9408228233599272, "get_ui_image": 0.04035647000585284, "step_physics": 0.2444348902929397, "survival_time": 8.349999999999984, "driven_lanedir": 1.5948372850935184, "get_state_dump": 0.005212509915942238, "get_robot_state": 0.00417214206286839, "sim_render-ego0": 0.004253422930127098, "get_duckie_state": 1.5553973969959075e-06, "in-drivable-lane": 1.0500000000000016, "deviation-heading": 3.337439724175414, "agent_compute-ego0": 0.012636471362341018, "complete-iteration": 0.3272856488114312, "set_robot_commands": 0.0025021121615455264, "deviation-center-line": 0.6567260122251289, "driven_lanedir_consec": 1.5948372850935184, "sim_compute_sim_state": 0.01125778044973101, "sim_compute_performance-ego0": 0.0023624811853681293}, "LF-norm-techtrack-000-ego0": {"driven_any": 4.264250313650623, "get_ui_image": 0.0347199349902397, "step_physics": 0.18500491214352985, "survival_time": 17.15000000000011, "driven_lanedir": 1.4882114847411274, "get_state_dump": 0.005354957525120225, "get_robot_state": 0.004171846217887346, "sim_render-ego0": 0.004200935363769531, "get_duckie_state": 1.5441761460415153e-06, "in-drivable-lane": 10.40000000000008, "deviation-heading": 2.3548263527417133, "agent_compute-ego0": 0.013061178285022117, "complete-iteration": 0.2634836109571679, "set_robot_commands": 0.002513465493224388, "deviation-center-line": 0.3854186530018473, "driven_lanedir_consec": 1.4882114847411274, "sim_compute_sim_state": 0.012090510407159494, "sim_compute_performance-ego0": 0.0022649570953014284}, "LF-norm-small_loop-000-ego0": {"driven_any": 15.839861094118952, "get_ui_image": 0.029778397946830196, "step_physics": 0.16531952374384465, "survival_time": 59.99999999999873, "driven_lanedir": 3.799888869736652, "get_state_dump": 0.005225003113059775, "get_robot_state": 0.0041543885531969416, "sim_render-ego0": 0.004192410063287003, "get_duckie_state": 1.5512096395500494e-06, "in-drivable-lane": 41.299999999999066, "deviation-heading": 10.69335099781207, "agent_compute-ego0": 0.012317698762180604, "complete-iteration": 0.23261273572288088, "set_robot_commands": 0.0024583986855664917, "deviation-center-line": 1.5205162257416307, "driven_lanedir_consec": 1.9784971650862369, "sim_compute_sim_state": 0.006835323090755771, "sim_compute_performance-ego0": 0.002236462354064484}}
set_robot_commands_max0.002513465493224388
set_robot_commands_mean0.002493756369422725
set_robot_commands_median0.0025015806494500105
set_robot_commands_min0.0024583986855664917
sim_compute_performance-ego0_max0.0023624811853681293
sim_compute_performance-ego0_mean0.0022938037978167437
sim_compute_performance-ego0_median0.002288135825917181
sim_compute_performance-ego0_min0.002236462354064484
sim_compute_sim_state_max0.012090510407159494
sim_compute_sim_state_mean0.010436400559509064
sim_compute_sim_state_median0.011409884370060491
sim_compute_sim_state_min0.006835323090755771
sim_render-ego0_max0.004253422930127098
sim_render-ego0_mean0.004215889265653168
sim_render-ego0_median0.004208862034599285
sim_render-ego0_min0.004192410063287003
simulation-passed1
step_physics_max0.2444348902929397
step_physics_mean0.1900553540348892
step_physics_median0.1752335010513862
step_physics_min0.16531952374384465
survival_time_max59.99999999999873
survival_time_mean26.34999999999974
survival_time_min8.349999999999984
No reset possible
5392511714Yishu Malhotra 🇨🇦sim-exercise-2aido5-LF-sim-validationLFv-simhost-erroryesnogpu-prod-070:03:22
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible