Duckietown Challenges Home Challenges Submissions

Evaluator 2280

ID2280
evaluatormont02-5bd91ed070bc-1
ownerI don't have one 😀
machineaido-ms-7b09
processmont02-5bd91ed070bc-1
versiond-c:6.1.23;d-c-r:6.0.59
first heard
last heard
statusinactive
# evaluating
# success3 41970
# timeout1 42017
# failed6 41974
# error1 41972
# aborted1 41981
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
4201711352Charlie Gauthier 🇨🇦Baseline solution using reinforcement learningaido5-LF-sim-validationLFv-simtimeoutnomont02-5bd91ed070bc-1----No reset possible
4201111350Martin Cote 🇨🇦template-randomaido5-LF-sim-validationLFv-simsuccessnomont02-5bd91ed070bc-10:04:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6440373265804349
survival_time_median4.7749999999999915
deviation-center-line_median0.10361109150784235
in-drivable-lane_median2.5499999999999914


other stats
agent_compute-ego_max0.018332974664096176
agent_compute-ego_mean0.01705309277789829
agent_compute-ego_median0.016893828880539946
agent_compute-ego_min0.016091738686417088
complete-iteration_max0.22911674400855755
complete-iteration_mean0.2088521389351591
complete-iteration_median0.20596528856890228
complete-iteration_min0.19436123459427448
deviation-center-line_max0.15583868303837567
deviation-center-line_mean0.10879855071843549
deviation-center-line_min0.07213333681968147
deviation-heading_max0.976837469261332
deviation-heading_mean0.4922932418762878
deviation-heading_median0.3609623398756611
deviation-heading_min0.27041081849249704
driven_any_max2.8894611145876925
driven_any_mean1.8510551780477624
driven_any_median1.998666635086524
driven_any_min0.5174263274303096
driven_lanedir_consec_max0.8904446665936326
driven_lanedir_consec_mean0.6302364342579737
driven_lanedir_consec_min0.3424264172773923
driven_lanedir_max0.8904446665936326
driven_lanedir_mean0.6302364342579737
driven_lanedir_median0.6440373265804349
driven_lanedir_min0.3424264172773923
get_duckie_state_max2.795252306707974e-06
get_duckie_state_mean2.578663059171315e-06
get_duckie_state_median2.5278706396305693e-06
get_duckie_state_min2.463658650716146e-06
get_robot_state_max0.01740405477326492
get_robot_state_mean0.015788773145011488
get_robot_state_median0.015615428429464088
get_robot_state_min0.014520180947852858
get_state_dump_max0.014733141866223563
get_state_dump_mean0.014135207173586049
get_state_dump_median0.01401049851323551
get_state_dump_min0.013786689801649614
get_ui_image_max0.039668527142754914
get_ui_image_mean0.03704448969624342
get_ui_image_median0.0368779785797176
get_ui_image_min0.03475347448278356
in-drivable-lane_max4.699999999999983
in-drivable-lane_mean2.5374999999999917
in-drivable-lane_min0.3500000000000003
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.1619321797959306, "get_ui_image": 0.037807985887689105, "step_physics": 0.0799028226884745, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6085881804686812, "get_state_dump": 0.014207367169654976, "sim_render-ego": 0.006152941008745614, "get_robot_state": 0.0156447685371011, "get_duckie_state": 2.590276427188162e-06, "in-drivable-lane": 1.1999999999999966, "agent_compute-ego": 0.01720050633963892, "deviation-heading": 0.27041081849249704, "complete-iteration": 0.20265177144842633, "set_robot_commands": 0.004583354723655572, "deviation-center-line": 0.1246386243322534, "driven_lanedir_consec": 0.6085881804686812, "sim_compute_sim_state": 0.02279869580672959, "sim_compute_performance-ego": 0.004205554218615516}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.5174263274303096, "get_ui_image": 0.039668527142754914, "step_physics": 0.09542172530601764, "survival_time": 1.4500000000000006, "driven_lanedir": 0.3424264172773923, "get_state_dump": 0.014733141866223563, "sim_render-ego": 0.007145881652832031, "get_robot_state": 0.01740405477326492, "get_duckie_state": 2.795252306707974e-06, "in-drivable-lane": 0.3500000000000003, "agent_compute-ego": 0.018332974664096176, "deviation-heading": 0.3106738984320317, "complete-iteration": 0.22911674400855755, "set_robot_commands": 0.005195823209039096, "deviation-center-line": 0.07213333681968147, "driven_lanedir_consec": 0.3424264172773923, "sim_compute_sim_state": 0.02644364587191878, "sim_compute_performance-ego": 0.004606172956269363}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.8354010903771174, "get_ui_image": 0.03475347448278356, "step_physics": 0.07550218546832049, "survival_time": 6.749999999999984, "driven_lanedir": 0.6794864726921884, "get_state_dump": 0.013813629856816044, "sim_render-ego": 0.006058491600884332, "get_robot_state": 0.015586088321827077, "get_duckie_state": 2.463658650716146e-06, "in-drivable-lane": 4.699999999999983, "agent_compute-ego": 0.016587151421440974, "deviation-heading": 0.41125078131929055, "complete-iteration": 0.19436123459427448, "set_robot_commands": 0.004702765853316695, "deviation-center-line": 0.08258355868343131, "driven_lanedir_consec": 0.6794864726921884, "sim_compute_sim_state": 0.023064586851331923, "sim_compute_performance-ego": 0.004142545770715784}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.8894611145876925, "get_ui_image": 0.035947971271746086, "step_physics": 0.08715363704796994, "survival_time": 6.5999999999999845, "driven_lanedir": 0.8904446665936326, "get_state_dump": 0.013786689801649614, "sim_render-ego": 0.0058988675926670885, "get_robot_state": 0.014520180947852858, "get_duckie_state": 2.465464852072976e-06, "in-drivable-lane": 3.899999999999986, "agent_compute-ego": 0.016091738686417088, "deviation-heading": 0.976837469261332, "complete-iteration": 0.2092788056893782, "set_robot_commands": 0.004619546008832527, "deviation-center-line": 0.15583868303837567, "driven_lanedir_consec": 0.8904446665936326, "sim_compute_sim_state": 0.02706627773516106, "sim_compute_performance-ego": 0.004041660915721546}}
set_robot_commands_max0.005195823209039096
set_robot_commands_mean0.004775372448710973
set_robot_commands_median0.004661155931074612
set_robot_commands_min0.004583354723655572
sim_compute_performance-ego_max0.004606172956269363
sim_compute_performance-ego_mean0.004248983465330552
sim_compute_performance-ego_median0.004174049994665649
sim_compute_performance-ego_min0.004041660915721546
sim_compute_sim_state_max0.02706627773516106
sim_compute_sim_state_mean0.024843301566285336
sim_compute_sim_state_median0.02475411636162535
sim_compute_sim_state_min0.02279869580672959
sim_render-ego_max0.007145881652832031
sim_render-ego_mean0.006314045463782266
sim_render-ego_median0.0061057163048149725
sim_render-ego_min0.0058988675926670885
simulation-passed1
step_physics_max0.09542172530601764
step_physics_mean0.08449509262769564
step_physics_median0.08352822986822223
step_physics_min0.07550218546832049
survival_time_max6.749999999999984
survival_time_mean4.437499999999992
survival_time_min1.4500000000000006
No reset possible
4201011349Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LF-sim-validationLFv-simsuccessnomont02-5bd91ed070bc-10:07:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.7672262347507517
survival_time_median12.425000000000042
deviation-center-line_median0.4307658461866393
in-drivable-lane_median3.5500000000000504


other stats
agent_compute-ego_max0.05725560360348102
agent_compute-ego_mean0.04742907712755868
agent_compute-ego_median0.044240718018994735
agent_compute-ego_min0.04397926886876424
complete-iteration_max0.23046512702076705
complete-iteration_mean0.22508566734403573
complete-iteration_median0.22507353162669752
complete-iteration_min0.21973047910198087
deviation-center-line_max0.5346054664514497
deviation-center-line_mean0.4052682315111727
deviation-center-line_min0.22493576721996264
deviation-heading_max2.6540483789397284
deviation-heading_mean1.1272938919790456
deviation-heading_median0.66395296739753
deviation-heading_min0.5272212541813943
driven_any_max3.5790896581213634
driven_any_mean2.9483319324235526
driven_any_median2.9605853140313183
driven_any_min2.2930674435102105
driven_lanedir_consec_max2.084630826177023
driven_lanedir_consec_mean1.696388154214319
driven_lanedir_consec_min1.1664693211787514
driven_lanedir_max2.084630826177023
driven_lanedir_mean1.696388154214319
driven_lanedir_median1.7672262347507517
driven_lanedir_min1.1664693211787514
get_duckie_state_max3.773358560377552e-06
get_duckie_state_mean3.675516043321198e-06
get_duckie_state_median3.66392384571244e-06
get_duckie_state_min3.600857921482362e-06
get_robot_state_max0.016307331950788997
get_robot_state_mean0.015410836653471944
get_robot_state_median0.015487933504965997
get_robot_state_min0.01436014765316678
get_state_dump_max0.01400156119435104
get_state_dump_mean0.013935811561482173
get_state_dump_median0.013946956782965545
get_state_dump_min0.013847771485646566
get_ui_image_max0.03458153962131485
get_ui_image_mean0.03396519678979574
get_ui_image_median0.034074474791044834
get_ui_image_min0.03313029795577846
in-drivable-lane_max8.850000000000044
in-drivable-lane_mean4.812500000000041
in-drivable-lane_min3.3000000000000176
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.2930674435102105, "get_ui_image": 0.03313029795577846, "step_physics": 0.07285704809365813, "survival_time": 9.700000000000005, "driven_lanedir": 1.4637010801847796, "get_state_dump": 0.01400156119435104, "sim_render-ego": 0.0063313786516484526, "get_robot_state": 0.01436014765316678, "get_duckie_state": 3.600857921482362e-06, "in-drivable-lane": 3.3000000000000176, "agent_compute-ego": 0.05725560360348102, "deviation-heading": 0.6422592254249154, "complete-iteration": 0.23046512702076705, "set_robot_commands": 0.0047524319481603875, "deviation-center-line": 0.22493576721996264, "driven_lanedir_consec": 1.4637010801847796, "sim_compute_sim_state": 0.02357427115292893, "sim_compute_performance-ego": 0.004051433396093624}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.954464326702849, "get_ui_image": 0.03392652157814272, "step_physics": 0.07324983035364459, "survival_time": 12.40000000000004, "driven_lanedir": 2.070751389316724, "get_state_dump": 0.01396602584469703, "sim_render-ego": 0.006166751346280498, "get_robot_state": 0.015779641366774035, "get_duckie_state": 3.773358560377552e-06, "in-drivable-lane": 3.5500000000000504, "agent_compute-ego": 0.0440948567082805, "deviation-heading": 0.6856467093701446, "complete-iteration": 0.21973047910198087, "set_robot_commands": 0.005100508851389731, "deviation-center-line": 0.46285553023792175, "driven_lanedir_consec": 2.070751389316724, "sim_compute_sim_state": 0.02319102806429709, "sim_compute_performance-ego": 0.004101407143377488}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 3.5790896581213634, "get_ui_image": 0.03422242800394694, "step_physics": 0.07408369064331055, "survival_time": 14.950000000000076, "driven_lanedir": 1.1664693211787514, "get_state_dump": 0.013847771485646566, "sim_render-ego": 0.006039498647054036, "get_robot_state": 0.015196225643157958, "get_duckie_state": 3.6787986755371097e-06, "in-drivable-lane": 8.850000000000044, "agent_compute-ego": 0.04397926886876424, "deviation-heading": 2.6540483789397284, "complete-iteration": 0.22040624539057413, "set_robot_commands": 0.004941291809082031, "deviation-center-line": 0.3986761621353569, "driven_lanedir_consec": 1.1664693211787514, "sim_compute_sim_state": 0.023863471349080404, "sim_compute_performance-ego": 0.004079896608988444}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.966706301359787, "get_ui_image": 0.03458153962131485, "step_physics": 0.07825508270876474, "survival_time": 12.450000000000042, "driven_lanedir": 2.084630826177023, "get_state_dump": 0.013927887721234057, "sim_render-ego": 0.006154121644046891, "get_robot_state": 0.016307331950788997, "get_duckie_state": 3.64904901588777e-06, "in-drivable-lane": 3.5500000000000504, "agent_compute-ego": 0.04438657932970897, "deviation-heading": 0.5272212541813943, "complete-iteration": 0.2297408178628209, "set_robot_commands": 0.005196428681951928, "deviation-center-line": 0.5346054664514497, "driven_lanedir_consec": 2.084630826177023, "sim_compute_sim_state": 0.026569988832895058, "sim_compute_performance-ego": 0.0042063107930991545}}
set_robot_commands_max0.005196428681951928
set_robot_commands_mean0.004997665322646019
set_robot_commands_median0.005020900330235881
set_robot_commands_min0.0047524319481603875
sim_compute_performance-ego_max0.0042063107930991545
sim_compute_performance-ego_mean0.004109761985389678
sim_compute_performance-ego_median0.004090651876182966
sim_compute_performance-ego_min0.004051433396093624
sim_compute_sim_state_max0.026569988832895058
sim_compute_sim_state_mean0.024299689849800372
sim_compute_sim_state_median0.023718871251004667
sim_compute_sim_state_min0.02319102806429709
sim_render-ego_max0.0063313786516484526
sim_render-ego_mean0.00617293757225747
sim_render-ego_median0.006160436495163695
sim_render-ego_min0.006039498647054036
simulation-passed1
step_physics_max0.07825508270876474
step_physics_mean0.0746114129498445
step_physics_median0.07366676049847756
step_physics_min0.07285704809365813
survival_time_max14.950000000000076
survival_time_mean12.375000000000044
survival_time_min9.700000000000005
No reset possible
4200511349Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:03:58
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 327, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego" aborted:

error in ego |Exception while handling a message on topic "get_commands".
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 331, in loop
             ||     handle_message_node(parsed, receiver0, context0)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 486, in handle_message_node
             ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 52, in on_received_get_commands
             ||     linear, angular = self.compute_action(self.to_predictor)
             ||   File "solution.py", line 47, in compute_action
             ||     (linear, angular) = self.model.predict(observation)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 88, in _method_wrapper
             ||     return method(self, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1268, in predict
             ||     tmp_batch_outputs = predict_function(iterator)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 580, in __call__
             ||     result = self._call(*args, **kwds)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 650, in _call
             ||     return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds)  # pylint: disable=protected-access
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call
             ||     return self._call_flat(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat
             ||     return self._build_call_outputs(self._inference_function.call(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 593, in call
             ||     outputs = execute.execute(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
             ||     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
             || tensorflow.python.framework.errors_impl.UnknownError:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[node FrankNet/conv2d_5/Conv2D (defined at solution.py:47) ]] [Op:__inference_predict_function_790]
             ||
             || Errors may have originated from an input operation.
             || Input Source operations connected to node FrankNet/conv2d_5/Conv2D:
             ||  FrankNet/lambda_1/truediv (defined at /submission/frankModel.py:42)
             ||
             || Function call stack:
             || predict_function
             ||
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4199911347Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:00:42
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4199611346Frank (Chude) Qian 🇨🇦template-tensorflowaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:00:54
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 327, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego" aborted:

error in ego |Exception while handling a message on topic "get_commands".
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1365, in _do_call
             ||     return fn(*args)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1349, in _run_fn
             ||     return self._call_tf_sessionrun(options, feed_dict, fetch_list,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1441, in _call_tf_sessionrun
             ||     return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
             || tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
             ||   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[{{node conv2d/Conv2D}}]]
             ||   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[{{node conv2d/Conv2D}}]]
             || 	 [[Tanh/_47]]
             || 0 successful operations.
             || 0 derived errors ignored.
             ||
             || During handling of the above exception, another exception occurred:
             ||
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 331, in loop
             ||     handle_message_node(parsed, receiver0, context0)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 486, in handle_message_node
             ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 64, in on_received_get_commands
             ||     pwm_left, pwm_right = self.compute_action(self.current_image)
             ||   File "solution.py", line 60, in compute_action
             ||     action = self.model.predict(observation)
             ||   File "/submission/model.py", line 64, in predict
             ||     action = self.tf_session.run(self._computation_graph, feed_dict={
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 957, in run
             ||     result = self._run(None, fetches, feed_dict, options_ptr,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1180, in _run
             ||     results = self._do_run(handle, final_targets, final_fetches,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1358, in _do_run
             ||     return self._do_call(_run_fn, feeds, fetches, targets, options,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1384, in _do_call
             ||     raise type(e)(node_def, op, message)
             || tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
             ||   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[node conv2d/Conv2D (defined at /submission/model.py:28) ]]
             ||   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[node conv2d/Conv2D (defined at /submission/model.py:28) ]]
             || 	 [[Tanh/_47]]
             || 0 successful operations.
             || 0 derived errors ignored.
             ||
             || Errors may have originated from an input operation.
             || Input Source operations connected to node conv2d/Conv2D:
             ||  map_1/TensorArrayV2Stack/TensorListStack (defined at /submission/model.py:91)
             ||
             || Input Source operations connected to node conv2d/Conv2D:
             ||  map_1/TensorArrayV2Stack/TensorListStack (defined at /submission/model.py:91)
             ||
             || Original stack trace for 'conv2d/Conv2D':
             ||   File "solution.py", line 98, in <module>
             ||     main()
             ||   File "solution.py", line 94, in main
             ||     wrap_direct(node=node, protocol=protocol)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/interface.py", line 24, in wrap_direct
             ||     run_loop(node, protocol, args)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 244, in run_loop
             ||     loop(node_name, fi, fo, node, protocol, tin, tout, config=config, fi_desc=fin, fo_desc=fout)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             ||     call_if_fun_exists(node, "init", context=context_data)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 42, in init
             ||     self.model = TfInference(observation_shape=(1,) + expect_shape,
             ||   File "/submission/model.py", line 61, in __init__
             ||     self._initialize(observation_shape, action_shape, graph_location)
             ||   File "/submission/model.py", line 86, in _initialize
             ||     self._create(input_shape, action_shape)
             ||   File "/submission/model.py", line 99, in _create
             ||     self._computation_graph = self.computation_graph()
             ||   File "/submission/model.py", line 70, in computation_graph
             ||     model = one_residual(self._preprocessed_state, seed=self.seed)
             ||   File "/submission/model.py", line 28, in one_residual
             ||     nn = tf.compat.v1.layers.conv2d(x, filters=32, kernel_size=5, strides=2, padding='same',
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
             ||     return func(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/layers/convolutional.py", line 424, in conv2d
             ||     return layer.apply(inputs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
             ||     return func(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 1695, in apply
             ||     return self.__call__(inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/layers/base.py", line 547, in __call__
             ||     outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 778, in __call__
             ||     outputs = call_fn(cast_inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 262, in wrapper
             ||     return converted_call(f, args, kwargs, options=options)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 492, in converted_call
             ||     return _call_unconverted(f, args, kwargs, options)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 346, in _call_unconverted
             ||     return f(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/layers/convolutional.py", line 207, in call
             ||     outputs = self._convolution_op(inputs, self.kernel)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 1106, in __call__
             ||     return self.conv_op(inp, filter)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 638, in __call__
             ||     return self.call(inp, filter)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 231, in __call__
             ||     return self.conv_op(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 2006, in conv2d
             ||     return gen_nn_ops.conv2d(input,  # pylint: disable=redefined-builtin
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 965, in conv2d
             ||     _, _, _op, _outputs = _op_def_library._apply_op_helper(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 742, in _apply_op_helper
             ||     op = g._create_op_internal(op_type_name, inputs, dtypes=None,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 3319, in _create_op_internal
             ||     ret = Operation(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1791, in __init__
             ||     self._traceback = tf_stack.extract_stack()
             ||
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4199411345Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:00:31
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4199011341Frank (Chude) Qian 🇨🇦template-tensorflowaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:01:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 334, in read_until_over
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: External node "ego" closed communication.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4198111343Frank (Chude) Qian 🇨🇦template-pytorchaido5-LF-sim-validationLFv-simabortednomont02-5bd91ed070bc-10:03:58
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1127, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 531, in heartbeat
    raise KeyboardInterrupt(msg)
KeyboardInterrupt: The server told us to abort the job because: The job has been aborted.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4197411340Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:04:36
The container "solut [...]
The container "solution" exited with code 139.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4197211339Bhavya Patwa 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simerrornomont02-5bd91ed070bc-11:00:36
Waited 3610 seconds [...]
Waited 3610 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4197011338Daniel Ng 🇸🇬template-randomaido5-LF-sim-validationLFv-simsuccessnomont02-5bd91ed070bc-10:03:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6440373265804349
survival_time_median4.7749999999999915
deviation-center-line_median0.10361109150784235
in-drivable-lane_median2.5499999999999914


other stats
agent_compute-ego_max0.016830239472565827
agent_compute-ego_mean0.016443185352300795
agent_compute-ego_median0.016389851807078522
agent_compute-ego_min0.016162798322480302
complete-iteration_max0.20908333496613937
complete-iteration_mean0.2021770182653326
complete-iteration_median0.20259444150279152
complete-iteration_min0.1944358550896079
deviation-center-line_max0.15583868303837567
deviation-center-line_mean0.10879855071843549
deviation-center-line_min0.07213333681968147
deviation-heading_max0.976837469261332
deviation-heading_mean0.4922932418762878
deviation-heading_median0.3609623398756611
deviation-heading_min0.27041081849249704
driven_any_max2.8894611145876925
driven_any_mean1.8510551780477624
driven_any_median1.998666635086524
driven_any_min0.5174263274303096
driven_lanedir_consec_max0.8904446665936326
driven_lanedir_consec_mean0.6302364342579737
driven_lanedir_consec_min0.3424264172773923
driven_lanedir_max0.8904446665936326
driven_lanedir_mean0.6302364342579737
driven_lanedir_median0.6440373265804349
driven_lanedir_min0.3424264172773923
get_duckie_state_max2.743834156101033e-06
get_duckie_state_mean2.628698683979644e-06
get_duckie_state_median2.6075928299515335e-06
get_duckie_state_min2.5557749199144766e-06
get_robot_state_max0.016265637786300094
get_robot_state_mean0.015729838904248533
get_robot_state_median0.016112052067693873
get_robot_state_min0.014429613695306293
get_state_dump_max0.01389695830264334
get_state_dump_mean0.013633973632889422
get_state_dump_median0.013749975708598644
get_state_dump_min0.013138984811717066
get_ui_image_max0.03545896565472638
get_ui_image_mean0.03490873211229739
get_ui_image_median0.0350322520573689
get_ui_image_min0.034111458679725384
in-drivable-lane_max4.699999999999983
in-drivable-lane_mean2.5374999999999917
in-drivable-lane_min0.3500000000000003
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.1619321797959306, "get_ui_image": 0.03501620535123146, "step_physics": 0.07750198800685042, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6085881804686812, "get_state_dump": 0.01389695830264334, "sim_render-ego": 0.005880440695811126, "get_robot_state": 0.014429613695306293, "get_duckie_state": 2.743834156101033e-06, "in-drivable-lane": 1.1999999999999966, "agent_compute-ego": 0.016554977934239275, "deviation-heading": 0.27041081849249704, "complete-iteration": 0.1944358550896079, "set_robot_commands": 0.004279407404236875, "deviation-center-line": 0.1246386243322534, "driven_lanedir_consec": 0.6085881804686812, "sim_compute_sim_state": 0.02268564499030679, "sim_compute_performance-ego": 0.004041736408815545}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.5174263274303096, "get_ui_image": 0.034111458679725384, "step_physics": 0.08032898245186641, "survival_time": 1.4500000000000006, "driven_lanedir": 0.3424264172773923, "get_state_dump": 0.013138984811717066, "sim_render-ego": 0.0060516801373711945, "get_robot_state": 0.015979676411069673, "get_duckie_state": 2.6226043701171875e-06, "in-drivable-lane": 0.3500000000000003, "agent_compute-ego": 0.016162798322480302, "deviation-heading": 0.3106738984320317, "complete-iteration": 0.2037599415614687, "set_robot_commands": 0.010406872321819437, "deviation-center-line": 0.07213333681968147, "driven_lanedir_consec": 0.3424264172773923, "sim_compute_sim_state": 0.023161205752142543, "sim_compute_performance-ego": 0.00426557968402731}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.8354010903771174, "get_ui_image": 0.03545896565472638, "step_physics": 0.08005266189575196, "survival_time": 6.749999999999984, "driven_lanedir": 0.6794864726921884, "get_state_dump": 0.013739536426685475, "sim_render-ego": 0.006266673405965169, "get_robot_state": 0.016265637786300094, "get_duckie_state": 2.5925812897858795e-06, "in-drivable-lane": 4.699999999999983, "agent_compute-ego": 0.016830239472565827, "deviation-heading": 0.41125078131929055, "complete-iteration": 0.2014289414441144, "set_robot_commands": 0.0048986805809868705, "deviation-center-line": 0.08258355868343131, "driven_lanedir_consec": 0.6794864726921884, "sim_compute_sim_state": 0.02361281359637225, "sim_compute_performance-ego": 0.004149989728574399}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.8894611145876925, "get_ui_image": 0.03504829876350634, "step_physics": 0.08522836728529497, "survival_time": 6.5999999999999845, "driven_lanedir": 0.8904446665936326, "get_state_dump": 0.013760414990511808, "sim_render-ego": 0.006140593326453007, "get_robot_state": 0.01624442772431807, "get_duckie_state": 2.5557749199144766e-06, "in-drivable-lane": 3.899999999999986, "agent_compute-ego": 0.01622472567991777, "deviation-heading": 0.976837469261332, "complete-iteration": 0.20908333496613937, "set_robot_commands": 0.004922306898868445, "deviation-center-line": 0.15583868303837567, "driven_lanedir_consec": 0.8904446665936326, "sim_compute_sim_state": 0.027182232249866833, "sim_compute_performance-ego": 0.004174563017758456}}
set_robot_commands_max0.010406872321819437
set_robot_commands_mean0.006126816801477907
set_robot_commands_median0.0049104937399276575
set_robot_commands_min0.004279407404236875
sim_compute_performance-ego_max0.00426557968402731
sim_compute_performance-ego_mean0.004157967209793928
sim_compute_performance-ego_median0.004162276373166428
sim_compute_performance-ego_min0.004041736408815545
sim_compute_sim_state_max0.027182232249866833
sim_compute_sim_state_mean0.024160474147172106
sim_compute_sim_state_median0.023387009674257397
sim_compute_sim_state_min0.02268564499030679
sim_render-ego_max0.006266673405965169
sim_render-ego_mean0.006084846891400124
sim_render-ego_median0.006096136731912101
sim_render-ego_min0.005880440695811126
simulation-passed1
step_physics_max0.08522836728529497
step_physics_mean0.08077799990994095
step_physics_median0.08019082217380918
step_physics_min0.07750198800685042
survival_time_max6.749999999999984
survival_time_mean4.437499999999992
survival_time_min1.4500000000000006
No reset possible