Duckietown Challenges Home Challenges Submissions

Evaluator 2270

ID2270
evaluatorreg02-2604b9bb4859-1
ownerI don't have one 😀
machinearchimede
processreg02-2604b9bb4859-1
versiond-c:6.1.23;d-c-r:6.0.59
first heard
last heard
statusinactive
# evaluating
# success3 40929
# timeout
# failed4 40920
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
4096610774Étienne Boucher 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg02-2604b9bb4859-10:07:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.6602558380688253
survival_time_median8.474999999999985
deviation-center-line_median0.22400568318785177
in-drivable-lane_median2.2499999999999996


other stats
agent_compute-ego_max0.026478673617045086
agent_compute-ego_mean0.025819200384261425
agent_compute-ego_median0.025789460710976292
agent_compute-ego_min0.025219206498048017
complete-iteration_max0.2998566232489411
complete-iteration_mean0.2873283419860252
complete-iteration_median0.2850713707185259
complete-iteration_min0.27931400325810796
deviation-center-line_max0.3153962554321791
deviation-center-line_mean0.2264158596316393
deviation-center-line_min0.14225581671867443
deviation-heading_max1.9576243208931403
deviation-heading_mean1.0418054624516788
deviation-heading_median0.8234594962880496
deviation-heading_min0.5626785363374757
driven_any_max5.105213879112654
driven_any_mean3.131310076730317
driven_any_median2.842847975866049
driven_any_min1.734330476076516
driven_lanedir_consec_max2.034658401143618
driven_lanedir_consec_mean1.5695505076838652
driven_lanedir_consec_min0.923031953454192
driven_lanedir_max2.034658401143618
driven_lanedir_mean1.5695505076838652
driven_lanedir_median1.6602558380688253
driven_lanedir_min0.923031953454192
get_duckie_state_max4.029273986816406e-06
get_duckie_state_mean3.642296865350361e-06
get_duckie_state_median3.540627310209185e-06
get_duckie_state_min3.4586588541666665e-06
get_robot_state_max0.024074457100862583
get_robot_state_mean0.023126757123455043
get_robot_state_median0.023002853356417836
get_robot_state_min0.022426864680121925
get_state_dump_max0.019915137571447035
get_state_dump_mean0.019139140214998924
get_state_dump_median0.01902886648156478
get_state_dump_min0.018583690325419108
get_ui_image_max0.05108675448852178
get_ui_image_mean0.05008560187049649
get_ui_image_median0.05011008868030473
get_ui_image_min0.04903547563285471
in-drivable-lane_max11.150000000000055
in-drivable-lane_mean4.200000000000013
in-drivable-lane_min1.149999999999996
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.734330476076516, "get_ui_image": 0.04903547563285471, "step_physics": 0.108919163730657, "survival_time": 5.349999999999989, "driven_lanedir": 1.303843610644295, "get_state_dump": 0.019034822410512194, "sim_render-ego": 0.009584932683784271, "get_robot_state": 0.023263160313401268, "get_duckie_state": 3.504975933894933e-06, "in-drivable-lane": 1.149999999999996, "agent_compute-ego": 0.025219206498048017, "deviation-heading": 0.5626785363374757, "complete-iteration": 0.27931400325810796, "set_robot_commands": 0.007134834182596653, "deviation-center-line": 0.14225581671867443, "driven_lanedir_consec": 1.303843610644295, "sim_compute_sim_state": 0.030557997872896283, "sim_compute_performance-ego": 0.006364087078058831}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.851578375784768, "get_ui_image": 0.04924344876233269, "step_physics": 0.11103253645055434, "survival_time": 8.499999999999986, "driven_lanedir": 2.034658401143618, "get_state_dump": 0.019915137571447035, "sim_render-ego": 0.009029740445754109, "get_robot_state": 0.022426864680121925, "get_duckie_state": 4.029273986816406e-06, "in-drivable-lane": 2.25, "agent_compute-ego": 0.02620403345893411, "deviation-heading": 0.9170916300386964, "complete-iteration": 0.2811646265142104, "set_robot_commands": 0.006869857451495002, "deviation-center-line": 0.3153962554321791, "driven_lanedir_consec": 2.034658401143618, "sim_compute_sim_state": 0.030130759407492245, "sim_compute_performance-ego": 0.006098991281845991}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 5.105213879112654, "get_ui_image": 0.050976728598276774, "step_physics": 0.11533054033915202, "survival_time": 14.950000000000076, "driven_lanedir": 0.923031953454192, "get_state_dump": 0.018583690325419108, "sim_render-ego": 0.009431763490041096, "get_robot_state": 0.022742546399434408, "get_duckie_state": 3.4586588541666665e-06, "in-drivable-lane": 11.150000000000055, "agent_compute-ego": 0.026478673617045086, "deviation-heading": 1.9576243208931403, "complete-iteration": 0.28897811492284137, "set_robot_commands": 0.006935097376505534, "deviation-center-line": 0.22214364257222513, "driven_lanedir_consec": 0.923031953454192, "sim_compute_sim_state": 0.03233898560206095, "sim_compute_performance-ego": 0.005963101387023926}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.8341175759473303, "get_ui_image": 0.05108675448852178, "step_physics": 0.12218186841208556, "survival_time": 8.449999999999985, "driven_lanedir": 2.016668065493356, "get_state_dump": 0.01902291055261736, "sim_render-ego": 0.009532687226696129, "get_robot_state": 0.024074457100862583, "get_duckie_state": 3.5762786865234375e-06, "in-drivable-lane": 2.249999999999999, "agent_compute-ego": 0.025374887963018473, "deviation-heading": 0.7298273625374028, "complete-iteration": 0.2998566232489411, "set_robot_commands": 0.007290498744806594, "deviation-center-line": 0.2258677238034784, "driven_lanedir_consec": 2.016668065493356, "sim_compute_sim_state": 0.03483947635402341, "sim_compute_performance-ego": 0.00624772923937916}}
set_robot_commands_max0.007290498744806594
set_robot_commands_mean0.007057571938850945
set_robot_commands_median0.007034965779551093
set_robot_commands_min0.006869857451495002
sim_compute_performance-ego_max0.006364087078058831
sim_compute_performance-ego_mean0.006168477246576977
sim_compute_performance-ego_median0.006173360260612576
sim_compute_performance-ego_min0.005963101387023926
sim_compute_sim_state_max0.03483947635402341
sim_compute_sim_state_mean0.03196680480911822
sim_compute_sim_state_median0.031448491737478615
sim_compute_sim_state_min0.030130759407492245
sim_render-ego_max0.009584932683784271
sim_render-ego_mean0.0093947809615689
sim_render-ego_median0.009482225358368611
sim_render-ego_min0.009029740445754109
simulation-passed1
step_physics_max0.12218186841208556
step_physics_mean0.11436602723311225
step_physics_median0.11318153839485318
step_physics_min0.108919163730657
survival_time_max14.950000000000076
survival_time_mean9.31250000000001
survival_time_min5.349999999999989
No reset possible
4095310788Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg02-2604b9bb4859-10:11:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.243528030383138
survival_time_median14.950000000000076
deviation-center-line_median0.4969369017827135
in-drivable-lane_median1.1500000000000163


other stats
agent_compute-ego_max0.06098001797993978
agent_compute-ego_mean0.05180733243624369
agent_compute-ego_median0.05915475726127624
agent_compute-ego_min0.0279397972424825
complete-iteration_max0.3379490860303243
complete-iteration_mean0.3180100429058075
complete-iteration_median0.32217575271924337
complete-iteration_min0.2897395801544189
deviation-center-line_max0.6190706183650999
deviation-center-line_mean0.4783141880356512
deviation-center-line_min0.30031233021207765
deviation-heading_max2.5517589147938056
deviation-heading_mean1.953276957366054
deviation-heading_median1.970941700983652
deviation-heading_min1.3194655127031063
driven_any_max2.5354551771300935
driven_any_mean2.4787980717207283
driven_any_median2.4694633220235973
driven_any_min2.440810465705625
driven_lanedir_consec_max2.4596628510129133
driven_lanedir_consec_mean1.931249872928235
driven_lanedir_consec_min0.7782805799337505
driven_lanedir_max2.4596628510129133
driven_lanedir_mean1.931249872928235
driven_lanedir_median2.243528030383138
driven_lanedir_min0.7782805799337505
get_duckie_state_max4.463990529378255e-06
get_duckie_state_mean4.156430562337239e-06
get_duckie_state_median4.12146250406901e-06
get_duckie_state_min3.9188067118326825e-06
get_robot_state_max0.025100844701131185
get_robot_state_mean0.024400306741396587
get_robot_state_median0.02430140733718872
get_robot_state_min0.023897567590077717
get_state_dump_max0.020414615472157796
get_state_dump_mean0.019892375469207763
get_state_dump_median0.01994049668312073
get_state_dump_min0.019273893038431803
get_ui_image_max0.05398976882298787
get_ui_image_mean0.0507543267806371
get_ui_image_median0.05003686865170797
get_ui_image_min0.04895380099614461
in-drivable-lane_max9.050000000000065
in-drivable-lane_mean2.837500000000024
in-drivable-lane_min0.0
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.5354551771300935, "get_ui_image": 0.05049943765004476, "step_physics": 0.1109316857655843, "survival_time": 14.950000000000076, "driven_lanedir": 2.3733586051408357, "get_state_dump": 0.01982628583908081, "sim_render-ego": 0.009972745577494304, "get_robot_state": 0.02459100325902303, "get_duckie_state": 4.262129465738932e-06, "in-drivable-lane": 0.6000000000000085, "agent_compute-ego": 0.0279397972424825, "deviation-heading": 1.9814843110893436, "complete-iteration": 0.2897395801544189, "set_robot_commands": 0.007255944410959879, "deviation-center-line": 0.4825873955294142, "driven_lanedir_consec": 2.3733586051408357, "sim_compute_sim_state": 0.03188876231511434, "sim_compute_performance-ego": 0.006609377066294352}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.440810465705625, "get_ui_image": 0.04957429965337117, "step_physics": 0.1116870085398356, "survival_time": 14.950000000000076, "driven_lanedir": 2.1136974556254406, "get_state_dump": 0.020414615472157796, "sim_render-ego": 0.009748864968617757, "get_robot_state": 0.025100844701131185, "get_duckie_state": 4.463990529378255e-06, "in-drivable-lane": 1.7000000000000242, "agent_compute-ego": 0.05919616858164469, "deviation-heading": 1.96039909087796, "complete-iteration": 0.32149790445963544, "set_robot_commands": 0.008020605246225993, "deviation-center-line": 0.5112864080360128, "driven_lanedir_consec": 2.1136974556254406, "sim_compute_sim_state": 0.031102526187896728, "sim_compute_performance-ego": 0.006427373091379801}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.457793067768224, "get_ui_image": 0.04895380099614461, "step_physics": 0.11245599190394084, "survival_time": 14.950000000000076, "driven_lanedir": 0.7782805799337505, "get_state_dump": 0.020054707527160643, "sim_render-ego": 0.010084809462229413, "get_robot_state": 0.02401181141535441, "get_duckie_state": 3.9188067118326825e-06, "in-drivable-lane": 9.050000000000065, "agent_compute-ego": 0.06098001797993978, "deviation-heading": 2.5517589147938056, "complete-iteration": 0.3228536009788513, "set_robot_commands": 0.007642800013224284, "deviation-center-line": 0.30031233021207765, "driven_lanedir_consec": 0.7782805799337505, "sim_compute_sim_state": 0.03175835212071736, "sim_compute_performance-ego": 0.006676882902781169}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.4811335762789706, "get_ui_image": 0.05398976882298787, "step_physics": 0.12098071734110516, "survival_time": 14.950000000000076, "driven_lanedir": 2.4596628510129133, "get_state_dump": 0.019273893038431803, "sim_render-ego": 0.010359220504760742, "get_robot_state": 0.023897567590077717, "get_duckie_state": 3.980795542399088e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.059113345940907794, "deviation-heading": 1.3194655127031063, "complete-iteration": 0.3379490860303243, "set_robot_commands": 0.007641720771789551, "deviation-center-line": 0.6190706183650999, "driven_lanedir_consec": 2.4596628510129133, "sim_compute_sim_state": 0.03587027390797933, "sim_compute_performance-ego": 0.006594287554423014}}
set_robot_commands_max0.008020605246225993
set_robot_commands_mean0.007640267610549927
set_robot_commands_median0.007642260392506917
set_robot_commands_min0.007255944410959879
sim_compute_performance-ego_max0.006676882902781169
sim_compute_performance-ego_mean0.006576980153719584
sim_compute_performance-ego_median0.006601832310358683
sim_compute_performance-ego_min0.006427373091379801
sim_compute_sim_state_max0.03587027390797933
sim_compute_sim_state_mean0.03265497863292694
sim_compute_sim_state_median0.031823557217915854
sim_compute_sim_state_min0.031102526187896728
sim_render-ego_max0.010359220504760742
sim_render-ego_mean0.010041410128275554
sim_render-ego_median0.010028777519861858
sim_render-ego_min0.009748864968617757
simulation-passed1
step_physics_max0.12098071734110516
step_physics_mean0.11401385088761648
step_physics_median0.11207150022188822
step_physics_min0.1109316857655843
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
4094910793Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailednoreg02-2604b9bb4859-10:03:23
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 5 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4094310797Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailednoreg02-2604b9bb4859-10:03:57
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 99 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4094210798Frank (Chude) Qian 🇨🇦template-tensorflowaido5-LF-sim-validationLFv-simfailednoreg02-2604b9bb4859-10:01:38
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 327, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego" aborted:

error in ego |Exception while handling a message on topic "get_commands".
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1365, in _do_call
             ||     return fn(*args)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1349, in _run_fn
             ||     return self._call_tf_sessionrun(options, feed_dict, fetch_list,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1441, in _call_tf_sessionrun
             ||     return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
             || tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
             ||   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[{{node conv2d/Conv2D}}]]
             ||   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[{{node conv2d/Conv2D}}]]
             || 	 [[Tanh/_47]]
             || 0 successful operations.
             || 0 derived errors ignored.
             ||
             || During handling of the above exception, another exception occurred:
             ||
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 331, in loop
             ||     handle_message_node(parsed, receiver0, context0)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 486, in handle_message_node
             ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 58, in on_received_get_commands
             ||     pwm_left, pwm_right = self.compute_action(self.current_image)
             ||   File "solution.py", line 54, in compute_action
             ||     action = self.model.predict(observation)
             ||   File "/submission/model.py", line 64, in predict
             ||     action = self.tf_session.run(self._computation_graph, feed_dict={
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 957, in run
             ||     result = self._run(None, fetches, feed_dict, options_ptr,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1180, in _run
             ||     results = self._do_run(handle, final_targets, final_fetches,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1358, in _do_run
             ||     return self._do_call(_run_fn, feeds, fetches, targets, options,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1384, in _do_call
             ||     raise type(e)(node_def, op, message)
             || tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
             ||   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[node conv2d/Conv2D (defined at /submission/model.py:28) ]]
             ||   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[node conv2d/Conv2D (defined at /submission/model.py:28) ]]
             || 	 [[Tanh/_47]]
             || 0 successful operations.
             || 0 derived errors ignored.
             ||
             || Errors may have originated from an input operation.
             || Input Source operations connected to node conv2d/Conv2D:
             ||  map_1/TensorArrayV2Stack/TensorListStack (defined at /submission/model.py:91)
             ||
             || Input Source operations connected to node conv2d/Conv2D:
             ||  map_1/TensorArrayV2Stack/TensorListStack (defined at /submission/model.py:91)
             ||
             || Original stack trace for 'conv2d/Conv2D':
             ||   File "solution.py", line 91, in <module>
             ||     main()
             ||   File "solution.py", line 87, in main
             ||     wrap_direct(node=node, protocol=protocol)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/interface.py", line 24, in wrap_direct
             ||     run_loop(node, protocol, args)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 244, in run_loop
             ||     loop(node_name, fi, fo, node, protocol, tin, tout, config=config, fi_desc=fin, fo_desc=fout)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             ||     call_if_fun_exists(node, "init", context=context_data)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 36, in init
             ||     self.model = TfInference(observation_shape=(1,) + expect_shape,
             ||   File "/submission/model.py", line 61, in __init__
             ||     self._initialize(observation_shape, action_shape, graph_location)
             ||   File "/submission/model.py", line 86, in _initialize
             ||     self._create(input_shape, action_shape)
             ||   File "/submission/model.py", line 99, in _create
             ||     self._computation_graph = self.computation_graph()
             ||   File "/submission/model.py", line 70, in computation_graph
             ||     model = one_residual(self._preprocessed_state, seed=self.seed)
             ||   File "/submission/model.py", line 28, in one_residual
             ||     nn = tf.compat.v1.layers.conv2d(x, filters=32, kernel_size=5, strides=2, padding='same',
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
             ||     return func(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/layers/convolutional.py", line 424, in conv2d
             ||     return layer.apply(inputs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
             ||     return func(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 1695, in apply
             ||     return self.__call__(inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/layers/base.py", line 547, in __call__
             ||     outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 778, in __call__
             ||     outputs = call_fn(cast_inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 262, in wrapper
             ||     return converted_call(f, args, kwargs, options=options)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 492, in converted_call
             ||     return _call_unconverted(f, args, kwargs, options)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 346, in _call_unconverted
             ||     return f(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/layers/convolutional.py", line 207, in call
             ||     outputs = self._convolution_op(inputs, self.kernel)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 1106, in __call__
             ||     return self.conv_op(inp, filter)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 638, in __call__
             ||     return self.call(inp, filter)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 231, in __call__
             ||     return self.conv_op(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 2006, in conv2d
             ||     return gen_nn_ops.conv2d(input,  # pylint: disable=redefined-builtin
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 965, in conv2d
             ||     _, _, _op, _outputs = _op_def_library._apply_op_helper(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 742, in _apply_op_helper
             ||     op = g._create_op_internal(op_type_name, inputs, dtypes=None,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 3319, in _create_op_internal
             ||     ret = Operation(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1791, in __init__
             ||     self._traceback = tf_stack.extract_stack()
             ||
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4092910807Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg02-2604b9bb4859-10:11:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.1271045587364013
survival_time_median14.950000000000076
deviation-center-line_median0.4234531829689497
in-drivable-lane_median2.80000000000004


other stats
agent_compute-ego_max0.05803770860036214
agent_compute-ego_mean0.034124048153559364
agent_compute-ego_median0.026488106250762943
agent_compute-ego_min0.025482271512349445
complete-iteration_max0.33714015007019044
complete-iteration_mean0.29932937065760296
complete-iteration_median0.2904088926315308
complete-iteration_min0.27935954729715984
deviation-center-line_max0.596249354776765
deviation-center-line_mean0.46080326697806706
deviation-center-line_min0.40005734719760383
deviation-heading_max1.5422306552366258
deviation-heading_mean1.1749306605419798
deviation-heading_median1.1402273203312654
deviation-heading_min0.8770373462687627
driven_any_max2.6547319493318846
driven_any_mean2.6525577420858757
driven_any_median2.654716855826919
driven_any_min2.6460653073577807
driven_lanedir_consec_max2.618375445199819
driven_lanedir_consec_mean2.1732805606367247
driven_lanedir_consec_min1.8205376798742776
driven_lanedir_max2.618375445199819
driven_lanedir_mean2.1732805606367247
driven_lanedir_median2.1271045587364013
driven_lanedir_min1.8205376798742776
get_duckie_state_max5.972385406494141e-06
get_duckie_state_mean5.6805213292439775e-06
get_duckie_state_median5.6560834248860676e-06
get_duckie_state_min5.437533060709636e-06
get_robot_state_max0.027064411640167235
get_robot_state_mean0.02505258599917094
get_robot_state_median0.024563397566477456
get_robot_state_min0.02401913722356161
get_state_dump_max0.021700788338979086
get_state_dump_mean0.02027811845143636
get_state_dump_median0.02014227906862895
get_state_dump_min0.019127127329508463
get_ui_image_max0.051195770899454755
get_ui_image_mean0.04992654303709666
get_ui_image_median0.04979930281639099
get_ui_image_min0.0489117956161499
in-drivable-lane_max4.450000000000063
in-drivable-lane_mean2.5125000000000357
in-drivable-lane_min0.0
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.6460653073577807, "get_ui_image": 0.0489117956161499, "step_physics": 0.10714075485865276, "survival_time": 14.950000000000076, "driven_lanedir": 2.618375445199819, "get_state_dump": 0.019127127329508463, "sim_render-ego": 0.009424367745717366, "get_robot_state": 0.02401913722356161, "get_duckie_state": 5.437533060709636e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.025482271512349445, "deviation-heading": 1.5422306552366258, "complete-iteration": 0.27935954729715984, "set_robot_commands": 0.007555611928304036, "deviation-center-line": 0.596249354776765, "driven_lanedir_consec": 2.618375445199819, "sim_compute_sim_state": 0.03114263852437337, "sim_compute_performance-ego": 0.006338185469309489}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.6547319493318846, "get_ui_image": 0.04981066385904948, "step_physics": 0.1148270042737325, "survival_time": 14.950000000000076, "driven_lanedir": 2.0398810292633343, "get_state_dump": 0.019703086217244467, "sim_render-ego": 0.009915189743041992, "get_robot_state": 0.024516031742095948, "get_duckie_state": 5.972385406494141e-06, "in-drivable-lane": 3.250000000000046, "agent_compute-ego": 0.02634411970774333, "deviation-heading": 1.0720541268996464, "complete-iteration": 0.2907706864674886, "set_robot_commands": 0.007434151967366537, "deviation-center-line": 0.41715226703954406, "driven_lanedir_consec": 2.0398810292633343, "sim_compute_sim_state": 0.03133636474609375, "sim_compute_performance-ego": 0.0066615859667460125}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.6547049630329425, "get_ui_image": 0.049787941773732504, "step_physics": 0.11312195380528768, "survival_time": 14.950000000000076, "driven_lanedir": 1.8205376798742776, "get_state_dump": 0.020581471920013427, "sim_render-ego": 0.009808731079101563, "get_robot_state": 0.024610763390858967, "get_duckie_state": 5.753040313720703e-06, "in-drivable-lane": 4.450000000000063, "agent_compute-ego": 0.026632092793782553, "deviation-heading": 1.2084005137628846, "complete-iteration": 0.29004709879557294, "set_robot_commands": 0.007682238419850667, "deviation-center-line": 0.4297540988983553, "driven_lanedir_consec": 1.8205376798742776, "sim_compute_sim_state": 0.03102808157602946, "sim_compute_performance-ego": 0.006570805708567301}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.654728748620896, "get_ui_image": 0.051195770899454755, "step_physics": 0.1130471126238505, "survival_time": 14.950000000000076, "driven_lanedir": 2.2143280882094687, "get_state_dump": 0.021700788338979086, "sim_render-ego": 0.011433331966400149, "get_robot_state": 0.027064411640167235, "get_duckie_state": 5.559126536051432e-06, "in-drivable-lane": 2.3500000000000334, "agent_compute-ego": 0.05803770860036214, "deviation-heading": 0.8770373462687627, "complete-iteration": 0.33714015007019044, "set_robot_commands": 0.008569160302480061, "deviation-center-line": 0.40005734719760383, "driven_lanedir_consec": 2.2143280882094687, "sim_compute_sim_state": 0.03731471459070841, "sim_compute_performance-ego": 0.008540722529093424}}
set_robot_commands_max0.008569160302480061
set_robot_commands_mean0.007810290654500325
set_robot_commands_median0.007618925174077352
set_robot_commands_min0.007434151967366537
sim_compute_performance-ego_max0.008540722529093424
sim_compute_performance-ego_mean0.007027824918429057
sim_compute_performance-ego_median0.006616195837656657
sim_compute_performance-ego_min0.006338185469309489
sim_compute_sim_state_max0.03731471459070841
sim_compute_sim_state_mean0.03270544985930125
sim_compute_sim_state_median0.03123950163523356
sim_compute_sim_state_min0.03102808157602946
sim_render-ego_max0.011433331966400149
sim_render-ego_mean0.010145405133565268
sim_render-ego_median0.009861960411071777
sim_render-ego_min0.009424367745717366
simulation-passed1
step_physics_max0.1148270042737325
step_physics_mean0.11203420639038086
step_physics_median0.1130845332145691
step_physics_min0.10714075485865276
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
4092010812Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailednoreg02-2604b9bb4859-10:03:41
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 49 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible