Duckietown Challenges Home Challenges Submissions

Evaluator 2278

ID2278
evaluatormont01-62097cce5404-1
ownerI don't have one 😀
machineaido-ms-7b09
processmont01-62097cce5404-1
versiond-c:6.1.23;d-c-r:6.0.59
first heard
last heard
statusinactive
# evaluating
# success2 41971
# timeout1 42016
# failed5 41975
# error2 41986
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
4201611352Charlie Gauthier 🇨🇦Baseline solution using reinforcement learningaido5-LF-sim-validationLFv-simtimeoutnomont01-62097cce5404-1----No reset possible
4201211350Martin Cote 🇨🇦template-randomaido5-LF-sim-validationLFv-simsuccessnomont01-62097cce5404-10:04:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6440373265804349
survival_time_median4.7749999999999915
deviation-center-line_median0.10361109150784235
in-drivable-lane_median2.5499999999999914


other stats
agent_compute-ego_max0.017877726719297212
agent_compute-ego_mean0.017080904657928036
agent_compute-ego_median0.01707431910850683
agent_compute-ego_min0.01629725369540128
complete-iteration_max0.2399575874723237
complete-iteration_mean0.2131698011164277
complete-iteration_median0.2071800924412826
complete-iteration_min0.19836143211082177
deviation-center-line_max0.15583868303837567
deviation-center-line_mean0.10879855071843549
deviation-center-line_min0.07213333681968147
deviation-heading_max0.976837469261332
deviation-heading_mean0.4922932418762878
deviation-heading_median0.3609623398756611
deviation-heading_min0.27041081849249704
driven_any_max2.8894611145876925
driven_any_mean1.8510551780477624
driven_any_median1.998666635086524
driven_any_min0.5174263274303096
driven_lanedir_consec_max0.8904446665936326
driven_lanedir_consec_mean0.6302364342579737
driven_lanedir_consec_min0.3424264172773923
driven_lanedir_max0.8904446665936326
driven_lanedir_mean0.6302364342579737
driven_lanedir_median0.6440373265804349
driven_lanedir_min0.3424264172773923
get_duckie_state_max5.401414016197468e-06
get_duckie_state_mean4.381320272192772e-06
get_duckie_state_median4.140456395083106e-06
get_duckie_state_min3.842954282407407e-06
get_robot_state_max0.01809477806091309
get_robot_state_mean0.016523395120121333
get_robot_state_median0.016054741376231252
get_robot_state_min0.015889319667109738
get_state_dump_max0.015606740425372946
get_state_dump_mean0.014555436737897234
get_state_dump_median0.01431059260275282
get_state_dump_min0.01399382132071036
get_ui_image_max0.04331287022294669
get_ui_image_mean0.03732821904269959
get_ui_image_median0.03537301842102223
get_ui_image_min0.03525396910580722
in-drivable-lane_max4.699999999999983
in-drivable-lane_mean2.5374999999999917
in-drivable-lane_min0.3500000000000003
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.1619321797959306, "get_ui_image": 0.03542330709554381, "step_physics": 0.08108266733460508, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6085881804686812, "get_state_dump": 0.014384075746697894, "sim_render-ego": 0.006123942843938278, "get_robot_state": 0.016158257500600008, "get_duckie_state": 4.065238823324947e-06, "in-drivable-lane": 1.1999999999999966, "agent_compute-ego": 0.017299777370388226, "deviation-heading": 0.27041081849249704, "complete-iteration": 0.20345300739094363, "set_robot_commands": 0.004809755389973269, "deviation-center-line": 0.1246386243322534, "driven_lanedir_consec": 0.6085881804686812, "sim_compute_sim_state": 0.023611185914379055, "sim_compute_performance-ego": 0.004403352737426758}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.5174263274303096, "get_ui_image": 0.04331287022294669, "step_physics": 0.10090232717579808, "survival_time": 1.4500000000000006, "driven_lanedir": 0.3424264172773923, "get_state_dump": 0.015606740425372946, "sim_render-ego": 0.007151044648269127, "get_robot_state": 0.01809477806091309, "get_duckie_state": 5.401414016197468e-06, "in-drivable-lane": 0.3500000000000003, "agent_compute-ego": 0.017877726719297212, "deviation-heading": 0.3106738984320317, "complete-iteration": 0.2399575874723237, "set_robot_commands": 0.004939021735355772, "deviation-center-line": 0.07213333681968147, "driven_lanedir_consec": 0.3424264172773923, "sim_compute_sim_state": 0.027080281027432145, "sim_compute_performance-ego": 0.004822961215315194}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.8354010903771174, "get_ui_image": 0.03532272974650065, "step_physics": 0.07792552135608814, "survival_time": 6.749999999999984, "driven_lanedir": 0.6794864726921884, "get_state_dump": 0.01399382132071036, "sim_render-ego": 0.006239537839536314, "get_robot_state": 0.015889319667109738, "get_duckie_state": 3.842954282407407e-06, "in-drivable-lane": 4.699999999999983, "agent_compute-ego": 0.016848860846625433, "deviation-heading": 0.41125078131929055, "complete-iteration": 0.19836143211082177, "set_robot_commands": 0.004902076721191406, "deviation-center-line": 0.08258355868343131, "driven_lanedir_consec": 0.6794864726921884, "sim_compute_sim_state": 0.022852874685216833, "sim_compute_performance-ego": 0.004235101629186559}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.8894611145876925, "get_ui_image": 0.03525396910580722, "step_physics": 0.08656950611056703, "survival_time": 6.5999999999999845, "driven_lanedir": 0.8904446665936326, "get_state_dump": 0.014237109458807745, "sim_render-ego": 0.0062037128390687885, "get_robot_state": 0.015951225251862496, "get_duckie_state": 4.215673966841265e-06, "in-drivable-lane": 3.899999999999986, "agent_compute-ego": 0.01629725369540128, "deviation-heading": 0.976837469261332, "complete-iteration": 0.21090717749162155, "set_robot_commands": 0.004892508188883464, "deviation-center-line": 0.15583868303837567, "driven_lanedir_consec": 0.8904446665936326, "sim_compute_sim_state": 0.027302828702059655, "sim_compute_performance-ego": 0.004043210636485706}}
set_robot_commands_max0.004939021735355772
set_robot_commands_mean0.004885840508850978
set_robot_commands_median0.004897292455037435
set_robot_commands_min0.004809755389973269
sim_compute_performance-ego_max0.004822961215315194
sim_compute_performance-ego_mean0.004376156554603554
sim_compute_performance-ego_median0.004319227183306658
sim_compute_performance-ego_min0.004043210636485706
sim_compute_sim_state_max0.027302828702059655
sim_compute_sim_state_mean0.025211792582271925
sim_compute_sim_state_median0.0253457334709056
sim_compute_sim_state_min0.022852874685216833
sim_render-ego_max0.007151044648269127
sim_render-ego_mean0.006429559542703127
sim_render-ego_median0.006221625339302551
sim_render-ego_min0.006123942843938278
simulation-passed1
step_physics_max0.10090232717579808
step_physics_mean0.08662000549426459
step_physics_median0.08382608672258605
step_physics_min0.07792552135608814
survival_time_max6.749999999999984
survival_time_mean4.437499999999992
survival_time_min1.4500000000000006
No reset possible
4200811349Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LF-sim-validationLFv-simfailednomont01-62097cce5404-10:00:50
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 327, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego" aborted:

error in ego |Exception while handling a message on topic "get_commands".
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 331, in loop
             ||     handle_message_node(parsed, receiver0, context0)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 486, in handle_message_node
             ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 52, in on_received_get_commands
             ||     linear, angular = self.compute_action(self.to_predictor)
             ||   File "solution.py", line 47, in compute_action
             ||     (linear, angular) = self.model.predict(observation)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 88, in _method_wrapper
             ||     return method(self, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py", line 1268, in predict
             ||     tmp_batch_outputs = predict_function(iterator)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 580, in __call__
             ||     result = self._call(*args, **kwds)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 650, in _call
             ||     return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds)  # pylint: disable=protected-access
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1661, in _filtered_call
             ||     return self._call_flat(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 1745, in _call_flat
             ||     return self._build_call_outputs(self._inference_function.call(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 593, in call
             ||     outputs = execute.execute(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
             ||     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
             || tensorflow.python.framework.errors_impl.UnknownError:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[node FrankNet/conv2d_5/Conv2D (defined at solution.py:47) ]] [Op:__inference_predict_function_790]
             ||
             || Errors may have originated from an input operation.
             || Input Source operations connected to node FrankNet/conv2d_5/Conv2D:
             ||  FrankNet/lambda_1/truediv (defined at /submission/frankModel.py:42)
             ||
             || Function call stack:
             || predict_function
             ||
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4200311349Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LF-sim-validationLFv-simfailednomont01-62097cce5404-10:06:13
The container "solut [...]
The container "solution" exited with code 139.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4200111348Bhavya Patwa 🇨🇦sim-exercise-1aido5-LF-sim-validationLFv-simerrornomont01-62097cce5404-11:00:26
Waited 3604 seconds [...]
Waited 3604 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4199211341Frank (Chude) Qian 🇨🇦template-tensorflowaido5-LF-sim-validationLFv-simfailednomont01-62097cce5404-10:00:51
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 327, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego" aborted:

error in ego |Exception while handling a message on topic "get_commands".
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1365, in _do_call
             ||     return fn(*args)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1349, in _run_fn
             ||     return self._call_tf_sessionrun(options, feed_dict, fetch_list,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1441, in _call_tf_sessionrun
             ||     return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
             || tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
             ||   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[{{node conv2d/Conv2D}}]]
             ||   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[{{node conv2d/Conv2D}}]]
             || 	 [[Tanh/_47]]
             || 0 successful operations.
             || 0 derived errors ignored.
             ||
             || During handling of the above exception, another exception occurred:
             ||
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 331, in loop
             ||     handle_message_node(parsed, receiver0, context0)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 486, in handle_message_node
             ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 66, in on_received_get_commands
             ||     pwm_left, pwm_right = self.compute_action(self.current_image)
             ||   File "solution.py", line 61, in compute_action
             ||     action = self.model.predict(observation)
             ||   File "/submission/model.py", line 64, in predict
             ||     action = self.tf_session.run(self._computation_graph, feed_dict={
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 957, in run
             ||     result = self._run(None, fetches, feed_dict, options_ptr,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1180, in _run
             ||     results = self._do_run(handle, final_targets, final_fetches,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1358, in _do_run
             ||     return self._do_call(_run_fn, feeds, fetches, targets, options,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1384, in _do_call
             ||     raise type(e)(node_def, op, message)
             || tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
             ||   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[node conv2d/Conv2D (defined at /submission/model.py:28) ]]
             ||   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
             || 	 [[node conv2d/Conv2D (defined at /submission/model.py:28) ]]
             || 	 [[Tanh/_47]]
             || 0 successful operations.
             || 0 derived errors ignored.
             ||
             || Errors may have originated from an input operation.
             || Input Source operations connected to node conv2d/Conv2D:
             ||  map_1/TensorArrayV2Stack/TensorListStack (defined at /submission/model.py:91)
             ||
             || Input Source operations connected to node conv2d/Conv2D:
             ||  map_1/TensorArrayV2Stack/TensorListStack (defined at /submission/model.py:91)
             ||
             || Original stack trace for 'conv2d/Conv2D':
             ||   File "solution.py", line 101, in <module>
             ||     main()
             ||   File "solution.py", line 97, in main
             ||     wrap_direct(node=node, protocol=protocol)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/interface.py", line 24, in wrap_direct
             ||     run_loop(node, protocol, args)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 244, in run_loop
             ||     loop(node_name, fi, fo, node, protocol, tin, tout, config=config, fi_desc=fin, fo_desc=fout)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 314, in loop
             ||     call_if_fun_exists(node, "init", context=context_data)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 42, in init
             ||     self.model = TfInference(observation_shape=(1,) + expect_shape,
             ||   File "/submission/model.py", line 61, in __init__
             ||     self._initialize(observation_shape, action_shape, graph_location)
             ||   File "/submission/model.py", line 86, in _initialize
             ||     self._create(input_shape, action_shape)
             ||   File "/submission/model.py", line 99, in _create
             ||     self._computation_graph = self.computation_graph()
             ||   File "/submission/model.py", line 70, in computation_graph
             ||     model = one_residual(self._preprocessed_state, seed=self.seed)
             ||   File "/submission/model.py", line 28, in one_residual
             ||     nn = tf.compat.v1.layers.conv2d(x, filters=32, kernel_size=5, strides=2, padding='same',
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
             ||     return func(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/layers/convolutional.py", line 424, in conv2d
             ||     return layer.apply(inputs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
             ||     return func(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 1695, in apply
             ||     return self.__call__(inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/layers/base.py", line 547, in __call__
             ||     outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 778, in __call__
             ||     outputs = call_fn(cast_inputs, *args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 262, in wrapper
             ||     return converted_call(f, args, kwargs, options=options)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 492, in converted_call
             ||     return _call_unconverted(f, args, kwargs, options)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 346, in _call_unconverted
             ||     return f(*args, **kwargs)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/layers/convolutional.py", line 207, in call
             ||     outputs = self._convolution_op(inputs, self.kernel)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 1106, in __call__
             ||     return self.conv_op(inp, filter)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 638, in __call__
             ||     return self.call(inp, filter)
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 231, in __call__
             ||     return self.conv_op(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/nn_ops.py", line 2006, in conv2d
             ||     return gen_nn_ops.conv2d(input,  # pylint: disable=redefined-builtin
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 965, in conv2d
             ||     _, _, _op, _outputs = _op_def_library._apply_op_helper(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 742, in _apply_op_helper
             ||     op = g._create_op_internal(op_type_name, inputs, dtypes=None,
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 3319, in _create_op_internal
             ||     ret = Operation(
             ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1791, in __init__
             ||     self._traceback = tf_stack.extract_stack()
             ||
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4198811341Frank (Chude) Qian 🇨🇦template-tensorflowaido5-LF-sim-validationLFv-simfailednomont01-62097cce5404-10:04:30
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 334, in read_until_over
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: External node "ego" closed communication.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4198611344Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simerrornomont01-62097cce5404-11:00:31
Waited 3606 seconds [...]
Waited 3606 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4197511340Frank (Chude) Qian 🇨🇦challenge-aido_LF-baseline-behavior-cloningaido5-LF-sim-validationLFv-simfailednomont01-62097cce5404-10:03:58
The container "solut [...]
The container "solution" exited with code 139.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4197111338Daniel Ng 🇸🇬template-randomaido5-LF-sim-validationLFv-simsuccessnomont01-62097cce5404-10:03:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6440373265804349
survival_time_median4.7749999999999915
deviation-center-line_median0.10361109150784235
in-drivable-lane_median2.5499999999999914


other stats
agent_compute-ego_max0.017159700393676758
agent_compute-ego_mean0.016576737167527143
agent_compute-ego_median0.016503350305374552
agent_compute-ego_min0.016140547665682705
complete-iteration_max0.20805240580529877
complete-iteration_mean0.20220640110877983
complete-iteration_median0.20111781737813547
complete-iteration_min0.19853756387354965
deviation-center-line_max0.15583868303837567
deviation-center-line_mean0.10879855071843549
deviation-center-line_min0.07213333681968147
deviation-heading_max0.976837469261332
deviation-heading_mean0.4922932418762878
deviation-heading_median0.3609623398756611
deviation-heading_min0.27041081849249704
driven_any_max2.8894611145876925
driven_any_mean1.8510551780477624
driven_any_median1.998666635086524
driven_any_min0.5174263274303096
driven_lanedir_consec_max0.8904446665936326
driven_lanedir_consec_mean0.6302364342579737
driven_lanedir_consec_min0.3424264172773923
driven_lanedir_max0.8904446665936326
driven_lanedir_mean0.6302364342579737
driven_lanedir_median0.6440373265804349
driven_lanedir_min0.3424264172773923
get_duckie_state_max2.6296686243127895e-06
get_duckie_state_mean2.5670757948048312e-06
get_duckie_state_median2.5631042337779897e-06
get_duckie_state_min2.512426087350556e-06
get_robot_state_max0.016256814532809787
get_robot_state_mean0.015318573198385676
get_robot_state_median0.015184544626323656
get_robot_state_min0.014648389008085606
get_state_dump_max0.014257164324744272
get_state_dump_mean0.014086337742529804
get_state_dump_median0.014070800230171735
get_state_dump_min0.013946586185031466
get_ui_image_max0.03592643384580259
get_ui_image_mean0.0353524959852512
get_ui_image_median0.03539883101077054
get_ui_image_min0.03468588807366111
in-drivable-lane_max4.699999999999983
in-drivable-lane_mean2.5374999999999917
in-drivable-lane_min0.3500000000000003
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.1619321797959306, "get_ui_image": 0.03554315890296031, "step_physics": 0.0791703159526243, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6085881804686812, "get_state_dump": 0.014257164324744272, "sim_render-ego": 0.006191079899416131, "get_robot_state": 0.014648389008085606, "get_duckie_state": 2.6104813915188028e-06, "in-drivable-lane": 1.1999999999999966, "agent_compute-ego": 0.017159700393676758, "deviation-heading": 0.27041081849249704, "complete-iteration": 0.19853756387354965, "set_robot_commands": 0.004695795350155588, "deviation-center-line": 0.1246386243322534, "driven_lanedir_consec": 0.6085881804686812, "sim_compute_sim_state": 0.022508588887877386, "sim_compute_performance-ego": 0.004212670407052767}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.5174263274303096, "get_ui_image": 0.035254503118580784, "step_physics": 0.08369758211333177, "survival_time": 1.4500000000000006, "driven_lanedir": 0.3424264172773923, "get_state_dump": 0.013962573018567317, "sim_render-ego": 0.006364509977143386, "get_robot_state": 0.015045470204846612, "get_duckie_state": 2.515727076037177e-06, "in-drivable-lane": 0.3500000000000003, "agent_compute-ego": 0.016325893073246396, "deviation-heading": 0.3106738984320317, "complete-iteration": 0.20282068745843296, "set_robot_commands": 0.0043163463987153155, "deviation-center-line": 0.07213333681968147, "driven_lanedir_consec": 0.3424264172773923, "sim_compute_sim_state": 0.02353807153372929, "sim_compute_performance-ego": 0.004156712828011349}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.8354010903771174, "get_ui_image": 0.03592643384580259, "step_physics": 0.07810545320864078, "survival_time": 6.749999999999984, "driven_lanedir": 0.6794864726921884, "get_state_dump": 0.013946586185031466, "sim_render-ego": 0.006144339949996383, "get_robot_state": 0.016256814532809787, "get_duckie_state": 2.6296686243127895e-06, "in-drivable-lane": 4.699999999999983, "agent_compute-ego": 0.01668080753750271, "deviation-heading": 0.41125078131929055, "complete-iteration": 0.199414947297838, "set_robot_commands": 0.004819741072478118, "deviation-center-line": 0.08258355868343131, "driven_lanedir_consec": 0.6794864726921884, "sim_compute_sim_state": 0.02322806958799009, "sim_compute_performance-ego": 0.00415088865492079}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.8894611145876925, "get_ui_image": 0.03468588807366111, "step_physics": 0.08618797858556111, "survival_time": 6.5999999999999845, "driven_lanedir": 0.8904446665936326, "get_state_dump": 0.01417902744177616, "sim_render-ego": 0.005988260110219319, "get_robot_state": 0.0153236190478007, "get_duckie_state": 2.512426087350556e-06, "in-drivable-lane": 3.899999999999986, "agent_compute-ego": 0.016140547665682705, "deviation-heading": 0.976837469261332, "complete-iteration": 0.20805240580529877, "set_robot_commands": 0.004763456908139316, "deviation-center-line": 0.15583868303837567, "driven_lanedir_consec": 0.8904446665936326, "sim_compute_sim_state": 0.026558791146133885, "sim_compute_performance-ego": 0.004064901308579879}}
set_robot_commands_max0.004819741072478118
set_robot_commands_mean0.004648834932372084
set_robot_commands_median0.004729626129147452
set_robot_commands_min0.0043163463987153155
sim_compute_performance-ego_max0.004212670407052767
sim_compute_performance-ego_mean0.004146293299641196
sim_compute_performance-ego_median0.004153800741466069
sim_compute_performance-ego_min0.004064901308579879
sim_compute_sim_state_max0.026558791146133885
sim_compute_sim_state_mean0.02395838028893266
sim_compute_sim_state_median0.02338307056085969
sim_compute_sim_state_min0.022508588887877386
sim_render-ego_max0.006364509977143386
sim_render-ego_mean0.006172047484193805
sim_render-ego_median0.006167709924706257
sim_render-ego_min0.005988260110219319
simulation-passed1
step_physics_max0.08618797858556111
step_physics_mean0.08179033246503949
step_physics_median0.08143394903297804
step_physics_min0.07810545320864078
survival_time_max6.749999999999984
survival_time_mean4.437499999999992
survival_time_min1.4500000000000006
No reset possible