Duckietown Challenges Home Challenges Submissions

Evaluator 571

ID571
evaluatoridsc-rudolf-5273
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-5273
versiond-c:4.0.11;d-c-r:4.0.10;d-s:4.0.12
first heard
last heard
statusinactive
# evaluating
# success1 17205
# timeout
# failed1 17215
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
172152351Andrea Censi 🇨🇭Baseline solution using imitation learning from logsaido2_LF_r3-v4-z2step1-simulationfailedyesidsc-rudolf-52730:01:53
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 285, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 92, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 173, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 254, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 282, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 66, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 41, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/project/graph_utils.py", line 8, in load_graph
               ||     graph_def.ParseFromString(f.read())
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 125, in read
               ||     self._preread_check()
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 85, in _preread_check
               ||     compat.as_bytes(self.__name), 1024 * 512, status)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
               ||     c_api.TF_GetCode(self.status.status))
               || tensorflow.python.framework.errors_impl.NotFoundError: frozen_graph.pb; No such file or directory
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 426, in <module>
    wrap(cie)
  File "experiment_manager.py", line 415, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 128, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 289, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
172052341Andrea Censi 🇨🇭Minimal agent (Python 2)aido2_LFV_r3-v4-z2step1-simulationsuccessyesidsc-rudolf-52730:09:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.2501898935829012
survival_time_median2.6499999999999986
deviation-center-line_median0.02537368915309797
in-drivable-lane_median0.4999999999999982


other stats
agent_compute-ego_max0.054201378012603184
agent_compute-ego_mean0.05117653767190814
agent_compute-ego_median0.0512409262035204
agent_compute-ego_min0.048940853639082474
deviation-center-line_max0.03472425682418009
deviation-center-line_mean0.026347234341453415
deviation-center-line_min0.020890767023363815
deviation-heading_max0.20304552908957563
deviation-heading_mean0.19365741902006536
deviation-heading_median0.2030454232515352
deviation-heading_min0.1784042009688801
driven_any_max2.7599999999999936
driven_any_mean1.7039999999999988
driven_any_median1.59
driven_any_min0.9900000000000012
driven_lanedir_consec_max2.526717455318104
driven_lanedir_consec_mean1.3866776713319673
driven_lanedir_consec_min0.6561020726944444
driven_lanedir_max2.526717455318104
driven_lanedir_mean1.3866776713319673
driven_lanedir_median1.2501898935829012
driven_lanedir_min0.6561020726944444
in-drivable-lane_max0.5000000000000004
in-drivable-lane_mean0.4699999999999987
in-drivable-lane_min0.34999999999999876
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9900000000000012, "sim_physics": 0.02015704097169818, "survival_time": 1.6500000000000008, "driven_lanedir": 0.6561020726944444, "sim_render-ego": 0.05994986765312426, "in-drivable-lane": 0.5000000000000004, "agent_compute-ego": 0.048940853639082474, "deviation-heading": 0.18074641270076017, "set_robot_commands": 0.07153375221021248, "deviation-center-line": 0.020890767023363815, "driven_lanedir_consec": 0.6561020726944444, "sim_compute_sim_state": 0.03310759139783455, "sim_compute_performance-ego": 0.061194044170957626, "sim_compute_robot_state-ego": 0.059384736147793854, "sim_compute_robot_state-npc0": 0.06428044492548163, "sim_compute_robot_state-npc1": 0.06126153107845422, "sim_compute_robot_state-npc2": 0.06006962602788752, "sim_compute_robot_state-npc3": 0.05993861863107392}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.7599999999999936, "sim_physics": 0.024204337078592052, "survival_time": 4.599999999999992, "driven_lanedir": 2.526717455318104, "sim_render-ego": 0.06204852332239565, "in-drivable-lane": 0.34999999999999876, "agent_compute-ego": 0.0512409262035204, "deviation-heading": 0.1784042009688801, "set_robot_commands": 0.07254914356314618, "deviation-center-line": 0.03472425682418009, "driven_lanedir_consec": 2.526717455318104, "sim_compute_sim_state": 0.03413891014845475, "sim_compute_performance-ego": 0.061486594055009926, "sim_compute_robot_state-ego": 0.06127390654190727, "sim_compute_robot_state-npc0": 0.06889450550079346, "sim_compute_robot_state-npc1": 0.0634070608926856, "sim_compute_robot_state-npc2": 0.06225109359492426, "sim_compute_robot_state-npc3": 0.06081017981404844}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.59, "sim_physics": 0.023029115964781563, "survival_time": 2.6499999999999986, "driven_lanedir": 1.2501898935829012, "sim_render-ego": 0.06072159533230763, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.05154379808677817, "deviation-heading": 0.20304552908957563, "set_robot_commands": 0.07540403671984403, "deviation-center-line": 0.02537368915309797, "driven_lanedir_consec": 1.2501898935829012, "sim_compute_sim_state": 0.03467410915302781, "sim_compute_performance-ego": 0.05903265611180719, "sim_compute_robot_state-ego": 0.06176119930339309, "sim_compute_robot_state-npc0": 0.06699384833281895, "sim_compute_robot_state-npc1": 0.062358091462333246, "sim_compute_robot_state-npc2": 0.060694856463738206, "sim_compute_robot_state-npc3": 0.06093659041062841}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.59, "sim_physics": 0.02158541499443774, "survival_time": 2.6499999999999986, "driven_lanedir": 1.2501898935829012, "sim_render-ego": 0.05818772315979004, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.04995573241755648, "deviation-heading": 0.20304552908957563, "set_robot_commands": 0.07419957754746923, "deviation-center-line": 0.02537368915309797, "driven_lanedir_consec": 1.2501898935829012, "sim_compute_sim_state": 0.03271445688211693, "sim_compute_performance-ego": 0.058459551829212114, "sim_compute_robot_state-ego": 0.05928401677113659, "sim_compute_robot_state-npc0": 0.06749619178052219, "sim_compute_robot_state-npc1": 0.06274864358722039, "sim_compute_robot_state-npc2": 0.0597369355975457, "sim_compute_robot_state-npc3": 0.059124249332356}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.59, "sim_physics": 0.022349200158748982, "survival_time": 2.6499999999999986, "driven_lanedir": 1.2501890414814851, "sim_render-ego": 0.06454455177738981, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.054201378012603184, "deviation-heading": 0.2030454232515352, "set_robot_commands": 0.07389748321389253, "deviation-center-line": 0.02537376955352725, "driven_lanedir_consec": 1.2501890414814851, "sim_compute_sim_state": 0.03577478876653707, "sim_compute_performance-ego": 0.062577818924526, "sim_compute_robot_state-ego": 0.06198740005493164, "sim_compute_robot_state-npc0": 0.07234203590537017, "sim_compute_robot_state-npc1": 0.06624097194311754, "sim_compute_robot_state-npc2": 0.06383624616658913, "sim_compute_robot_state-npc3": 0.06364142219975309}}
set_robot_commands_max0.07540403671984403
set_robot_commands_mean0.0735167986509129
set_robot_commands_median0.07389748321389253
set_robot_commands_min0.07153375221021248
sim_compute_performance-ego_max0.062577818924526
sim_compute_performance-ego_mean0.060550133018302574
sim_compute_performance-ego_median0.061194044170957626
sim_compute_performance-ego_min0.058459551829212114
sim_compute_robot_state-ego_max0.06198740005493164
sim_compute_robot_state-ego_mean0.0607382517638325
sim_compute_robot_state-ego_median0.06127390654190727
sim_compute_robot_state-ego_min0.05928401677113659
sim_compute_robot_state-npc0_max0.07234203590537017
sim_compute_robot_state-npc0_mean0.06800140528899727
sim_compute_robot_state-npc0_median0.06749619178052219
sim_compute_robot_state-npc0_min0.06428044492548163
sim_compute_robot_state-npc1_max0.06624097194311754
sim_compute_robot_state-npc1_mean0.0632032597927622
sim_compute_robot_state-npc1_median0.06274864358722039
sim_compute_robot_state-npc1_min0.06126153107845422
sim_compute_robot_state-npc2_max0.06383624616658913
sim_compute_robot_state-npc2_mean0.06131775157013697
sim_compute_robot_state-npc2_median0.060694856463738206
sim_compute_robot_state-npc2_min0.0597369355975457
sim_compute_robot_state-npc3_max0.06364142219975309
sim_compute_robot_state-npc3_mean0.06089021207757197
sim_compute_robot_state-npc3_median0.06081017981404844
sim_compute_robot_state-npc3_min0.059124249332356
sim_compute_sim_state_max0.03577478876653707
sim_compute_sim_state_mean0.034081971269594225
sim_compute_sim_state_median0.03413891014845475
sim_compute_sim_state_min0.03271445688211693
sim_physics_max0.024204337078592052
sim_physics_mean0.022265021833651703
sim_physics_median0.022349200158748982
sim_physics_min0.02015704097169818
sim_render-ego_max0.06454455177738981
sim_render-ego_mean0.06109045224900148
sim_render-ego_median0.06072159533230763
sim_render-ego_min0.05818772315979004
simulation-passed1
survival_time_max4.599999999999992
survival_time_mean2.8399999999999976
survival_time_min1.6500000000000008
No reset possible