Duckietown Challenges Home Challenges Submissions

Evaluator 570

ID570
evaluatoridsc-rudolf-5021
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-5021
versiond-c:4.0.11;d-c-r:4.0.10;d-s:4.0.12
first heard
last heard
statusinactive
# evaluating
# success1 17202
# timeout
# failed1 17212
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
172122348Andrea Censi 🇨🇭Baseline solution using imitation learning from logsaido2_LF_r3-v4-z2step1-simulationfailedyesidsc-rudolf-50210:02:00
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 285, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 92, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 173, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 254, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 282, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 66, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 41, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/project/graph_utils.py", line 8, in load_graph
               ||     graph_def.ParseFromString(f.read())
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 125, in read
               ||     self._preread_check()
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 85, in _preread_check
               ||     compat.as_bytes(self.__name), 1024 * 512, status)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
               ||     c_api.TF_GetCode(self.status.status))
               || tensorflow.python.framework.errors_impl.NotFoundError: frozen_graph.pb; No such file or directory
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 426, in <module>
    wrap(cie)
  File "experiment_manager.py", line 415, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 128, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 289, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
172022342Andrea Censi 🇨🇭Minimal agent (Python 2)aido2_LFVI_r3-v4-z2step1-simulationsuccessyesidsc-rudolf-50210:10:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.826098585215639
survival_time_median3.599999999999995
deviation-center-line_median0.025373690085396063
in-drivable-lane_median0.4999999999999982


other stats
agent_compute-ego_max0.05346925362296726
agent_compute-ego_mean0.0525686128599161
agent_compute-ego_median0.05317525068918864
agent_compute-ego_min0.05117119442332874
deviation-center-line_max0.03958642879007866
deviation-center-line_mean0.029265581092197568
deviation-center-line_min0.020890783684312287
deviation-heading_max0.20304544699722185
deviation-heading_mean0.19087562509549852
deviation-heading_median0.19491992237395445
deviation-heading_min0.1807463331880334
driven_any_max2.7599999999999896
driven_any_mean1.8119999999999952
driven_any_median2.1599999999999926
driven_any_min0.9900000000000008
driven_lanedir_consec_max2.4201910166177933
driven_lanedir_consec_mean1.5279452742756314
driven_lanedir_consec_min0.6561021167721157
driven_lanedir_max2.4201910166177933
driven_lanedir_mean1.5279452742756314
driven_lanedir_median1.826098585215639
driven_lanedir_min0.6561021167721157
in-drivable-lane_max0.5000000000000004
in-drivable-lane_mean0.4199999999999992
in-drivable-lane_min0.29999999999999893
per-episodes
details{"udem1-0-0": {"driven_any": 2.1599999999999944, "sim_physics": 0.027287479903962877, "survival_time": 3.599999999999995, "driven_lanedir": 1.953669056780216, "sim_render-ego": 0.06110626459121704, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.05343579583697849, "deviation-heading": 0.19491992237395445, "set_robot_commands": 0.07620099518034193, "deviation-center-line": 0.03958642879007866, "driven_lanedir_consec": 1.953669056780216, "sim_compute_sim_state": 0.033701048956976995, "sim_compute_performance-ego": 0.0594425532552931, "sim_compute_robot_state-ego": 0.06153918968306647, "sim_compute_robot_state-npc0": 0.06714367866516113, "sim_compute_robot_state-npc1": 0.06214068002170987, "sim_compute_robot_state-npc2": 0.06313915054003398, "sim_compute_robot_state-npc3": 0.06060884396235148}, "udem1-1-0": {"driven_any": 0.9900000000000012, "sim_physics": 0.024123358004020923, "survival_time": 1.6500000000000008, "driven_lanedir": 0.6561021167721157, "sim_render-ego": 0.060758388403690224, "in-drivable-lane": 0.5000000000000004, "agent_compute-ego": 0.05159156972711736, "deviation-heading": 0.1807463331880334, "set_robot_commands": 0.07674280079928311, "deviation-center-line": 0.020890783684312287, "driven_lanedir_consec": 0.6561021167721157, "sim_compute_sim_state": 0.03621233593333851, "sim_compute_performance-ego": 0.061396938381773056, "sim_compute_robot_state-ego": 0.062445994579430786, "sim_compute_robot_state-npc0": 0.06739663355278247, "sim_compute_robot_state-npc1": 0.06592965126037598, "sim_compute_robot_state-npc2": 0.06266837409048369, "sim_compute_robot_state-npc3": 0.06320419456019546}, "udem1-2-0": {"driven_any": 2.1599999999999926, "sim_physics": 0.027884158823225234, "survival_time": 3.599999999999995, "driven_lanedir": 1.826098585215639, "sim_render-ego": 0.05998926361401876, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.05317525068918864, "deviation-heading": 0.1807464733723929, "set_robot_commands": 0.07615980174806383, "deviation-center-line": 0.02089092244274933, "driven_lanedir_consec": 1.826098585215639, "sim_compute_sim_state": 0.03415017326672872, "sim_compute_performance-ego": 0.06203640500704447, "sim_compute_robot_state-ego": 0.062329398261176214, "sim_compute_robot_state-npc0": 0.06677951084242927, "sim_compute_robot_state-npc1": 0.061951541238360934, "sim_compute_robot_state-npc2": 0.062280989355511136, "sim_compute_robot_state-npc3": 0.0623138944307963}, "udem1-3-0": {"driven_any": 2.7599999999999896, "sim_physics": 0.02818237698596457, "survival_time": 4.599999999999992, "driven_lanedir": 2.4201910166177933, "sim_render-ego": 0.05964158928912619, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.05346925362296726, "deviation-heading": 0.20304544699722185, "set_robot_commands": 0.07583614017652429, "deviation-center-line": 0.025373690085396063, "driven_lanedir_consec": 2.4201910166177933, "sim_compute_sim_state": 0.03456672378208326, "sim_compute_performance-ego": 0.06040255401445472, "sim_compute_robot_state-ego": 0.06066909561986509, "sim_compute_robot_state-npc0": 0.06449560756268709, "sim_compute_robot_state-npc1": 0.06157075581343278, "sim_compute_robot_state-npc2": 0.0610857631849206, "sim_compute_robot_state-npc3": 0.05981564003488292}, "udem1-4-0": {"driven_any": 0.9900000000000008, "sim_physics": 0.023354956597992867, "survival_time": 1.6500000000000008, "driven_lanedir": 0.7836655959923924, "sim_render-ego": 0.058161959503636215, "in-drivable-lane": 0.30000000000000027, "agent_compute-ego": 0.05117119442332874, "deviation-heading": 0.19491994954589, "set_robot_commands": 0.07419719118060487, "deviation-center-line": 0.039586080458451495, "driven_lanedir_consec": 0.7836655959923924, "sim_compute_sim_state": 0.03313369461984345, "sim_compute_performance-ego": 0.05965720523487438, "sim_compute_robot_state-ego": 0.05980584838173606, "sim_compute_robot_state-npc0": 0.062778783567024, "sim_compute_robot_state-npc1": 0.0595857302347819, "sim_compute_robot_state-npc2": 0.05771480184612853, "sim_compute_robot_state-npc3": 0.05980733669165409}}
set_robot_commands_max0.07674280079928311
set_robot_commands_mean0.0758273858169636
set_robot_commands_median0.07615980174806383
set_robot_commands_min0.07419719118060487
sim_compute_performance-ego_max0.06203640500704447
sim_compute_performance-ego_mean0.06058713117868795
sim_compute_performance-ego_median0.06040255401445472
sim_compute_performance-ego_min0.0594425532552931
sim_compute_robot_state-ego_max0.062445994579430786
sim_compute_robot_state-ego_mean0.06135790530505493
sim_compute_robot_state-ego_median0.06153918968306647
sim_compute_robot_state-ego_min0.05980584838173606
sim_compute_robot_state-npc0_max0.06739663355278247
sim_compute_robot_state-npc0_mean0.0657188428380168
sim_compute_robot_state-npc0_median0.06677951084242927
sim_compute_robot_state-npc0_min0.062778783567024
sim_compute_robot_state-npc1_max0.06592965126037598
sim_compute_robot_state-npc1_mean0.0622356717137323
sim_compute_robot_state-npc1_median0.061951541238360934
sim_compute_robot_state-npc1_min0.0595857302347819
sim_compute_robot_state-npc2_max0.06313915054003398
sim_compute_robot_state-npc2_mean0.06137781580341558
sim_compute_robot_state-npc2_median0.062280989355511136
sim_compute_robot_state-npc2_min0.05771480184612853
sim_compute_robot_state-npc3_max0.06320419456019546
sim_compute_robot_state-npc3_mean0.06114998193597605
sim_compute_robot_state-npc3_median0.06060884396235148
sim_compute_robot_state-npc3_min0.05980733669165409
sim_compute_sim_state_max0.03621233593333851
sim_compute_sim_state_mean0.034352795311794186
sim_compute_sim_state_median0.03415017326672872
sim_compute_sim_state_min0.03313369461984345
sim_physics_max0.02818237698596457
sim_physics_mean0.026166466063033293
sim_physics_median0.027287479903962877
sim_physics_min0.023354956597992867
sim_render-ego_max0.06110626459121704
sim_render-ego_mean0.05993149308033768
sim_render-ego_median0.05998926361401876
sim_render-ego_min0.058161959503636215
simulation-passed1
survival_time_max4.599999999999992
survival_time_mean3.0199999999999965
survival_time_min1.6500000000000008
No reset possible