Duckietown Challenges Home Challenges Submissions

Job 22280

Job ID22280
submission3452
userAnastasiya Nikolskaya 🇷🇺
user labelJetBrains Research
challengeaido2-LFVI-sim-validation
stepstep1-simulation
statusfailed
up to dateyes
evaluatorip-172-31-25-98-2470
date started
date completed
duration1 day, 3:11:15
message
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 293, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 297, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 236, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 1 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 441, in <module>
    wrap(cie)
  File "experiment_manager.py", line 429, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.894634354252406
survival_time_median14.950000000000076
deviation-center-line_median1.0750269113686055
in-drivable-lane_median1.7500000000000249


other stats
agent_compute-ego_max0.21811933914820353
agent_compute-ego_mean0.12223773339283037
agent_compute-ego_median0.08779063940048218
agent_compute-ego_min0.084898746157267
deviation-center-line_max1.2961775979669246
deviation-center-line_mean0.9937003543566776
deviation-center-line_min0.4412642023290711
deviation-heading_max4.623851797560124
deviation-heading_mean3.681972142004979
deviation-heading_median3.831121292519462
deviation-heading_min2.415804935060974
driven_any_max3.6902566655861566
driven_any_mean3.1199339744598955
driven_any_median3.419418874509516
driven_any_min1.6694894924348076
driven_lanedir_consec_max2.948195558631825
driven_lanedir_consec_mean2.400817074981129
driven_lanedir_consec_min0.875744873385248
driven_lanedir_max3.170424746784956
driven_lanedir_mean2.5236350213527956
driven_lanedir_median2.926415292065797
driven_lanedir_min0.9321966990147958
in-drivable-lane_max3.4499999999999975
in-drivable-lane_mean2.160000000000005
in-drivable-lane_min0.9500000000000136
per-episodes
details{"udem1-0-0": {"driven_any": 3.6902566655861566, "sim_physics": 0.2959579602877299, "survival_time": 14.950000000000076, "driven_lanedir": 3.157838132598873, "sim_render-ego": 0.08020962238311767, "in-drivable-lane": 1.7500000000000249, "agent_compute-ego": 0.21811933914820353, "deviation-heading": 3.831121292519462, "set_robot_commands": 0.14027276992797852, "deviation-center-line": 1.2961775979669246, "driven_lanedir_consec": 2.894634354252406, "sim_compute_sim_state": 0.049777410825093585, "sim_compute_performance-ego": 0.08999715487162273, "sim_compute_robot_state-ego": 0.11034743944803874, "sim_compute_robot_state-npc0": 0.0922820512453715, "sim_compute_robot_state-npc1": 0.0907405916849772, "sim_compute_robot_state-npc2": 0.08952558120091757, "sim_compute_robot_state-npc3": 0.08917390743891399}, "udem1-1-0": {"driven_any": 3.419418874509516, "sim_physics": 0.2673739417394002, "survival_time": 14.950000000000076, "driven_lanedir": 2.926415292065797, "sim_render-ego": 0.07168716589609782, "in-drivable-lane": 1.5000000000000018, "agent_compute-ego": 0.13428436199824015, "deviation-heading": 4.623851797560124, "set_robot_commands": 0.124509113629659, "deviation-center-line": 0.924642188180164, "driven_lanedir_consec": 2.926415292065797, "sim_compute_sim_state": 0.04650932947794596, "sim_compute_performance-ego": 0.07997717857360839, "sim_compute_robot_state-ego": 0.09376842419306436, "sim_compute_robot_state-npc0": 0.08685780207316081, "sim_compute_robot_state-npc1": 0.08518275101979574, "sim_compute_robot_state-npc2": 0.08465395927429199, "sim_compute_robot_state-npc3": 0.08592929363250733}, "udem1-2-0": {"driven_any": 3.268606159461751, "sim_physics": 0.24375551621119185, "survival_time": 14.950000000000076, "driven_lanedir": 2.4313002362995566, "sim_render-ego": 0.06500558853149414, "in-drivable-lane": 3.4499999999999975, "agent_compute-ego": 0.0860955802599589, "deviation-heading": 3.5659669410336807, "set_robot_commands": 0.11262636264165242, "deviation-center-line": 1.0750269113686055, "driven_lanedir_consec": 2.3590952965703686, "sim_compute_sim_state": 0.044104844729105634, "sim_compute_performance-ego": 0.07310290257136028, "sim_compute_robot_state-ego": 0.0788605268796285, "sim_compute_robot_state-npc0": 0.07910483837127685, "sim_compute_robot_state-npc1": 0.08250422080357869, "sim_compute_robot_state-npc2": 0.08026126543680827, "sim_compute_robot_state-npc3": 0.07905077219009399}, "udem1-3-0": {"driven_any": 1.6694894924348076, "sim_physics": 0.22845309039196335, "survival_time": 8.299999999999983, "driven_lanedir": 0.9321966990147958, "sim_render-ego": 0.062130093574523926, "in-drivable-lane": 3.149999999999989, "agent_compute-ego": 0.084898746157267, "deviation-heading": 2.415804935060974, "set_robot_commands": 0.10662843951259753, "deviation-center-line": 0.4412642023290711, "driven_lanedir_consec": 0.875744873385248, "sim_compute_sim_state": 0.04304432150829269, "sim_compute_performance-ego": 0.06898655517991767, "sim_compute_robot_state-ego": 0.07582767900214138, "sim_compute_robot_state-npc0": 0.07690930510141764, "sim_compute_robot_state-npc1": 0.07611721682261272, "sim_compute_robot_state-npc2": 0.07426904770265143, "sim_compute_robot_state-npc3": 0.07520913216004889}, "udem1-4-0": {"driven_any": 3.551898680307246, "sim_physics": 0.2245457355181376, "survival_time": 14.950000000000076, "driven_lanedir": 3.170424746784956, "sim_render-ego": 0.06650614817937216, "in-drivable-lane": 0.9500000000000136, "agent_compute-ego": 0.08779063940048218, "deviation-heading": 3.973115743850656, "set_robot_commands": 0.1186728811264038, "deviation-center-line": 1.2313908719386224, "driven_lanedir_consec": 2.948195558631825, "sim_compute_sim_state": 0.044820773601531985, "sim_compute_performance-ego": 0.07430827299753825, "sim_compute_robot_state-ego": 0.08149157524108887, "sim_compute_robot_state-npc0": 0.08018644332885742, "sim_compute_robot_state-npc1": 0.08007022937138876, "sim_compute_robot_state-npc2": 0.08002737840016683, "sim_compute_robot_state-npc3": 0.0802197249730428}}
set_robot_commands_max0.14027276992797852
set_robot_commands_mean0.12054191336765828
set_robot_commands_median0.1186728811264038
set_robot_commands_min0.10662843951259753
sim_compute_performance-ego_max0.08999715487162273
sim_compute_performance-ego_mean0.07727441283880947
sim_compute_performance-ego_median0.07430827299753825
sim_compute_performance-ego_min0.06898655517991767
sim_compute_robot_state-ego_max0.11034743944803874
sim_compute_robot_state-ego_mean0.08805912895279237
sim_compute_robot_state-ego_median0.08149157524108887
sim_compute_robot_state-ego_min0.07582767900214138
sim_compute_robot_state-npc0_max0.0922820512453715
sim_compute_robot_state-npc0_mean0.08306808802401686
sim_compute_robot_state-npc0_median0.08018644332885742
sim_compute_robot_state-npc0_min0.07690930510141764
sim_compute_robot_state-npc1_max0.0907405916849772
sim_compute_robot_state-npc1_mean0.08292300194047061
sim_compute_robot_state-npc1_median0.08250422080357869
sim_compute_robot_state-npc1_min0.07611721682261272
sim_compute_robot_state-npc2_max0.08952558120091757
sim_compute_robot_state-npc2_mean0.08174744640296722
sim_compute_robot_state-npc2_median0.08026126543680827
sim_compute_robot_state-npc2_min0.07426904770265143
sim_compute_robot_state-npc3_max0.08917390743891399
sim_compute_robot_state-npc3_mean0.0819165660789214
sim_compute_robot_state-npc3_median0.0802197249730428
sim_compute_robot_state-npc3_min0.07520913216004889
sim_compute_sim_state_max0.049777410825093585
sim_compute_sim_state_mean0.045651336028393975
sim_compute_sim_state_median0.044820773601531985
sim_compute_sim_state_min0.04304432150829269
sim_physics_max0.2959579602877299
sim_physics_mean0.2520172488296846
sim_physics_median0.24375551621119185
sim_physics_min0.2245457355181376
sim_render-ego_max0.08020962238311767
sim_render-ego_mean0.06910772371292115
sim_render-ego_median0.06650614817937216
sim_render-ego_min0.062130093574523926
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean13.620000000000058
survival_time_min8.299999999999983

Highlights

22280

Click the images to see detailed statistics about the episode.

udem1-0-0

udem1-1-0

udem1-2-0

udem1-3-0

udem1-4-0

Artifacts

The artifacts are hidden.

Container logs

The logs are hidden.