AI Driving Olympics Home Challenges Submissions Jobs

Submission 3452

Submission3452
Competingyes
Challengeaido2-LFVI-sim-validation
UserJBRRussia
Date submitted
Completecomplete
Resultfailed
Jobsstep1-simulation: 22280
Next
User labelJetBrains Research
Admin priority50
User priority50

22280

Click the images to see detailed statistics about the episode.

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
222803452JBRRussiaJetBrains Researchaido2-LFVI-sim-validationstep1-simulationfailedyes-1 day, 3:11:15
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 293, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 297, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 236, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 1 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 441, in <module>
    wrap(cie)
  File "experiment_manager.py", line 429, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
(hidden)
driven_lanedir_consec_median2.894634354252406
survival_time_median14.950000000000076
deviation-center-line_median1.0750269113686055
in-drivable-lane_median1.7500000000000249


other stats
agent_compute-ego_max0.21811933914820353
agent_compute-ego_mean0.12223773339283037
agent_compute-ego_median0.08779063940048218
agent_compute-ego_min0.084898746157267
deviation-center-line_max1.2961775979669246
deviation-center-line_mean0.9937003543566776
deviation-center-line_min0.4412642023290711
deviation-heading_max4.623851797560124
deviation-heading_mean3.681972142004979
deviation-heading_median3.831121292519462
deviation-heading_min2.415804935060974
driven_any_max3.6902566655861566
driven_any_mean3.1199339744598955
driven_any_median3.419418874509516
driven_any_min1.6694894924348076
driven_lanedir_consec_max2.948195558631825
driven_lanedir_consec_mean2.400817074981129
driven_lanedir_consec_min0.875744873385248
driven_lanedir_max3.170424746784956
driven_lanedir_mean2.5236350213527956
driven_lanedir_median2.926415292065797
driven_lanedir_min0.9321966990147958
in-drivable-lane_max3.4499999999999975
in-drivable-lane_mean2.160000000000005
in-drivable-lane_min0.9500000000000136
per-episodes
details{"udem1-0-0": {"driven_any": 3.6902566655861566, "sim_physics": 0.2959579602877299, "survival_time": 14.950000000000076, "driven_lanedir": 3.157838132598873, "sim_render-ego": 0.08020962238311767, "in-drivable-lane": 1.7500000000000249, "agent_compute-ego": 0.21811933914820353, "deviation-heading": 3.831121292519462, "set_robot_commands": 0.14027276992797852, "deviation-center-line": 1.2961775979669246, "driven_lanedir_consec": 2.894634354252406, "sim_compute_sim_state": 0.049777410825093585, "sim_compute_performance-ego": 0.08999715487162273, "sim_compute_robot_state-ego": 0.11034743944803874, "sim_compute_robot_state-npc0": 0.0922820512453715, "sim_compute_robot_state-npc1": 0.0907405916849772, "sim_compute_robot_state-npc2": 0.08952558120091757, "sim_compute_robot_state-npc3": 0.08917390743891399}, "udem1-1-0": {"driven_any": 3.419418874509516, "sim_physics": 0.2673739417394002, "survival_time": 14.950000000000076, "driven_lanedir": 2.926415292065797, "sim_render-ego": 0.07168716589609782, "in-drivable-lane": 1.5000000000000018, "agent_compute-ego": 0.13428436199824015, "deviation-heading": 4.623851797560124, "set_robot_commands": 0.124509113629659, "deviation-center-line": 0.924642188180164, "driven_lanedir_consec": 2.926415292065797, "sim_compute_sim_state": 0.04650932947794596, "sim_compute_performance-ego": 0.07997717857360839, "sim_compute_robot_state-ego": 0.09376842419306436, "sim_compute_robot_state-npc0": 0.08685780207316081, "sim_compute_robot_state-npc1": 0.08518275101979574, "sim_compute_robot_state-npc2": 0.08465395927429199, "sim_compute_robot_state-npc3": 0.08592929363250733}, "udem1-2-0": {"driven_any": 3.268606159461751, "sim_physics": 0.24375551621119185, "survival_time": 14.950000000000076, "driven_lanedir": 2.4313002362995566, "sim_render-ego": 0.06500558853149414, "in-drivable-lane": 3.4499999999999975, "agent_compute-ego": 0.0860955802599589, "deviation-heading": 3.5659669410336807, "set_robot_commands": 0.11262636264165242, "deviation-center-line": 1.0750269113686055, "driven_lanedir_consec": 2.3590952965703686, "sim_compute_sim_state": 0.044104844729105634, "sim_compute_performance-ego": 0.07310290257136028, "sim_compute_robot_state-ego": 0.0788605268796285, "sim_compute_robot_state-npc0": 0.07910483837127685, "sim_compute_robot_state-npc1": 0.08250422080357869, "sim_compute_robot_state-npc2": 0.08026126543680827, "sim_compute_robot_state-npc3": 0.07905077219009399}, "udem1-3-0": {"driven_any": 1.6694894924348076, "sim_physics": 0.22845309039196335, "survival_time": 8.299999999999983, "driven_lanedir": 0.9321966990147958, "sim_render-ego": 0.062130093574523926, "in-drivable-lane": 3.149999999999989, "agent_compute-ego": 0.084898746157267, "deviation-heading": 2.415804935060974, "set_robot_commands": 0.10662843951259753, "deviation-center-line": 0.4412642023290711, "driven_lanedir_consec": 0.875744873385248, "sim_compute_sim_state": 0.04304432150829269, "sim_compute_performance-ego": 0.06898655517991767, "sim_compute_robot_state-ego": 0.07582767900214138, "sim_compute_robot_state-npc0": 0.07690930510141764, "sim_compute_robot_state-npc1": 0.07611721682261272, "sim_compute_robot_state-npc2": 0.07426904770265143, "sim_compute_robot_state-npc3": 0.07520913216004889}, "udem1-4-0": {"driven_any": 3.551898680307246, "sim_physics": 0.2245457355181376, "survival_time": 14.950000000000076, "driven_lanedir": 3.170424746784956, "sim_render-ego": 0.06650614817937216, "in-drivable-lane": 0.9500000000000136, "agent_compute-ego": 0.08779063940048218, "deviation-heading": 3.973115743850656, "set_robot_commands": 0.1186728811264038, "deviation-center-line": 1.2313908719386224, "driven_lanedir_consec": 2.948195558631825, "sim_compute_sim_state": 0.044820773601531985, "sim_compute_performance-ego": 0.07430827299753825, "sim_compute_robot_state-ego": 0.08149157524108887, "sim_compute_robot_state-npc0": 0.08018644332885742, "sim_compute_robot_state-npc1": 0.08007022937138876, "sim_compute_robot_state-npc2": 0.08002737840016683, "sim_compute_robot_state-npc3": 0.0802197249730428}}
set_robot_commands_max0.14027276992797852
set_robot_commands_mean0.12054191336765828
set_robot_commands_median0.1186728811264038
set_robot_commands_min0.10662843951259753
sim_compute_performance-ego_max0.08999715487162273
sim_compute_performance-ego_mean0.07727441283880947
sim_compute_performance-ego_median0.07430827299753825
sim_compute_performance-ego_min0.06898655517991767
sim_compute_robot_state-ego_max0.11034743944803874
sim_compute_robot_state-ego_mean0.08805912895279237
sim_compute_robot_state-ego_median0.08149157524108887
sim_compute_robot_state-ego_min0.07582767900214138
sim_compute_robot_state-npc0_max0.0922820512453715
sim_compute_robot_state-npc0_mean0.08306808802401686
sim_compute_robot_state-npc0_median0.08018644332885742
sim_compute_robot_state-npc0_min0.07690930510141764
sim_compute_robot_state-npc1_max0.0907405916849772
sim_compute_robot_state-npc1_mean0.08292300194047061
sim_compute_robot_state-npc1_median0.08250422080357869
sim_compute_robot_state-npc1_min0.07611721682261272
sim_compute_robot_state-npc2_max0.08952558120091757
sim_compute_robot_state-npc2_mean0.08174744640296722
sim_compute_robot_state-npc2_median0.08026126543680827
sim_compute_robot_state-npc2_min0.07426904770265143
sim_compute_robot_state-npc3_max0.08917390743891399
sim_compute_robot_state-npc3_mean0.0819165660789214
sim_compute_robot_state-npc3_median0.0802197249730428
sim_compute_robot_state-npc3_min0.07520913216004889
sim_compute_sim_state_max0.049777410825093585
sim_compute_sim_state_mean0.045651336028393975
sim_compute_sim_state_median0.044820773601531985
sim_compute_sim_state_min0.04304432150829269
sim_physics_max0.2959579602877299
sim_physics_mean0.2520172488296846
sim_physics_median0.24375551621119185
sim_physics_min0.2245457355181376
sim_render-ego_max0.08020962238311767
sim_render-ego_mean0.06910772371292115
sim_render-ego_median0.06650614817937216
sim_render-ego_min0.062130093574523926
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean13.620000000000058
survival_time_min8.299999999999983
221243452JBRRussiaJetBrains Researchaido2-LFVI-sim-validationstep1-simulationsuccessno-0:31:31(hidden)
driven_lanedir_consec_median2.300489392928398
survival_time_median14.950000000000076
deviation-center-line_median0.9407381474519764
in-drivable-lane_median3.7000000000000295


other stats
agent_compute-ego_max0.08527889410654704
agent_compute-ego_mean0.0784413893834047
agent_compute-ego_median0.0764496869054334
agent_compute-ego_min0.07547996838887533
deviation-center-line_max1.1059561988401976
deviation-center-line_mean0.809301320776275
deviation-center-line_min0.15605124275592083
deviation-heading_max4.584624958749389
deviation-heading_mean3.125669354227097
deviation-heading_median3.321149767962176
deviation-heading_min0.7370302793621415
driven_any_max3.634716228658636
driven_any_mean2.6520353274716
driven_any_median2.9344424926328907
driven_any_min0.4954102473838164
driven_lanedir_consec_max2.7986283702136134
driven_lanedir_consec_mean1.9889044542020593
driven_lanedir_consec_min0.4471313072656349
driven_lanedir_max2.7986283702136134
driven_lanedir_mean2.0431150011563086
driven_lanedir_median2.300489392928398
driven_lanedir_min0.4471313072656349
in-drivable-lane_max4.749999999999983
in-drivable-lane_mean2.5900000000000096
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 3.634716228658636, "sim_physics": 0.19763355096181232, "survival_time": 14.950000000000076, "driven_lanedir": 2.7986283702136134, "sim_render-ego": 0.05810040394465128, "in-drivable-lane": 3.7000000000000295, "agent_compute-ego": 0.08527889410654704, "deviation-heading": 3.321149767962176, "set_robot_commands": 0.10547964334487916, "deviation-center-line": 0.9407381474519764, "driven_lanedir_consec": 2.7986283702136134, "sim_compute_sim_state": 0.04195064067840576, "sim_compute_performance-ego": 0.06334868510564168, "sim_compute_robot_state-ego": 0.06946679751078287, "sim_compute_robot_state-npc0": 0.07472358783086142, "sim_compute_robot_state-npc1": 0.07333938360214233, "sim_compute_robot_state-npc2": 0.07361606121063233, "sim_compute_robot_state-npc3": 0.07305935780207316}, "udem1-1-0": {"driven_any": 2.9344424926328907, "sim_physics": 0.19311234128886257, "survival_time": 14.500000000000073, "driven_lanedir": 2.7820851774150355, "sim_render-ego": 0.05651245610467319, "in-drivable-lane": 0, "agent_compute-ego": 0.0764496869054334, "deviation-heading": 3.991616208207847, "set_robot_commands": 0.10383836976413068, "deviation-center-line": 1.1059561988401976, "driven_lanedir_consec": 2.5213287718164907, "sim_compute_sim_state": 0.03865460609567577, "sim_compute_performance-ego": 0.060993593314598346, "sim_compute_robot_state-ego": 0.06600828335202974, "sim_compute_robot_state-npc0": 0.07355545553667792, "sim_compute_robot_state-npc1": 0.07168858462366565, "sim_compute_robot_state-npc2": 0.07176687388584532, "sim_compute_robot_state-npc3": 0.06926059147407268}, "udem1-2-0": {"driven_any": 0.4954102473838164, "sim_physics": 0.22419349006984543, "survival_time": 2.3, "driven_lanedir": 0.4471313072656349, "sim_render-ego": 0.05485440855440886, "in-drivable-lane": 0, "agent_compute-ego": 0.07640149800673775, "deviation-heading": 0.7370302793621415, "set_robot_commands": 0.1130374929179316, "deviation-center-line": 0.15605124275592083, "driven_lanedir_consec": 0.4471313072656349, "sim_compute_sim_state": 0.042487896007040275, "sim_compute_performance-ego": 0.059000823808752975, "sim_compute_robot_state-ego": 0.06527265776758609, "sim_compute_robot_state-npc0": 0.07388730152793553, "sim_compute_robot_state-npc1": 0.07411272111146347, "sim_compute_robot_state-npc2": 0.07592684289683467, "sim_compute_robot_state-npc3": 0.07557127268418022}, "udem1-3-0": {"driven_any": 3.498923018461184, "sim_physics": 0.2036175235112508, "survival_time": 14.950000000000076, "driven_lanedir": 2.300489392928398, "sim_render-ego": 0.05600693384806315, "in-drivable-lane": 4.749999999999983, "agent_compute-ego": 0.07547996838887533, "deviation-heading": 2.9939255568539314, "set_robot_commands": 0.1015994970003764, "deviation-center-line": 0.8717378124770294, "driven_lanedir_consec": 2.300489392928398, "sim_compute_sim_state": 0.03981265942255656, "sim_compute_performance-ego": 0.061115779876708985, "sim_compute_robot_state-ego": 0.06753332138061524, "sim_compute_robot_state-npc0": 0.07391225099563599, "sim_compute_robot_state-npc1": 0.07432888507843018, "sim_compute_robot_state-npc2": 0.072287384668986, "sim_compute_robot_state-npc3": 0.07162748336791992}, "udem1-4-0": {"driven_any": 2.6966846502214734, "sim_physics": 0.18530344883600872, "survival_time": 14.950000000000076, "driven_lanedir": 1.8872407579588617, "sim_render-ego": 0.0568522842725118, "in-drivable-lane": 4.5000000000000355, "agent_compute-ego": 0.07859689950942993, "deviation-heading": 4.584624958749389, "set_robot_commands": 0.10373901605606078, "deviation-center-line": 0.9720232023562508, "driven_lanedir_consec": 1.87694442878616, "sim_compute_sim_state": 0.03856804370880127, "sim_compute_performance-ego": 0.06129135926564534, "sim_compute_robot_state-ego": 0.06668085813522338, "sim_compute_robot_state-npc0": 0.07244851748148601, "sim_compute_robot_state-npc1": 0.07408414443333944, "sim_compute_robot_state-npc2": 0.07119992653528849, "sim_compute_robot_state-npc3": 0.06995994249979655}}
set_robot_commands_max0.1130374929179316
set_robot_commands_mean0.10553880381667571
set_robot_commands_median0.10383836976413068
set_robot_commands_min0.1015994970003764
sim_compute_performance-ego_max0.06334868510564168
sim_compute_performance-ego_mean0.06115004827426947
sim_compute_performance-ego_median0.061115779876708985
sim_compute_performance-ego_min0.059000823808752975
sim_compute_robot_state-ego_max0.06946679751078287
sim_compute_robot_state-ego_mean0.06699238362924746
sim_compute_robot_state-ego_median0.06668085813522338
sim_compute_robot_state-ego_min0.06527265776758609
sim_compute_robot_state-npc0_max0.07472358783086142
sim_compute_robot_state-npc0_mean0.07370542267451938
sim_compute_robot_state-npc0_median0.07388730152793553
sim_compute_robot_state-npc0_min0.07244851748148601
sim_compute_robot_state-npc1_max0.07432888507843018
sim_compute_robot_state-npc1_mean0.07351074376980822
sim_compute_robot_state-npc1_median0.07408414443333944
sim_compute_robot_state-npc1_min0.07168858462366565
sim_compute_robot_state-npc2_max0.07592684289683467
sim_compute_robot_state-npc2_mean0.07295941783951736
sim_compute_robot_state-npc2_median0.072287384668986
sim_compute_robot_state-npc2_min0.07119992653528849
sim_compute_robot_state-npc3_max0.07557127268418022
sim_compute_robot_state-npc3_mean0.0718957295656085
sim_compute_robot_state-npc3_median0.07162748336791992
sim_compute_robot_state-npc3_min0.06926059147407268
sim_compute_sim_state_max0.042487896007040275
sim_compute_sim_state_mean0.04029476918249592
sim_compute_sim_state_median0.03981265942255656
sim_compute_sim_state_min0.03856804370880127
sim_physics_max0.22419349006984543
sim_physics_mean0.200772070933556
sim_physics_median0.19763355096181232
sim_physics_min0.18530344883600872
sim_render-ego_max0.05810040394465128
sim_render-ego_mean0.056465297344861665
sim_render-ego_median0.05651245610467319
sim_render-ego_min0.05485440855440886
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean12.33000000000006
survival_time_min2.3