AI Driving Olympics Home Challenges Submissions Jobs

Submission 3342

Submission3342
Competingyes
Challengeaido2-LF-sim-validation
UserJBRRussia
Date submitted
Completecomplete
Resulterror
Jobsstep1-simulation: 22391
Next
User labelJetBrains Research
Admin priority50
User priority50

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
223913342JBRRussiaJetBrains Researchaido2-LF-sim-validationstep1-simulationerroryes-0:01:32
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 87, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 441, in <module>
    wrap(cie)
  File "experiment_manager.py", line 429, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 87, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 > 
(hidden)
223903342JBRRussiaJetBrains Researchaido2-LF-sim-validationstep1-simulationsuccessyes-0:35:36(hidden)
driven_lanedir_consec_median2.6061195165241595
survival_time_median14.950000000000076
deviation-center-line_median0.9158554167971322
in-drivable-lane_median0.6499999999999977


other stats
agent_compute-ego_max0.3332826638221741
agent_compute-ego_mean0.3065102955500285
agent_compute-ego_median0.3126683608690898
agent_compute-ego_min0.2709287230173747
deviation-center-line_max1.3900540864280544
deviation-center-line_mean1.0478876261231291
deviation-center-line_min0.8170491003396274
deviation-heading_max3.2814669740520683
deviation-heading_mean2.280953131159643
deviation-heading_median2.235364113960447
deviation-heading_min1.2850574087601645
driven_any_max2.878026120260703
driven_any_mean2.713939627368959
driven_any_median2.667858048351088
driven_any_min2.5107444223692195
driven_lanedir_consec_max2.7234767923157013
driven_lanedir_consec_mean2.5578402399429794
driven_lanedir_consec_min2.26727702033328
driven_lanedir_max2.7234767923157013
driven_lanedir_mean2.5578402399429794
driven_lanedir_median2.6061195165241595
driven_lanedir_min2.26727702033328
in-drivable-lane_max2.150000000000002
in-drivable-lane_mean0.8000000000000034
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.5107444223692195, "sim_physics": 0.15085184176762895, "survival_time": 14.950000000000076, "driven_lanedir": 2.26727702033328, "sim_render-ego": 0.07198540846506754, "in-drivable-lane": 1.200000000000017, "agent_compute-ego": 0.2709287230173747, "deviation-heading": 3.2814669740520683, "set_robot_commands": 0.12256356239318848, "deviation-center-line": 0.8170491003396274, "driven_lanedir_consec": 2.26727702033328, "sim_compute_sim_state": 0.046000825564066566, "sim_compute_performance-ego": 0.07941846927007039, "sim_compute_robot_state-ego": 0.09270612637201946}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.8519477342001416, "sim_physics": 0.15287224610646566, "survival_time": 14.950000000000076, "driven_lanedir": 2.7234767923157013, "sim_render-ego": 0.07266729990641276, "in-drivable-lane": 0.6499999999999977, "agent_compute-ego": 0.3332826638221741, "deviation-heading": 2.139728816373815, "set_robot_commands": 0.12005263408025106, "deviation-center-line": 1.3900540864280544, "driven_lanedir_consec": 2.7234767923157013, "sim_compute_sim_state": 0.0458188247680664, "sim_compute_performance-ego": 0.08290475447972616, "sim_compute_robot_state-ego": 0.0951720635096232}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.878026120260703, "sim_physics": 0.13622482776641845, "survival_time": 14.950000000000076, "driven_lanedir": 2.5775981565066965, "sim_render-ego": 0.07277334849039714, "in-drivable-lane": 2.150000000000002, "agent_compute-ego": 0.3126683608690898, "deviation-heading": 1.2850574087601645, "set_robot_commands": 0.11943259477615356, "deviation-center-line": 1.2397826767779554, "driven_lanedir_consec": 2.5775981565066965, "sim_compute_sim_state": 0.04607876062393188, "sim_compute_performance-ego": 0.08166768789291382, "sim_compute_robot_state-ego": 0.0933917530377706}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.667858048351088, "sim_physics": 0.1460714817047119, "survival_time": 14.950000000000076, "driven_lanedir": 2.61472971403506, "sim_render-ego": 0.07214704672495524, "in-drivable-lane": 0, "agent_compute-ego": 0.2984927678108215, "deviation-heading": 2.4631483426517207, "set_robot_commands": 0.11639736175537108, "deviation-center-line": 0.876696850272876, "driven_lanedir_consec": 2.61472971403506, "sim_compute_sim_state": 0.045822412967681886, "sim_compute_performance-ego": 0.08229336659113566, "sim_compute_robot_state-ego": 0.0901212708155314}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.6611218116636426, "sim_physics": 0.14282263278961185, "survival_time": 14.950000000000076, "driven_lanedir": 2.6061195165241595, "sim_render-ego": 0.07049689054489136, "in-drivable-lane": 0, "agent_compute-ego": 0.31717896223068237, "deviation-heading": 2.235364113960447, "set_robot_commands": 0.1152267034848531, "deviation-center-line": 0.9158554167971322, "driven_lanedir_consec": 2.6061195165241595, "sim_compute_sim_state": 0.046239012082417805, "sim_compute_performance-ego": 0.08189732869466146, "sim_compute_robot_state-ego": 0.08969166278839111}}
set_robot_commands_max0.12256356239318848
set_robot_commands_mean0.11873457129796346
set_robot_commands_median0.11943259477615356
set_robot_commands_min0.1152267034848531
sim_compute_performance-ego_max0.08290475447972616
sim_compute_performance-ego_mean0.0816363213857015
sim_compute_performance-ego_median0.08189732869466146
sim_compute_performance-ego_min0.07941846927007039
sim_compute_robot_state-ego_max0.0951720635096232
sim_compute_robot_state-ego_mean0.09221657530466716
sim_compute_robot_state-ego_median0.09270612637201946
sim_compute_robot_state-ego_min0.08969166278839111
sim_compute_sim_state_max0.046239012082417805
sim_compute_sim_state_mean0.04599196720123291
sim_compute_sim_state_median0.046000825564066566
sim_compute_sim_state_min0.0458188247680664
sim_physics_max0.15287224610646566
sim_physics_mean0.14576860602696734
sim_physics_median0.1460714817047119
sim_physics_min0.13622482776641845
sim_render-ego_max0.07277334849039714
sim_render-ego_mean0.07201399882634481
sim_render-ego_median0.07214704672495524
sim_render-ego_min0.07049689054489136
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
223893342JBRRussiaJetBrains Researchaido2-LF-sim-validationstep1-simulationsuccessyes-0:15:16(hidden)
driven_lanedir_consec_median2.626357918056235
survival_time_median14.950000000000076
deviation-center-line_median1.0728148011450065
in-drivable-lane_median0.6500000000000092


other stats
agent_compute-ego_max0.0612602694829305
agent_compute-ego_mean0.05975389353434245
agent_compute-ego_median0.05939456939697266
agent_compute-ego_min0.05824174006779989
deviation-center-line_max1.2387977241132044
deviation-center-line_mean1.07883478115771
deviation-center-line_min0.9074019310870384
deviation-heading_max3.5365493538765342
deviation-heading_mean2.503925066682135
deviation-heading_median2.164922511360252
deviation-heading_min1.598657744158576
driven_any_max2.8845698127371353
driven_any_mean2.691746455095876
driven_any_median2.6763081483749778
driven_any_min2.519976678028265
driven_lanedir_consec_max2.6889967773293963
driven_lanedir_consec_mean2.5444504663274414
driven_lanedir_consec_min2.3201273244454264
driven_lanedir_max2.6889967773293963
driven_lanedir_mean2.5444504663274414
driven_lanedir_median2.626357918056235
driven_lanedir_min2.3201273244454264
in-drivable-lane_max1.799999999999999
in-drivable-lane_mean0.690000000000001
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.519976678028265, "sim_physics": 0.05123478889465332, "survival_time": 14.950000000000076, "driven_lanedir": 2.3201273244454264, "sim_render-ego": 0.05050426006317139, "in-drivable-lane": 0.6500000000000092, "agent_compute-ego": 0.0612602694829305, "deviation-heading": 3.5365493538765342, "set_robot_commands": 0.07276546716690063, "deviation-center-line": 0.9074019310870384, "driven_lanedir_consec": 2.3201273244454264, "sim_compute_sim_state": 0.03263264815012614, "sim_compute_performance-ego": 0.055164332389831545, "sim_compute_robot_state-ego": 0.055934718449910485}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.6540182547471067, "sim_physics": 0.050925236543019614, "survival_time": 14.950000000000076, "driven_lanedir": 2.447793082869153, "sim_render-ego": 0.050751494566599525, "in-drivable-lane": 0.9999999999999964, "agent_compute-ego": 0.05939456939697266, "deviation-heading": 3.2733460184351064, "set_robot_commands": 0.07315482616424561, "deviation-center-line": 1.1305868697113268, "driven_lanedir_consec": 2.447793082869153, "sim_compute_sim_state": 0.03201790889104207, "sim_compute_performance-ego": 0.05490786552429199, "sim_compute_robot_state-ego": 0.056366393566131594}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.8845698127371353, "sim_physics": 0.051839412053426105, "survival_time": 14.950000000000076, "driven_lanedir": 2.626357918056235, "sim_render-ego": 0.05025802453358968, "in-drivable-lane": 1.799999999999999, "agent_compute-ego": 0.06098735491434733, "deviation-heading": 1.598657744158576, "set_robot_commands": 0.07115639925003052, "deviation-center-line": 1.2387977241132044, "driven_lanedir_consec": 2.626357918056235, "sim_compute_sim_state": 0.033466375668843584, "sim_compute_performance-ego": 0.05450125853220622, "sim_compute_robot_state-ego": 0.055819993019104}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.6763081483749778, "sim_physics": 0.05080472469329834, "survival_time": 14.950000000000076, "driven_lanedir": 2.6389772289369953, "sim_render-ego": 0.04991124947865804, "in-drivable-lane": 0, "agent_compute-ego": 0.05888553380966186, "deviation-heading": 2.164922511360252, "set_robot_commands": 0.07296591599782308, "deviation-center-line": 1.044572579731974, "driven_lanedir_consec": 2.6389772289369953, "sim_compute_sim_state": 0.0333162784576416, "sim_compute_performance-ego": 0.05540862480799357, "sim_compute_robot_state-ego": 0.056234030723571776}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.723859381591895, "sim_physics": 0.05056487480799357, "survival_time": 14.950000000000076, "driven_lanedir": 2.6889967773293963, "sim_render-ego": 0.04971277634302775, "in-drivable-lane": 0, "agent_compute-ego": 0.05824174006779989, "deviation-heading": 1.9461497055802073, "set_robot_commands": 0.07237608432769775, "deviation-center-line": 1.0728148011450065, "driven_lanedir_consec": 2.6889967773293963, "sim_compute_sim_state": 0.032417858441670735, "sim_compute_performance-ego": 0.05409464200337728, "sim_compute_robot_state-ego": 0.05712558348973592}}
set_robot_commands_max0.07315482616424561
set_robot_commands_mean0.07248373858133952
set_robot_commands_median0.07276546716690063
set_robot_commands_min0.07115639925003052
sim_compute_performance-ego_max0.05540862480799357
sim_compute_performance-ego_mean0.05481534465154012
sim_compute_performance-ego_median0.05490786552429199
sim_compute_performance-ego_min0.05409464200337728
sim_compute_robot_state-ego_max0.05712558348973592
sim_compute_robot_state-ego_mean0.05629614384969075
sim_compute_robot_state-ego_median0.056234030723571776
sim_compute_robot_state-ego_min0.055819993019104
sim_compute_sim_state_max0.033466375668843584
sim_compute_sim_state_mean0.03277021392186482
sim_compute_sim_state_median0.03263264815012614
sim_compute_sim_state_min0.03201790889104207
sim_physics_max0.051839412053426105
sim_physics_mean0.05107380739847819
sim_physics_median0.050925236543019614
sim_physics_min0.05056487480799357
sim_render-ego_max0.050751494566599525
sim_render-ego_mean0.05022756099700928
sim_render-ego_median0.05025802453358968
sim_render-ego_min0.04971277634302775
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
218993342JBRRussiaJetBrains Researchaido2-LF-sim-validationstep1-simulationsuccessno-0:15:49(hidden)
driven_lanedir_consec_median2.5904177726870414
survival_time_median14.950000000000076
deviation-center-line_median0.9598279900815052
in-drivable-lane_median0.7999999999999972


other stats
agent_compute-ego_max0.05864317019780477
agent_compute-ego_mean0.05741784985860189
agent_compute-ego_median0.05719944397608439
agent_compute-ego_min0.05658427715301514
deviation-center-line_max1.25665146486598
deviation-center-line_mean1.020439929940885
deviation-center-line_min0.8405036566263417
deviation-heading_max2.91939955261839
deviation-heading_mean2.4565805545580703
deviation-heading_median2.516985603474602
deviation-heading_min2.07164407751621
driven_any_max2.8728962331979986
driven_any_mean2.712983828417254
driven_any_median2.695150877968261
driven_any_min2.5681853380402258
driven_lanedir_consec_max2.649735791051815
driven_lanedir_consec_mean2.528092902848954
driven_lanedir_consec_min2.2258610730257415
driven_lanedir_max2.649735791051815
driven_lanedir_mean2.528092902848954
driven_lanedir_median2.5904177726870414
driven_lanedir_min2.2258610730257415
in-drivable-lane_max2.05
in-drivable-lane_mean0.9300000000000044
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.5681853380402258, "sim_physics": 0.06312215805053711, "survival_time": 14.950000000000076, "driven_lanedir": 2.2258610730257415, "sim_render-ego": 0.039488210678100585, "in-drivable-lane": 1.8000000000000256, "agent_compute-ego": 0.05864317019780477, "deviation-heading": 2.91939955261839, "set_robot_commands": 0.07609338442484538, "deviation-center-line": 0.8405036566263417, "driven_lanedir_consec": 2.2258610730257415, "sim_compute_sim_state": 0.027352038224538168, "sim_compute_performance-ego": 0.04337710301081339, "sim_compute_robot_state-ego": 0.0456618070602417}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.7897444297519547, "sim_physics": 0.06231427669525146, "survival_time": 14.950000000000076, "driven_lanedir": 2.642023982826482, "sim_render-ego": 0.038668870131174725, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.05658427715301514, "deviation-heading": 2.6656775041906005, "set_robot_commands": 0.07591223398844402, "deviation-center-line": 1.25665146486598, "driven_lanedir_consec": 2.642023982826482, "sim_compute_sim_state": 0.027146731217702228, "sim_compute_performance-ego": 0.043126122156778975, "sim_compute_robot_state-ego": 0.045628286997477216}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.8728962331979986, "sim_physics": 0.06337115526199341, "survival_time": 14.950000000000076, "driven_lanedir": 2.5324258946536906, "sim_render-ego": 0.03930384874343872, "in-drivable-lane": 2.05, "agent_compute-ego": 0.05786890029907227, "deviation-heading": 2.1091960349905485, "set_robot_commands": 0.07558017174402873, "deviation-center-line": 1.1774235855961184, "driven_lanedir_consec": 2.5324258946536906, "sim_compute_sim_state": 0.026950047810872395, "sim_compute_performance-ego": 0.043392256895701094, "sim_compute_robot_state-ego": 0.04543720881144206}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.6389422631278294, "sim_physics": 0.06340885957082112, "survival_time": 14.950000000000076, "driven_lanedir": 2.5904177726870414, "sim_render-ego": 0.03913957595825195, "in-drivable-lane": 0, "agent_compute-ego": 0.05679345766703288, "deviation-heading": 2.516985603474602, "set_robot_commands": 0.07748787641525269, "deviation-center-line": 0.9598279900815052, "driven_lanedir_consec": 2.5904177726870414, "sim_compute_sim_state": 0.0271937894821167, "sim_compute_performance-ego": 0.043114014466603595, "sim_compute_robot_state-ego": 0.04553077379862468}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.695150877968261, "sim_physics": 0.06120834668477376, "survival_time": 14.950000000000076, "driven_lanedir": 2.649735791051815, "sim_render-ego": 0.03938365777333577, "in-drivable-lane": 0, "agent_compute-ego": 0.05719944397608439, "deviation-heading": 2.07164407751621, "set_robot_commands": 0.07580571571985881, "deviation-center-line": 0.8677929525344791, "driven_lanedir_consec": 2.649735791051815, "sim_compute_sim_state": 0.027814029852549237, "sim_compute_performance-ego": 0.04345343351364136, "sim_compute_robot_state-ego": 0.04590293010075887}}
set_robot_commands_max0.07748787641525269
set_robot_commands_mean0.07617587645848593
set_robot_commands_median0.07591223398844402
set_robot_commands_min0.07558017174402873
sim_compute_performance-ego_max0.04345343351364136
sim_compute_performance-ego_mean0.04329258600870768
sim_compute_performance-ego_median0.04337710301081339
sim_compute_performance-ego_min0.043114014466603595
sim_compute_robot_state-ego_max0.04590293010075887
sim_compute_robot_state-ego_mean0.0456322013537089
sim_compute_robot_state-ego_median0.045628286997477216
sim_compute_robot_state-ego_min0.04543720881144206
sim_compute_sim_state_max0.027814029852549237
sim_compute_sim_state_mean0.027291327317555743
sim_compute_sim_state_median0.0271937894821167
sim_compute_sim_state_min0.026950047810872395
sim_physics_max0.06340885957082112
sim_physics_mean0.06268495925267538
sim_physics_median0.06312215805053711
sim_physics_min0.06120834668477376
sim_render-ego_max0.039488210678100585
sim_render-ego_mean0.03919683265686035
sim_render-ego_median0.03930384874343872
sim_render-ego_min0.038668870131174725
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076