AI Driving Olympics Home Challenges Submissions Jobs

Submission 3344

Submission3344
Competingyes
Challengeaido2-LFV-sim-validation
UserJBRRussia
Date submitted
Completecomplete
Resultsuccess
Jobsstep1-simulation: 22388
Next
User labelJetBrains Research
Admin priority50
User priority50

22388

Click the images to see detailed statistics about the episode.

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
223883344JBRRussiaJetBrains Researchaido2-LFV-sim-validationstep1-simulationsuccessyes-0:41:37(hidden)
driven_lanedir_consec_median2.5528718052464026
survival_time_median14.950000000000076
deviation-center-line_median1.0013510186572738
in-drivable-lane_median0


other stats
agent_compute-ego_max0.10190905253092448
agent_compute-ego_mean0.09873348135727746
agent_compute-ego_median0.09922749891768406
agent_compute-ego_min0.09561718384424846
deviation-center-line_max1.2083534012513113
deviation-center-line_mean0.9837684505486972
deviation-center-line_min0.7793499477662331
deviation-heading_max2.6934656062975613
deviation-heading_mean2.0715631450970093
deviation-heading_median1.8763653376599545
deviation-heading_min1.8396032412496717
driven_any_max2.846374679535044
driven_any_mean2.615106079673468
driven_any_median2.592767586378671
driven_any_min2.4471188628876672
driven_lanedir_consec_max2.641913136272697
driven_lanedir_consec_mean2.497226719119475
driven_lanedir_consec_min2.335429427736802
driven_lanedir_max2.641913136272697
driven_lanedir_mean2.497226719119475
driven_lanedir_median2.5528718052464026
driven_lanedir_min2.335429427736802
in-drivable-lane_max1.9500000000000275
in-drivable-lane_mean0.6700000000000095
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.519306794706149, "sim_physics": 0.21668468554814657, "survival_time": 14.950000000000076, "driven_lanedir": 2.335429427736802, "sim_render-ego": 0.06614806572596232, "in-drivable-lane": 1.40000000000002, "agent_compute-ego": 0.10057286580403646, "deviation-heading": 2.0920169163572933, "set_robot_commands": 0.12371594587961832, "deviation-center-line": 0.8675375942880417, "driven_lanedir_consec": 2.335429427736802, "sim_compute_sim_state": 0.04312630335489909, "sim_compute_performance-ego": 0.07451350450515747, "sim_compute_robot_state-ego": 0.08328669548034667, "sim_compute_robot_state-npc0": 0.0824186642964681, "sim_compute_robot_state-npc1": 0.07939558823903402, "sim_compute_robot_state-npc2": 0.07484789609909058, "sim_compute_robot_state-npc3": 0.07628479560216268}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.846374679535044, "sim_physics": 0.22288448492685953, "survival_time": 14.950000000000076, "driven_lanedir": 2.5528718052464026, "sim_render-ego": 0.07209946632385254, "in-drivable-lane": 1.9500000000000275, "agent_compute-ego": 0.10190905253092448, "deviation-heading": 1.8396032412496717, "set_robot_commands": 0.13091478983561197, "deviation-center-line": 1.2083534012513113, "driven_lanedir_consec": 2.5528718052464026, "sim_compute_sim_state": 0.04580481290817261, "sim_compute_performance-ego": 0.08058137178421021, "sim_compute_robot_state-ego": 0.09226917346318562, "sim_compute_robot_state-npc0": 0.08640130043029785, "sim_compute_robot_state-npc1": 0.08363185326258342, "sim_compute_robot_state-npc2": 0.08087248961130777, "sim_compute_robot_state-npc3": 0.08132250467936197}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.592767586378671, "sim_physics": 0.21631168762842817, "survival_time": 14.950000000000076, "driven_lanedir": 2.5679472246138033, "sim_render-ego": 0.07132217327753702, "in-drivable-lane": 0, "agent_compute-ego": 0.0963408056894938, "deviation-heading": 1.8563646239205664, "set_robot_commands": 0.12679818630218506, "deviation-center-line": 1.0013510186572738, "driven_lanedir_consec": 2.5679472246138033, "sim_compute_sim_state": 0.04451848109563192, "sim_compute_performance-ego": 0.08086414178212484, "sim_compute_robot_state-ego": 0.08719781398773194, "sim_compute_robot_state-npc0": 0.08691550652186075, "sim_compute_robot_state-npc1": 0.0823401133219401, "sim_compute_robot_state-npc2": 0.07960812012354533, "sim_compute_robot_state-npc3": 0.07937823216120402}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.669962474859809, "sim_physics": 0.20524457772572835, "survival_time": 14.950000000000076, "driven_lanedir": 2.641913136272697, "sim_render-ego": 0.06998920917510987, "in-drivable-lane": 0, "agent_compute-ego": 0.09561718384424846, "deviation-heading": 1.8763653376599545, "set_robot_commands": 0.12379836241404216, "deviation-center-line": 1.0622502907806248, "driven_lanedir_consec": 2.641913136272697, "sim_compute_sim_state": 0.044194118181864424, "sim_compute_performance-ego": 0.07810549815495808, "sim_compute_robot_state-ego": 0.08523449977238973, "sim_compute_robot_state-npc0": 0.0839932107925415, "sim_compute_robot_state-npc1": 0.08120386838912964, "sim_compute_robot_state-npc2": 0.07836428721745808, "sim_compute_robot_state-npc3": 0.07845160563786825}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.4471188628876672, "sim_physics": 0.18992267994985093, "survival_time": 13.70000000000006, "driven_lanedir": 2.38797200172767, "sim_render-ego": 0.06915012415308151, "in-drivable-lane": 0, "agent_compute-ego": 0.09922749891768406, "deviation-heading": 2.6934656062975613, "set_robot_commands": 0.12304565245217652, "deviation-center-line": 0.7793499477662331, "driven_lanedir_consec": 2.38797200172767, "sim_compute_sim_state": 0.0432339523830553, "sim_compute_performance-ego": 0.07635629090079427, "sim_compute_robot_state-ego": 0.08543991172400704, "sim_compute_robot_state-npc0": 0.08324357801980346, "sim_compute_robot_state-npc1": 0.08204155159692694, "sim_compute_robot_state-npc2": 0.07886221112996122, "sim_compute_robot_state-npc3": 0.07927904076819872}}
set_robot_commands_max0.13091478983561197
set_robot_commands_mean0.1256545873767268
set_robot_commands_median0.12379836241404216
set_robot_commands_min0.12304565245217652
sim_compute_performance-ego_max0.08086414178212484
sim_compute_performance-ego_mean0.07808416142544897
sim_compute_performance-ego_median0.07810549815495808
sim_compute_performance-ego_min0.07451350450515747
sim_compute_robot_state-ego_max0.09226917346318562
sim_compute_robot_state-ego_mean0.0866856188855322
sim_compute_robot_state-ego_median0.08543991172400704
sim_compute_robot_state-ego_min0.08328669548034667
sim_compute_robot_state-npc0_max0.08691550652186075
sim_compute_robot_state-npc0_mean0.08459445201219434
sim_compute_robot_state-npc0_median0.0839932107925415
sim_compute_robot_state-npc0_min0.0824186642964681
sim_compute_robot_state-npc1_max0.08363185326258342
sim_compute_robot_state-npc1_mean0.08172259496192283
sim_compute_robot_state-npc1_median0.08204155159692694
sim_compute_robot_state-npc1_min0.07939558823903402
sim_compute_robot_state-npc2_max0.08087248961130777
sim_compute_robot_state-npc2_mean0.0785110008362726
sim_compute_robot_state-npc2_median0.07886221112996122
sim_compute_robot_state-npc2_min0.07484789609909058
sim_compute_robot_state-npc3_max0.08132250467936197
sim_compute_robot_state-npc3_mean0.07894323576975912
sim_compute_robot_state-npc3_median0.07927904076819872
sim_compute_robot_state-npc3_min0.07628479560216268
sim_compute_sim_state_max0.04580481290817261
sim_compute_sim_state_mean0.04417553358472466
sim_compute_sim_state_median0.044194118181864424
sim_compute_sim_state_min0.04312630335489909
sim_physics_max0.22288448492685953
sim_physics_mean0.2102096231558027
sim_physics_median0.21631168762842817
sim_physics_min0.18992267994985093
sim_render-ego_max0.07209946632385254
sim_render-ego_mean0.06974180773110866
sim_render-ego_median0.06998920917510987
sim_render-ego_min0.06614806572596232
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.700000000000074
survival_time_min13.70000000000006
223863344JBRRussiaJetBrains Researchaido2-LFV-sim-validationstep1-simulationerroryes-0:01:21
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 87, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 441, in <module>
    wrap(cie)
  File "experiment_manager.py", line 429, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 87, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 > 
(hidden)
219043344JBRRussiaJetBrains Researchaido2-LFV-sim-validationstep1-simulationsuccessno-0:17:57(hidden)
driven_lanedir_consec_median2.512066858408012
survival_time_median14.950000000000076
deviation-center-line_median1.0425284673387196
in-drivable-lane_median0


other stats
agent_compute-ego_max0.05487339496612549
agent_compute-ego_mean0.05374903228955391
agent_compute-ego_median0.05346897919972737
agent_compute-ego_min0.053398774220393255
deviation-center-line_max1.100356688114419
deviation-center-line_mean0.9272221386231512
deviation-center-line_min0.5264218539247022
deviation-heading_max2.613933276575648
deviation-heading_mean1.988113038694307
deviation-heading_median1.9294836444798416
deviation-heading_min1.3326880547533084
driven_any_max2.7212596585265274
driven_any_mean2.3528185197344627
driven_any_median2.611458948271722
driven_any_min1.1642798532864451
driven_lanedir_consec_max2.6957291349061223
driven_lanedir_consec_mean2.263393288539296
driven_lanedir_consec_min1.133167987145537
driven_lanedir_max2.6957291349061223
driven_lanedir_mean2.263393288539296
driven_lanedir_median2.512066858408012
driven_lanedir_min1.133167987145537
in-drivable-lane_max1.200000000000017
in-drivable-lane_mean0.4400000000000063
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.569130923005739, "sim_physics": 0.09431596676508586, "survival_time": 14.950000000000076, "driven_lanedir": 2.3953165373877434, "sim_render-ego": 0.03710454305013021, "in-drivable-lane": 1.0000000000000142, "agent_compute-ego": 0.05487339496612549, "deviation-heading": 2.1361636471408723, "set_robot_commands": 0.06997842868169149, "deviation-center-line": 0.8860134572974047, "driven_lanedir_consec": 2.3953165373877434, "sim_compute_sim_state": 0.022826600074768066, "sim_compute_performance-ego": 0.03964339335759481, "sim_compute_robot_state-ego": 0.04187292098999024, "sim_compute_robot_state-npc0": 0.04217913389205932, "sim_compute_robot_state-npc1": 0.040383372306823734, "sim_compute_robot_state-npc2": 0.04014780124028524, "sim_compute_robot_state-npc3": 0.04004256089528402}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.1642798532864451, "sim_physics": 0.0976025911477896, "survival_time": 6.499999999999985, "driven_lanedir": 1.133167987145537, "sim_render-ego": 0.03636988859910231, "in-drivable-lane": 0, "agent_compute-ego": 0.053398774220393255, "deviation-heading": 1.3326880547533084, "set_robot_commands": 0.06887165216299204, "deviation-center-line": 0.5264218539247022, "driven_lanedir_consec": 1.133167987145537, "sim_compute_sim_state": 0.022785551731403057, "sim_compute_performance-ego": 0.039153381494375375, "sim_compute_robot_state-ego": 0.04206007994138278, "sim_compute_robot_state-npc0": 0.04174444308647743, "sim_compute_robot_state-npc1": 0.03997483620276818, "sim_compute_robot_state-npc2": 0.040056127768296465, "sim_compute_robot_state-npc3": 0.039967852372389576}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.7212596585265274, "sim_physics": 0.09341395616531371, "survival_time": 14.950000000000076, "driven_lanedir": 2.6957291349061223, "sim_render-ego": 0.036442792415618895, "in-drivable-lane": 0, "agent_compute-ego": 0.05346897919972737, "deviation-heading": 1.9294836444798416, "set_robot_commands": 0.06823683818181356, "deviation-center-line": 1.100356688114419, "driven_lanedir_consec": 2.6957291349061223, "sim_compute_sim_state": 0.022943239212036132, "sim_compute_performance-ego": 0.03954721768697103, "sim_compute_robot_state-ego": 0.04175877412160238, "sim_compute_robot_state-npc0": 0.041948161125183105, "sim_compute_robot_state-npc1": 0.04007127602895101, "sim_compute_robot_state-npc2": 0.04001702785491943, "sim_compute_robot_state-npc3": 0.03975885470708211}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.611458948271722, "sim_physics": 0.09193161487579346, "survival_time": 14.950000000000076, "driven_lanedir": 2.580685924849065, "sim_render-ego": 0.03635475794474284, "in-drivable-lane": 0, "agent_compute-ego": 0.05345877170562744, "deviation-heading": 1.9282965705218669, "set_robot_commands": 0.06865007003148396, "deviation-center-line": 1.0807902264405114, "driven_lanedir_consec": 2.580685924849065, "sim_compute_sim_state": 0.02285640239715576, "sim_compute_performance-ego": 0.03964262406031291, "sim_compute_robot_state-ego": 0.04188123861948649, "sim_compute_robot_state-npc0": 0.04193736076354981, "sim_compute_robot_state-npc1": 0.04009686549504598, "sim_compute_robot_state-npc2": 0.0400542140007019, "sim_compute_robot_state-npc3": 0.03980696598688761}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.6979632155818796, "sim_physics": 0.08757522503534952, "survival_time": 14.950000000000076, "driven_lanedir": 2.512066858408012, "sim_render-ego": 0.03626610517501831, "in-drivable-lane": 1.200000000000017, "agent_compute-ego": 0.053545241355895994, "deviation-heading": 2.613933276575648, "set_robot_commands": 0.07099072933197022, "deviation-center-line": 1.0425284673387196, "driven_lanedir_consec": 2.512066858408012, "sim_compute_sim_state": 0.022754673957824708, "sim_compute_performance-ego": 0.039507656892140704, "sim_compute_robot_state-ego": 0.04163734118143717, "sim_compute_robot_state-npc0": 0.0417992385228475, "sim_compute_robot_state-npc1": 0.04024520715077718, "sim_compute_robot_state-npc2": 0.04007348457972209, "sim_compute_robot_state-npc3": 0.03984867095947266}}
set_robot_commands_max0.07099072933197022
set_robot_commands_mean0.06934554367799026
set_robot_commands_median0.06887165216299204
set_robot_commands_min0.06823683818181356
sim_compute_performance-ego_max0.03964339335759481
sim_compute_performance-ego_mean0.03949885469827896
sim_compute_performance-ego_median0.03954721768697103
sim_compute_performance-ego_min0.039153381494375375
sim_compute_robot_state-ego_max0.04206007994138278
sim_compute_robot_state-ego_mean0.04184207097077981
sim_compute_robot_state-ego_median0.04187292098999024
sim_compute_robot_state-ego_min0.04163734118143717
sim_compute_robot_state-npc0_max0.04217913389205932
sim_compute_robot_state-npc0_mean0.04192166747802344
sim_compute_robot_state-npc0_median0.04193736076354981
sim_compute_robot_state-npc0_min0.04174444308647743
sim_compute_robot_state-npc1_max0.040383372306823734
sim_compute_robot_state-npc1_mean0.040154311436873216
sim_compute_robot_state-npc1_median0.04009686549504598
sim_compute_robot_state-npc1_min0.03997483620276818
sim_compute_robot_state-npc2_max0.04014780124028524
sim_compute_robot_state-npc2_mean0.04006973108878502
sim_compute_robot_state-npc2_median0.040056127768296465
sim_compute_robot_state-npc2_min0.04001702785491943
sim_compute_robot_state-npc3_max0.04004256089528402
sim_compute_robot_state-npc3_mean0.0398849809842232
sim_compute_robot_state-npc3_median0.03984867095947266
sim_compute_robot_state-npc3_min0.03975885470708211
sim_compute_sim_state_max0.022943239212036132
sim_compute_sim_state_mean0.022833293474637546
sim_compute_sim_state_median0.022826600074768066
sim_compute_sim_state_min0.022754673957824708
sim_physics_max0.0976025911477896
sim_physics_mean0.09296787079786643
sim_physics_median0.09341395616531371
sim_physics_min0.08757522503534952
sim_render-ego_max0.03710454305013021
sim_render-ego_mean0.03650761743692251
sim_render-ego_median0.03636988859910231
sim_render-ego_min0.03626610517501831
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean13.260000000000058
survival_time_min6.499999999999985
218983344JBRRussiaJetBrains Researchaido2-LFV-sim-validationstep1-simulationhost-errorno-0:25:15
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 488, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 347, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in convert
    with open(fn, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-validation/submission3344/step1-simulation-ip-172-31-42-167-7357-job21898/logs/challenges-runner/stdout.html'
(hidden)