Duckietown Challenges Home Challenges Submissions

Evaluator 1581

ID1581
evaluatorip-172-31-46-148-10491
ownerI don't have one 😀
machineip-172-31-46-148
processip-172-31-46-148-10491
versiond-c:5.0.7;d-c-r:5.0.10;d-s:5.0.3
first heard
last heard
statusinactive
# evaluating
# success29 24458
# timeout
# failed8 24411
# error11 24396
# aborted
# host-error4 24635
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
251254393Jeick Hincapie Barrera 🇨🇴challenge-aido_LF-template-ros - Template solution using ROSaido3-LFV-sim-validationstep1-simulationsuccessnoip-172-31-46-148-104910:08:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.36592094708219225
survival_time_median2.5999999999999988
deviation-center-line_median0.16987818035809085
in-drivable-lane_median0.05000000000000005


other stats
agent_compute-ego_max0.023931377677507297
agent_compute-ego_mean0.022950370113858403
agent_compute-ego_median0.0229877619676187
agent_compute-ego_min0.02194032214936756
deviation-center-line_max0.2294363663374862
deviation-center-line_mean0.14972083567701736
deviation-center-line_min0.06426934298501713
deviation-heading_max0.6986268167330367
deviation-heading_mean0.4280304166285302
deviation-heading_median0.41847631252347706
deviation-heading_min0.2596285487829773
driven_any_max1.2972979671834337
driven_any_mean0.7173901327940223
driven_any_median0.7336463183586179
driven_any_min0.26903550091980144
driven_lanedir_consec_max0.97716132702229
driven_lanedir_consec_mean0.5227475277218055
driven_lanedir_consec_min0.2407721645627201
driven_lanedir_max0.97716132702229
driven_lanedir_mean0.5227475277218055
driven_lanedir_median0.36592094708219225
driven_lanedir_min0.2407721645627201
in-drivable-lane_max1.8999999999999944
in-drivable-lane_mean0.5799999999999983
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9226822529128632, "sim_physics": 0.14627440546599912, "survival_time": 3.5499999999999954, "driven_lanedir": 0.36592094708219225, "sim_render-ego": 0.01236469309094926, "in-drivable-lane": 1.8999999999999944, "agent_compute-ego": 0.0229877619676187, "deviation-heading": 0.2596285487829773, "set_robot_commands": 0.00896929015576, "deviation-center-line": 0.16987818035809085, "driven_lanedir_consec": 0.36592094708219225, "sim_compute_sim_state": 0.004753596346143266, "sim_compute_performance-ego": 0.007200136990614341, "sim_compute_robot_state-ego": 0.009162859177925217, "sim_compute_robot_state-npc0": 0.007945974108198998, "sim_compute_robot_state-npc1": 0.007794168633474431, "sim_compute_robot_state-npc2": 0.007718777992355991, "sim_compute_robot_state-npc3": 0.007370572694590394}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.3642886245953953, "sim_physics": 0.17665474116802216, "survival_time": 1.6000000000000008, "driven_lanedir": 0.3026144243018116, "sim_render-ego": 0.01234499365091324, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.02299170196056366, "deviation-heading": 0.6986268167330367, "set_robot_commands": 0.009348683059215546, "deviation-center-line": 0.10463456752710912, "driven_lanedir_consec": 0.3026144243018116, "sim_compute_sim_state": 0.0047586336731910706, "sim_compute_performance-ego": 0.006771288812160492, "sim_compute_robot_state-ego": 0.008488275110721588, "sim_compute_robot_state-npc0": 0.007684774696826935, "sim_compute_robot_state-npc1": 0.007822483777999878, "sim_compute_robot_state-npc2": 0.00815165787935257, "sim_compute_robot_state-npc3": 0.008060827851295471}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.7336463183586179, "sim_physics": 0.17846371118838972, "survival_time": 2.5999999999999988, "driven_lanedir": 0.727268775640014, "sim_render-ego": 0.012294952686016375, "in-drivable-lane": 0, "agent_compute-ego": 0.022900686814234808, "deviation-heading": 0.27230767118055227, "set_robot_commands": 0.00826191443663377, "deviation-center-line": 0.2294363663374862, "driven_lanedir_consec": 0.727268775640014, "sim_compute_sim_state": 0.004943682597233699, "sim_compute_performance-ego": 0.007248497926271879, "sim_compute_robot_state-ego": 0.0087622908445505, "sim_compute_robot_state-npc0": 0.00818321338066688, "sim_compute_robot_state-npc1": 0.007910059048579289, "sim_compute_robot_state-npc2": 0.007841183589054989, "sim_compute_robot_state-npc3": 0.00789413543847891}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.2972979671834337, "sim_physics": 0.1700707943208756, "survival_time": 4.6499999999999915, "driven_lanedir": 0.97716132702229, "sim_render-ego": 0.011817039981965096, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.023931377677507297, "deviation-heading": 0.49111273392260785, "set_robot_commands": 0.009269470809608375, "deviation-center-line": 0.18038572117738352, "driven_lanedir_consec": 0.97716132702229, "sim_compute_sim_state": 0.004819095775645267, "sim_compute_performance-ego": 0.007002358795494162, "sim_compute_robot_state-ego": 0.009096612212478473, "sim_compute_robot_state-npc0": 0.008714986103837209, "sim_compute_robot_state-npc1": 0.008282312782861853, "sim_compute_robot_state-npc2": 0.008022177603936965, "sim_compute_robot_state-npc3": 0.008022780059486307}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.26903550091980144, "sim_physics": 0.15963204701741537, "survival_time": 1.0500000000000005, "driven_lanedir": 0.2407721645627201, "sim_render-ego": 0.01218980834597633, "in-drivable-lane": 0, "agent_compute-ego": 0.02194032214936756, "deviation-heading": 0.41847631252347706, "set_robot_commands": 0.008326144445510138, "deviation-center-line": 0.06426934298501713, "driven_lanedir_consec": 0.2407721645627201, "sim_compute_sim_state": 0.004236005601428804, "sim_compute_performance-ego": 0.007335310890561058, "sim_compute_robot_state-ego": 0.009826705569312684, "sim_compute_robot_state-npc0": 0.007518972669328962, "sim_compute_robot_state-npc1": 0.006844986052740188, "sim_compute_robot_state-npc2": 0.006479740142822266, "sim_compute_robot_state-npc3": 0.007009120214553107}}
set_robot_commands_max0.009348683059215546
set_robot_commands_mean0.008835100581345566
set_robot_commands_median0.00896929015576
set_robot_commands_min0.00826191443663377
sim_compute_performance-ego_max0.007335310890561058
sim_compute_performance-ego_mean0.007111518683020386
sim_compute_performance-ego_median0.007200136990614341
sim_compute_performance-ego_min0.006771288812160492
sim_compute_robot_state-ego_max0.009826705569312684
sim_compute_robot_state-ego_mean0.009067348582997691
sim_compute_robot_state-ego_median0.009096612212478473
sim_compute_robot_state-ego_min0.008488275110721588
sim_compute_robot_state-npc0_max0.008714986103837209
sim_compute_robot_state-npc0_mean0.008009584191771797
sim_compute_robot_state-npc0_median0.007945974108198998
sim_compute_robot_state-npc0_min0.007518972669328962
sim_compute_robot_state-npc1_max0.008282312782861853
sim_compute_robot_state-npc1_mean0.007730802059131128
sim_compute_robot_state-npc1_median0.007822483777999878
sim_compute_robot_state-npc1_min0.006844986052740188
sim_compute_robot_state-npc2_max0.00815165787935257
sim_compute_robot_state-npc2_mean0.007642707441504556
sim_compute_robot_state-npc2_median0.007841183589054989
sim_compute_robot_state-npc2_min0.006479740142822266
sim_compute_robot_state-npc3_max0.008060827851295471
sim_compute_robot_state-npc3_mean0.007671487251680838
sim_compute_robot_state-npc3_median0.00789413543847891
sim_compute_robot_state-npc3_min0.007009120214553107
sim_compute_sim_state_max0.004943682597233699
sim_compute_sim_state_mean0.004702202798728421
sim_compute_sim_state_median0.0047586336731910706
sim_compute_sim_state_min0.004236005601428804
sim_physics_max0.17846371118838972
sim_physics_mean0.16621913983214037
sim_physics_median0.1700707943208756
sim_physics_min0.14627440546599912
sim_render-ego_max0.01236469309094926
sim_render-ego_mean0.01220229755116406
sim_render-ego_median0.012294952686016375
sim_render-ego_min0.011817039981965096
simulation-passed1
survival_time_max4.6499999999999915
survival_time_mean2.6899999999999973
survival_time_min1.0500000000000005
No reset possible
251104388Soroush Saryazdi 🇨🇦bluejayaido3-LF-sim-validationstep1-simulationfailednoip-172-31-46-148-104910:06:07
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 296, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 239, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 123 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
250974385Soroush Saryazdi 🇨🇦bluejayaido3-LFVI-sim-validationstep1-simulationfailednoip-172-31-46-148-104910:10:14
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 296, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 239, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 368 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
250914376Frank (Chude) Qian 🇨🇦Frank Raw Test 2aido3-LF-sim-testingstep1-simulationsuccessnoip-172-31-46-148-104910:11:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.7372197859840972
survival_time_median4.099999999999993
deviation-center-line_median0.20422783425757424
in-drivable-lane_median0.8999999999999972


other stats
agent_compute-ego_max0.01836734584399632
agent_compute-ego_mean0.016978791423037258
agent_compute-ego_median0.016908059637230563
agent_compute-ego_min0.016391876267223823
deviation-center-line_max0.3778692654759731
deviation-center-line_mean0.18235758106064148
deviation-center-line_min0.0687052670084276
deviation-heading_max1.8111725662056624
deviation-heading_mean0.6599658838859124
deviation-heading_median0.542408610925007
deviation-heading_min0.271545745880084
driven_any_max3.7435516018492407
driven_any_mean1.5135830326385669
driven_any_median1.6666484997994206
driven_any_min0.470129994922083
driven_lanedir_consec_max2.0029477347977385
driven_lanedir_consec_mean0.9902974129946204
driven_lanedir_consec_min0.44676916846206893
driven_lanedir_max2.0029477347977385
driven_lanedir_mean0.9902974129946204
driven_lanedir_median0.7372197859840972
driven_lanedir_min0.44676916846206893
in-drivable-lane_max4.4499999999999895
in-drivable-lane_mean1.08333333333333
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0936509593633597, "sim_physics": 0.05620239036423819, "survival_time": 2.799999999999998, "driven_lanedir": 0.6784572508682953, "sim_render-ego": 0.00732941712651934, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.01836734584399632, "deviation-heading": 0.542408610925007, "set_robot_commands": 0.005250492266246251, "deviation-center-line": 0.10196172347969104, "driven_lanedir_consec": 0.6784572508682953, "sim_compute_sim_state": 0.0030304363795689176, "sim_compute_performance-ego": 0.004311859607696533, "sim_compute_robot_state-ego": 0.005297669342585972}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.49872139620633654, "sim_physics": 0.05817729029162177, "survival_time": 1.4500000000000006, "driven_lanedir": 0.477992831080523, "sim_render-ego": 0.007251862821907833, "in-drivable-lane": 0, "agent_compute-ego": 0.017003125157849543, "deviation-heading": 0.33140928716742846, "set_robot_commands": 0.00500360028497104, "deviation-center-line": 0.0687052670084276, "driven_lanedir_consec": 0.477992831080523, "sim_compute_sim_state": 0.0028739057738205484, "sim_compute_performance-ego": 0.003922034954202586, "sim_compute_robot_state-ego": 0.005155941535686624}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.7373587267508206, "sim_physics": 0.053107860476471654, "survival_time": 4.299999999999993, "driven_lanedir": 1.1998413068510208, "sim_render-ego": 0.00701614590578301, "in-drivable-lane": 1.2499999999999956, "agent_compute-ego": 0.01668145213016244, "deviation-heading": 0.6095443331131409, "set_robot_commands": 0.004927269248075263, "deviation-center-line": 0.20422783425757424, "driven_lanedir_consec": 1.1998413068510208, "sim_compute_sim_state": 0.0027932261311730674, "sim_compute_performance-ego": 0.0039048887962518735, "sim_compute_robot_state-ego": 0.004965574242347895}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 3.7435516018492407, "sim_physics": 0.053440118103884576, "survival_time": 8.899999999999991, "driven_lanedir": 1.6300732578005124, "sim_render-ego": 0.007044883256547906, "in-drivable-lane": 4.4499999999999895, "agent_compute-ego": 0.016904079512263952, "deviation-heading": 1.8111725662056624, "set_robot_commands": 0.00499488530534037, "deviation-center-line": 0.32185623243243555, "driven_lanedir_consec": 1.6300732578005124, "sim_compute_sim_state": 0.0028580762027354723, "sim_compute_performance-ego": 0.003937228342120567, "sim_compute_robot_state-ego": 0.004990952738215414}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.6881365170486962, "sim_physics": 0.053757548332214355, "survival_time": 4.099999999999993, "driven_lanedir": 1.1970605993672958, "sim_render-ego": 0.007046277930096882, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.016391876267223823, "deviation-heading": 0.6931358754651502, "set_robot_commands": 0.007440936274644806, "deviation-center-line": 0.20578839442927097, "driven_lanedir_consec": 1.1970605993672958, "sim_compute_sim_state": 0.002782577421607041, "sim_compute_performance-ego": 0.0038880400541352065, "sim_compute_robot_state-ego": 0.004836634891789134}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.470129994922083, "sim_physics": 0.05712769712720599, "survival_time": 1.4000000000000006, "driven_lanedir": 0.44676916846206893, "sim_render-ego": 0.00727980477469308, "in-drivable-lane": 0, "agent_compute-ego": 0.01700129679271153, "deviation-heading": 0.43455948443566017, "set_robot_commands": 0.00515756436756679, "deviation-center-line": 0.1084326701635877, "driven_lanedir_consec": 0.44676916846206893, "sim_compute_sim_state": 0.002929295812334333, "sim_compute_performance-ego": 0.0039460403578622, "sim_compute_robot_state-ego": 0.005060791969299316}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.059155573138954, "sim_physics": 0.05475655469027432, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5060214906173806, "sim_render-ego": 0.007054268230091441, "in-drivable-lane": 1.1999999999999975, "agent_compute-ego": 0.01654305024580522, "deviation-heading": 0.4102196248382458, "set_robot_commands": 0.004893905466253107, "deviation-center-line": 0.08610092632466662, "driven_lanedir_consec": 0.5060214906173806, "sim_compute_sim_state": 0.002762950550426136, "sim_compute_performance-ego": 0.003907416083595969, "sim_compute_robot_state-ego": 0.004832883314652876}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 1.0136692334104231, "sim_physics": 0.05373215216856737, "survival_time": 2.5999999999999988, "driven_lanedir": 0.7372197859840972, "sim_render-ego": 0.0071084040861863354, "in-drivable-lane": 0.6499999999999977, "agent_compute-ego": 0.016502394126011774, "deviation-heading": 0.271545745880084, "set_robot_commands": 0.004941325921278734, "deviation-center-line": 0.20683802061811252, "driven_lanedir_consec": 0.7372197859840972, "sim_compute_sim_state": 0.002782959204453689, "sim_compute_performance-ego": 0.00394667570407574, "sim_compute_robot_state-ego": 0.004912458933316744}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.6756863990446962, "sim_physics": 0.055426225429627954, "survival_time": 4.099999999999993, "driven_lanedir": 1.5065067943502224, "sim_render-ego": 0.007054808663158882, "in-drivable-lane": 0.2499999999999991, "agent_compute-ego": 0.016831409640428497, "deviation-heading": 0.9007737649801568, "set_robot_commands": 0.004965796703245581, "deviation-center-line": 0.26742489653161494, "driven_lanedir_consec": 1.5065067943502224, "sim_compute_sim_state": 0.0028085476014672256, "sim_compute_performance-ego": 0.003918985041176401, "sim_compute_robot_state-ego": 0.004955893609581924}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.6666484997994206, "sim_physics": 0.05491946404238781, "survival_time": 4.149999999999993, "driven_lanedir": 0.6379384771652008, "sim_render-ego": 0.007053271833672581, "in-drivable-lane": 2.3499999999999925, "agent_compute-ego": 0.016908059637230563, "deviation-heading": 0.3415117151705017, "set_robot_commands": 0.004920551575810076, "deviation-center-line": 0.08491723223482874, "driven_lanedir_consec": 0.6379384771652008, "sim_compute_sim_state": 0.002793352287935923, "sim_compute_performance-ego": 0.0038998701486242823, "sim_compute_robot_state-ego": 0.004862173494086208}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 2.2372326784177967, "sim_physics": 0.05526683263689558, "survival_time": 5.349999999999989, "driven_lanedir": 1.2962216181147637, "sim_render-ego": 0.007013929224459924, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.016815225654673353, "deviation-heading": 1.0660428397056596, "set_robot_commands": 0.004880508529805691, "deviation-center-line": 0.3067057439903003, "driven_lanedir_consec": 1.2962216181147637, "sim_compute_sim_state": 0.0027825564981621004, "sim_compute_performance-ego": 0.003919048844096816, "sim_compute_robot_state-ego": 0.004898102484016775}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.078306767438422, "sim_physics": 0.05583717606284402, "survival_time": 2.7499999999999982, "driven_lanedir": 0.6366213565033307, "sim_render-ego": 0.007153315977616744, "in-drivable-lane": 0.8999999999999972, "agent_compute-ego": 0.017032081430608578, "deviation-heading": 0.4548706040821874, "set_robot_commands": 0.004804385792125355, "deviation-center-line": 0.08789843946613897, "driven_lanedir_consec": 0.6366213565033307, "sim_compute_sim_state": 0.0028015223416415127, "sim_compute_performance-ego": 0.003938146071000533, "sim_compute_robot_state-ego": 0.004958659952337092}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.49396053113098926, "sim_physics": 0.056803900620033, "survival_time": 1.4500000000000006, "driven_lanedir": 0.47357664783136344, "sim_render-ego": 0.007230988864240976, "in-drivable-lane": 0, "agent_compute-ego": 0.017283710940130825, "deviation-heading": 0.32253358773908075, "set_robot_commands": 0.004872527615777378, "deviation-center-line": 0.08769655696212493, "driven_lanedir_consec": 0.47357664783136344, "sim_compute_sim_state": 0.0029784893167429955, "sim_compute_performance-ego": 0.003970590131036167, "sim_compute_robot_state-ego": 0.0051725075162690265}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 2.005093500891576, "sim_physics": 0.053014792894062246, "survival_time": 4.749999999999991, "driven_lanedir": 1.4272128751254942, "sim_render-ego": 0.007092340368973582, "in-drivable-lane": 1.149999999999996, "agent_compute-ego": 0.016938513203671103, "deviation-heading": 0.9752233733893584, "set_robot_commands": 0.004986685200741417, "deviation-center-line": 0.2189405125348749, "driven_lanedir_consec": 1.4272128751254942, "sim_compute_sim_state": 0.0028952498185007194, "sim_compute_performance-ego": 0.003951815554970189, "sim_compute_robot_state-ego": 0.0050407033217580695}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.242443110165688, "sim_physics": 0.05902733154667234, "survival_time": 5.14999999999999, "driven_lanedir": 2.0029477347977385, "sim_render-ego": 0.00719743099027467, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.01747825076279131, "deviation-heading": 0.734536845191363, "set_robot_commands": 0.005015148699862286, "deviation-center-line": 0.3778692654759731, "driven_lanedir_consec": 2.0029477347977385, "sim_compute_sim_state": 0.0029700936622989987, "sim_compute_performance-ego": 0.003939908685036076, "sim_compute_robot_state-ego": 0.005179222347666916}}
set_robot_commands_max0.007440936274644806
set_robot_commands_mean0.0051370388834496095
set_robot_commands_median0.004965796703245581
set_robot_commands_min0.004804385792125355
sim_compute_performance-ego_max0.004311859607696533
sim_compute_performance-ego_mean0.003953503225058743
sim_compute_performance-ego_median0.003937228342120567
sim_compute_performance-ego_min0.0038880400541352065
sim_compute_robot_state-ego_max0.005297669342585972
sim_compute_robot_state-ego_mean0.005008011312907332
sim_compute_robot_state-ego_median0.004965574242347895
sim_compute_robot_state-ego_min0.004832883314652876
sim_compute_sim_state_max0.0030304363795689176
sim_compute_sim_state_mean0.002856215933524579
sim_compute_sim_state_median0.0028085476014672256
sim_compute_sim_state_min0.002762950550426136
sim_physics_max0.05902733154667234
sim_physics_mean0.05537315565246674
sim_physics_median0.05526683263689558
sim_physics_min0.053014792894062246
sim_render-ego_max0.00732941712651934
sim_render-ego_mean0.0071284766702815455
sim_render-ego_median0.007092340368973582
sim_render-ego_min0.007013929224459924
simulation-passed1
survival_time_max8.899999999999991
survival_time_mean3.733333333333329
survival_time_min1.4000000000000006
No reset possible
250894374Frank (Chude) Qian 🇨🇦Frank raw run 1aido3-LF-sim-testingstep1-simulationsuccessnoip-172-31-46-148-104910:15:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.23392617595498064
survival_time_median6.449999999999985
deviation-center-line_median0.3400930915781064
in-drivable-lane_median1.7500000000000062


other stats
agent_compute-ego_max0.01741458387935863
agent_compute-ego_mean0.016018792766258515
agent_compute-ego_median0.015922630986859723
agent_compute-ego_min0.015477780942563656
deviation-center-line_max0.939712292626745
deviation-center-line_mean0.4554419650555461
deviation-center-line_min0.17900228372028948
deviation-heading_max10.279209070179563
deviation-heading_mean4.351494432555884
deviation-heading_median2.7171675651657963
deviation-heading_min1.1046669941235976
driven_any_max2.7870195235428024
driven_any_mean1.2517925667787333
driven_any_median0.983857067240226
driven_any_min0.43033908408394145
driven_lanedir_consec_max0.7032190955787656
driven_lanedir_consec_mean0.3275679137666734
driven_lanedir_consec_min0.08609262859199884
driven_lanedir_max1.828068484382404
driven_lanedir_mean0.6686404394858584
driven_lanedir_median0.3196336839974814
driven_lanedir_min0.20750988057552444
in-drivable-lane_max2.9499999999999913
in-drivable-lane_mean1.483333333333335
in-drivable-lane_min0.2999999999999998
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.4605315548618175, "sim_physics": 0.05707816984139237, "survival_time": 2.549999999999999, "driven_lanedir": 0.21866172897046487, "sim_render-ego": 0.015337387720743816, "in-drivable-lane": 0.7500000000000007, "agent_compute-ego": 0.01741458387935863, "deviation-heading": 1.1046669941235976, "set_robot_commands": 0.0049811064028272446, "deviation-center-line": 0.17900228372028948, "driven_lanedir_consec": 0.1513009826391798, "sim_compute_sim_state": 0.002847021701289158, "sim_compute_performance-ego": 0.0042560661540311925, "sim_compute_robot_state-ego": 0.0050239656485763255}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.5146520310975962, "sim_physics": 0.0555783805026803, "survival_time": 4.6499999999999915, "driven_lanedir": 0.31054081687641477, "sim_render-ego": 0.006981188251126197, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.01604471924484417, "deviation-heading": 2.662988402879346, "set_robot_commands": 0.004856294201266381, "deviation-center-line": 0.32084746500277245, "driven_lanedir_consec": 0.15471399983560974, "sim_compute_sim_state": 0.0027539601889989707, "sim_compute_performance-ego": 0.003874550583542034, "sim_compute_robot_state-ego": 0.004866059108447003}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.304745373802772, "sim_physics": 0.05534216515103677, "survival_time": 6.649999999999984, "driven_lanedir": 0.7603155036427349, "sim_render-ego": 0.006877060223342781, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.015765657998565445, "deviation-heading": 2.7171675651657963, "set_robot_commands": 0.004814918776203815, "deviation-center-line": 0.3400930915781064, "driven_lanedir_consec": 0.40500960350149473, "sim_compute_sim_state": 0.002810044396192507, "sim_compute_performance-ego": 0.003871693646997437, "sim_compute_robot_state-ego": 0.004761461028479096}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.5035930985868354, "sim_physics": 0.056923041578199045, "survival_time": 3.049999999999997, "driven_lanedir": 0.257392584773219, "sim_render-ego": 0.007114711354990475, "in-drivable-lane": 0.3499999999999994, "agent_compute-ego": 0.016119484041557938, "deviation-heading": 1.8219139321684537, "set_robot_commands": 0.005018730632594374, "deviation-center-line": 0.20234869289459936, "driven_lanedir_consec": 0.1790905663915794, "sim_compute_sim_state": 0.00288322714508557, "sim_compute_performance-ego": 0.003941219361101995, "sim_compute_robot_state-ego": 0.005009264242453653}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.9038799878915832, "sim_physics": 0.0541647473971049, "survival_time": 5.399999999999989, "driven_lanedir": 0.2825540563889284, "sim_render-ego": 0.006882510803363941, "in-drivable-lane": 2.6499999999999924, "agent_compute-ego": 0.015477780942563656, "deviation-heading": 2.00483306495742, "set_robot_commands": 0.004910140125839798, "deviation-center-line": 0.21073213404502025, "driven_lanedir_consec": 0.1252781342824525, "sim_compute_sim_state": 0.0027497763986940736, "sim_compute_performance-ego": 0.003881717169726337, "sim_compute_robot_state-ego": 0.004715723020059091}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.983857067240226, "sim_physics": 0.05485767911570941, "survival_time": 6.449999999999985, "driven_lanedir": 0.3196336839974814, "sim_render-ego": 0.0070249923439913015, "in-drivable-lane": 2.9499999999999913, "agent_compute-ego": 0.016273809033770893, "deviation-heading": 2.3721501969947525, "set_robot_commands": 0.004937717156816823, "deviation-center-line": 0.3489991852357689, "driven_lanedir_consec": 0.1207043515434938, "sim_compute_sim_state": 0.002818697182707084, "sim_compute_performance-ego": 0.003918113634567852, "sim_compute_robot_state-ego": 0.004890512126360753}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.6242083302524022, "sim_physics": 0.05299765521829779, "survival_time": 11.00000000000002, "driven_lanedir": 0.9214322467586236, "sim_render-ego": 0.00690000057220459, "in-drivable-lane": 1.8000000000000045, "agent_compute-ego": 0.016009434786709874, "deviation-heading": 6.1471820154387, "set_robot_commands": 0.004808455163782293, "deviation-center-line": 0.6965470786366902, "driven_lanedir_consec": 0.5923748470967549, "sim_compute_sim_state": 0.002755752476778898, "sim_compute_performance-ego": 0.0038825392723083494, "sim_compute_robot_state-ego": 0.004783291166478938}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 1.240622040870624, "sim_physics": 0.0533136967066172, "survival_time": 7.399999999999982, "driven_lanedir": 0.5686350999779588, "sim_render-ego": 0.006873293502910717, "in-drivable-lane": 2.399999999999993, "agent_compute-ego": 0.015620358892389248, "deviation-heading": 3.3173364260303573, "set_robot_commands": 0.004833097393448288, "deviation-center-line": 0.29268108065340726, "driven_lanedir_consec": 0.23392617595498064, "sim_compute_sim_state": 0.002779498293593123, "sim_compute_performance-ego": 0.0038897781758695034, "sim_compute_robot_state-ego": 0.00476501600162403}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 2.692271286185296, "sim_physics": 0.053409388860066735, "survival_time": 14.950000000000076, "driven_lanedir": 1.6139359991719584, "sim_render-ego": 0.006853654384613037, "in-drivable-lane": 2.500000000000023, "agent_compute-ego": 0.016465152104695638, "deviation-heading": 8.411116569649812, "set_robot_commands": 0.004886645476023356, "deviation-center-line": 0.939712292626745, "driven_lanedir_consec": 0.6209190509002489, "sim_compute_sim_state": 0.0027624114354451498, "sim_compute_performance-ego": 0.0039050873120625817, "sim_compute_robot_state-ego": 0.004786221186319987}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.6665917630475674, "sim_physics": 0.059093985506283336, "survival_time": 4.6499999999999915, "driven_lanedir": 0.2791576219820424, "sim_render-ego": 0.007045592031171245, "in-drivable-lane": 0.6999999999999986, "agent_compute-ego": 0.015922630986859723, "deviation-heading": 3.2240652860087584, "set_robot_commands": 0.004937789773428312, "deviation-center-line": 0.3601671609731881, "driven_lanedir_consec": 0.23825094246779344, "sim_compute_sim_state": 0.0027974446614583335, "sim_compute_performance-ego": 0.004031929918514785, "sim_compute_robot_state-ego": 0.004824087183962586}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.5171316279641012, "sim_physics": 0.05637209875541821, "survival_time": 2.849999999999998, "driven_lanedir": 0.2342680661414066, "sim_render-ego": 0.006970539427640145, "in-drivable-lane": 0.2999999999999998, "agent_compute-ego": 0.015745430661920915, "deviation-heading": 2.2987305742411173, "set_robot_commands": 0.004824395765338028, "deviation-center-line": 0.2218955514810732, "driven_lanedir_consec": 0.11933933535416552, "sim_compute_sim_state": 0.0027375890497575725, "sim_compute_performance-ego": 0.003877936748036167, "sim_compute_robot_state-ego": 0.004756366997434382}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 2.1621364486830372, "sim_physics": 0.05422053019205729, "survival_time": 14.950000000000076, "driven_lanedir": 1.1009335435407446, "sim_render-ego": 0.006785808404286702, "in-drivable-lane": 2.3000000000000154, "agent_compute-ego": 0.015795161724090578, "deviation-heading": 10.279209070179563, "set_robot_commands": 0.004880797863006592, "deviation-center-line": 0.9308740173042273, "driven_lanedir_consec": 0.6778457105812771, "sim_compute_sim_state": 0.002789583206176758, "sim_compute_performance-ego": 0.003910171190897624, "sim_compute_robot_state-ego": 0.004820861021677653}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.43033908408394145, "sim_physics": 0.05738648267892691, "survival_time": 3.2499999999999964, "driven_lanedir": 0.20750988057552444, "sim_render-ego": 0.007029148248525767, "in-drivable-lane": 0.4000000000000003, "agent_compute-ego": 0.016202251727764422, "deviation-heading": 2.4218211456562564, "set_robot_commands": 0.004751462202805739, "deviation-center-line": 0.2697603838557514, "driven_lanedir_consec": 0.08609262859199884, "sim_compute_sim_state": 0.0027640452751746545, "sim_compute_performance-ego": 0.003900876412024865, "sim_compute_robot_state-ego": 0.0048485315763033355}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 2.7870195235428024, "sim_physics": 0.05311930576960246, "survival_time": 14.950000000000076, "driven_lanedir": 1.828068484382404, "sim_render-ego": 0.006797926425933838, "in-drivable-lane": 1.7500000000000062, "agent_compute-ego": 0.015662923653920493, "deviation-heading": 9.14112042772404, "set_robot_commands": 0.004818286101023356, "deviation-center-line": 0.8568705548152229, "driven_lanedir_consec": 0.7032190955787656, "sim_compute_sim_state": 0.002726901372273763, "sim_compute_performance-ego": 0.003874436219533285, "sim_compute_robot_state-ego": 0.004711151123046875}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 1.9853092835703976, "sim_physics": 0.055154840522837416, "survival_time": 10.700000000000015, "driven_lanedir": 1.1265672751079694, "sim_render-ego": 0.006820268720109886, "in-drivable-lane": 1.2000000000000048, "agent_compute-ego": 0.015762511814866112, "deviation-heading": 7.348114817120294, "set_robot_commands": 0.0048639763181454665, "deviation-center-line": 0.6610985030103296, "driven_lanedir_consec": 0.5054532817803064, "sim_compute_sim_state": 0.0027552876517037365, "sim_compute_performance-ego": 0.0038790246036565193, "sim_compute_robot_state-ego": 0.004778704910634834}}
set_robot_commands_max0.005018730632594374
set_robot_commands_mean0.004874920890169992
set_robot_commands_median0.0048639763181454665
set_robot_commands_min0.004751462202805739
sim_compute_performance-ego_max0.0042560661540311925
sim_compute_performance-ego_mean0.003926342693524701
sim_compute_performance-ego_median0.0038897781758695034
sim_compute_performance-ego_min0.003871693646997437
sim_compute_robot_state-ego_max0.0050239656485763255
sim_compute_robot_state-ego_mean0.004822747756123904
sim_compute_robot_state-ego_median0.004786221186319987
sim_compute_robot_state-ego_min0.004711151123046875
sim_compute_sim_state_max0.00288322714508557
sim_compute_sim_state_mean0.002782082695688623
sim_compute_sim_state_median0.0027640452751746545
sim_compute_sim_state_min0.002726901372273763
sim_physics_max0.059093985506283336
sim_physics_mean0.05526747785308201
sim_physics_median0.055154840522837416
sim_physics_min0.05299765521829779
sim_render-ego_max0.015337387720743816
sim_render-ego_mean0.007486272160996964
sim_render-ego_median0.00690000057220459
sim_render-ego_min0.006785808404286702
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean7.563333333333345
survival_time_min2.549999999999999
No reset possible
250804369Jeick Hincapie Barrera 🇨🇴challenge-aido_LF-template-tensorflowaido3-LFV-sim-validationstep1-simulationsuccessnoip-172-31-46-148-104910:07:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.2105373214924744
survival_time_median1.0000000000000002
deviation-center-line_median0.09046060766595944
in-drivable-lane_median0


other stats
agent_compute-ego_max0.032265210151672365
agent_compute-ego_mean0.026906436591768983
agent_compute-ego_median0.026336526870727538
agent_compute-ego_min0.02329115812168565
deviation-center-line_max0.3387768689144781
deviation-center-line_mean0.14773041537790532
deviation-center-line_min0.06574019788387248
deviation-heading_max1.9000233423448305
deviation-heading_mean0.7988223789960747
deviation-heading_median0.6112707688690542
deviation-heading_min0.24489108606781845
driven_any_max0.9971365984164938
driven_any_mean0.43931180124120583
driven_any_median0.22854748650887163
driven_any_min0.21144067954583665
driven_lanedir_consec_max0.8493468721967012
driven_lanedir_consec_mean0.3649769033683262
driven_lanedir_consec_min0.11778987385731464
driven_lanedir_max0.8493468721967012
driven_lanedir_mean0.3649769033683262
driven_lanedir_median0.2105373214924744
driven_lanedir_min0.11778987385731464
in-drivable-lane_max0.05000000000000005
in-drivable-lane_mean0.010000000000000007
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.21144067954583665, "sim_physics": 0.21464200019836427, "survival_time": 1.0000000000000002, "driven_lanedir": 0.11778987385731464, "sim_render-ego": 0.012111485004425049, "in-drivable-lane": 0, "agent_compute-ego": 0.032265210151672365, "deviation-heading": 0.6112707688690542, "set_robot_commands": 0.009484195709228515, "deviation-center-line": 0.06574019788387248, "driven_lanedir_consec": 0.11778987385731464, "sim_compute_sim_state": 0.004468750953674316, "sim_compute_performance-ego": 0.007492852210998535, "sim_compute_robot_state-ego": 0.008811914920806884, "sim_compute_robot_state-npc0": 0.008249235153198243, "sim_compute_robot_state-npc1": 0.007834219932556152, "sim_compute_robot_state-npc2": 0.008020389080047607, "sim_compute_robot_state-npc3": 0.007935857772827149}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.5377713527957281, "sim_physics": 0.1599921092652438, "survival_time": 2.849999999999998, "driven_lanedir": 0.4653379505946149, "sim_render-ego": 0.011661211649576822, "in-drivable-lane": 0, "agent_compute-ego": 0.02508787523236191, "deviation-heading": 0.9611222818816488, "set_robot_commands": 0.008607404273852967, "deviation-center-line": 0.16557552545797438, "driven_lanedir_consec": 0.4653379505946149, "sim_compute_sim_state": 0.004348499733105041, "sim_compute_performance-ego": 0.006617387135823567, "sim_compute_robot_state-ego": 0.008353162230106822, "sim_compute_robot_state-npc0": 0.007876504931533546, "sim_compute_robot_state-npc1": 0.007377348448100843, "sim_compute_robot_state-npc2": 0.007371781165139717, "sim_compute_robot_state-npc3": 0.007121956139280085}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.2216628889390988, "sim_physics": 0.2566783177225213, "survival_time": 0.9500000000000004, "driven_lanedir": 0.181872498700526, "sim_render-ego": 0.01312667445132607, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.02755141258239746, "deviation-heading": 0.24489108606781845, "set_robot_commands": 0.01084983976263749, "deviation-center-line": 0.07809887696724217, "driven_lanedir_consec": 0.181872498700526, "sim_compute_sim_state": 0.004920645763999538, "sim_compute_performance-ego": 0.00817477075677169, "sim_compute_robot_state-ego": 0.010218996750681024, "sim_compute_robot_state-npc0": 0.009493852916516755, "sim_compute_robot_state-npc1": 0.008988957656057258, "sim_compute_robot_state-npc2": 0.009074060540450247, "sim_compute_robot_state-npc3": 0.008844601480584396}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.22854748650887163, "sim_physics": 0.22145016193389896, "survival_time": 1.0000000000000002, "driven_lanedir": 0.2105373214924744, "sim_render-ego": 0.011350286006927493, "in-drivable-lane": 0, "agent_compute-ego": 0.026336526870727538, "deviation-heading": 0.2768044158170217, "set_robot_commands": 0.008685731887817382, "deviation-center-line": 0.09046060766595944, "driven_lanedir_consec": 0.2105373214924744, "sim_compute_sim_state": 0.004439389705657959, "sim_compute_performance-ego": 0.00679234266281128, "sim_compute_robot_state-ego": 0.008042550086975098, "sim_compute_robot_state-npc0": 0.008105206489562988, "sim_compute_robot_state-npc1": 0.00746234655380249, "sim_compute_robot_state-npc2": 0.007573163509368897, "sim_compute_robot_state-npc3": 0.0075878739356994625}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.9971365984164938, "sim_physics": 0.12256281597669734, "survival_time": 4.299999999999993, "driven_lanedir": 0.8493468721967012, "sim_render-ego": 0.011375488236893057, "in-drivable-lane": 0, "agent_compute-ego": 0.02329115812168565, "deviation-heading": 1.9000233423448305, "set_robot_commands": 0.007019309110419695, "deviation-center-line": 0.3387768689144781, "driven_lanedir_consec": 0.8493468721967012, "sim_compute_sim_state": 0.003727175468622252, "sim_compute_performance-ego": 0.005620848300845124, "sim_compute_robot_state-ego": 0.006795747335567031, "sim_compute_robot_state-npc0": 0.00667865609013757, "sim_compute_robot_state-npc1": 0.006395631058271541, "sim_compute_robot_state-npc2": 0.006230268367501192, "sim_compute_robot_state-npc3": 0.006189958993778672}}
set_robot_commands_max0.01084983976263749
set_robot_commands_mean0.00892929614879121
set_robot_commands_median0.008685731887817382
set_robot_commands_min0.007019309110419695
sim_compute_performance-ego_max0.00817477075677169
sim_compute_performance-ego_mean0.006939640213450039
sim_compute_performance-ego_median0.00679234266281128
sim_compute_performance-ego_min0.005620848300845124
sim_compute_robot_state-ego_max0.010218996750681024
sim_compute_robot_state-ego_mean0.008444474264827372
sim_compute_robot_state-ego_median0.008353162230106822
sim_compute_robot_state-ego_min0.006795747335567031
sim_compute_robot_state-npc0_max0.009493852916516755
sim_compute_robot_state-npc0_mean0.00808069111618982
sim_compute_robot_state-npc0_median0.008105206489562988
sim_compute_robot_state-npc0_min0.00667865609013757
sim_compute_robot_state-npc1_max0.008988957656057258
sim_compute_robot_state-npc1_mean0.007611700729757656
sim_compute_robot_state-npc1_median0.00746234655380249
sim_compute_robot_state-npc1_min0.006395631058271541
sim_compute_robot_state-npc2_max0.009074060540450247
sim_compute_robot_state-npc2_mean0.007653932532501533
sim_compute_robot_state-npc2_median0.007573163509368897
sim_compute_robot_state-npc2_min0.006230268367501192
sim_compute_robot_state-npc3_max0.008844601480584396
sim_compute_robot_state-npc3_mean0.007536049664433953
sim_compute_robot_state-npc3_median0.0075878739356994625
sim_compute_robot_state-npc3_min0.006189958993778672
sim_compute_sim_state_max0.004920645763999538
sim_compute_sim_state_mean0.004380892325011822
sim_compute_sim_state_median0.004439389705657959
sim_compute_sim_state_min0.003727175468622252
sim_physics_max0.2566783177225213
sim_physics_mean0.1950650810193451
sim_physics_median0.21464200019836427
sim_physics_min0.12256281597669734
sim_render-ego_max0.01312667445132607
sim_render-ego_mean0.011925029069829698
sim_render-ego_median0.011661211649576822
sim_render-ego_min0.011350286006927493
simulation-passed1
survival_time_max4.299999999999993
survival_time_mean2.0199999999999982
survival_time_min0.9500000000000004
No reset possible
250244362Soroush Saryazdi 🇨🇦bluejayaido3-LFVI-sim-testingstep1-simulationerrornoip-172-31-46-148-104910:02:30
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
250034126Andrea Censi 🇨🇭rotationaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-46-148-104910:15:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median-0.0005547867508171045
survival_time_median14.950000000000076
deviation-center-line_median0.44901648578509457
in-drivable-lane_median7.600000000000044


other stats
agent_compute-ego_max0.10970013856887816
agent_compute-ego_mean0.10400235239664712
agent_compute-ego_median0.1065246605873108
agent_compute-ego_min0.0953879443804423
deviation-center-line_max0.5610133203215761
deviation-center-line_mean0.3827485453764907
deviation-center-line_min0.13860669343031343
deviation-heading_max5.87668078883977
deviation-heading_mean5.630778146323924
deviation-heading_median5.657649106857694
deviation-heading_min5.445457843647877
driven_any_max0.026478672834058543
driven_any_mean0.024864239592249697
driven_any_median0.025930195541195415
driven_any_min0.022165452661378815
driven_lanedir_consec_max0.0021599934447711355
driven_lanedir_consec_mean-0.00046586963337813536
driven_lanedir_consec_min-0.003361806018059577
driven_lanedir_max0.0021599934447711355
driven_lanedir_mean-0.00046586963337813536
driven_lanedir_median-0.0005547867508171045
driven_lanedir_min-0.003361806018059577
in-drivable-lane_max7.700000000000042
in-drivable-lane_mean7.620000000000043
in-drivable-lane_min7.550000000000042
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.026478672834058543, "sim_physics": 0.11031422773996988, "survival_time": 14.950000000000076, "driven_lanedir": 0.0021599934447711355, "sim_render-ego": 0.012085114320119222, "in-drivable-lane": 7.600000000000043, "agent_compute-ego": 0.1065246605873108, "deviation-heading": 5.657649106857694, "set_robot_commands": 0.00916763385136922, "deviation-center-line": 0.5610133203215761, "driven_lanedir_consec": 0.0021599934447711355, "sim_compute_sim_state": 0.004978666305541992, "sim_compute_performance-ego": 0.007112902005513509, "sim_compute_robot_state-ego": 0.008862034479777018}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.025930195541195415, "sim_physics": 0.09346266190210978, "survival_time": 14.950000000000076, "driven_lanedir": -0.003361806018059577, "sim_render-ego": 0.011716043949127198, "in-drivable-lane": 7.600000000000044, "agent_compute-ego": 0.0953879443804423, "deviation-heading": 5.445457843647877, "set_robot_commands": 0.008896360397338867, "deviation-center-line": 0.13860669343031343, "driven_lanedir_consec": -0.003361806018059577, "sim_compute_sim_state": 0.00487368106842041, "sim_compute_performance-ego": 0.0069377930959065755, "sim_compute_robot_state-ego": 0.008689845403035482}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.023804404672436115, "sim_physics": 0.09850303888320924, "survival_time": 14.950000000000076, "driven_lanedir": 0.002118105955655203, "sim_render-ego": 0.012315864562988282, "in-drivable-lane": 7.550000000000042, "agent_compute-ego": 0.101023694674174, "deviation-heading": 5.87668078883977, "set_robot_commands": 0.009218656222025551, "deviation-center-line": 0.47845174752595304, "driven_lanedir_consec": 0.002118105955655203, "sim_compute_sim_state": 0.004943937460581462, "sim_compute_performance-ego": 0.007392756144205729, "sim_compute_robot_state-ego": 0.009251983165740966}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.022165452661378815, "sim_physics": 0.11514880975087484, "survival_time": 14.950000000000076, "driven_lanedir": -0.002690854798440334, "sim_render-ego": 0.012264773845672608, "in-drivable-lane": 7.700000000000042, "agent_compute-ego": 0.10970013856887816, "deviation-heading": 5.721005148732665, "set_robot_commands": 0.0089356263478597, "deviation-center-line": 0.44901648578509457, "driven_lanedir_consec": -0.002690854798440334, "sim_compute_sim_state": 0.004959495862325033, "sim_compute_performance-ego": 0.00726104736328125, "sim_compute_robot_state-ego": 0.009163438479105631}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.025942472252179608, "sim_physics": 0.11410626490910848, "survival_time": 14.950000000000076, "driven_lanedir": -0.0005547867508171045, "sim_render-ego": 0.012358726660410563, "in-drivable-lane": 7.650000000000044, "agent_compute-ego": 0.10737532377243042, "deviation-heading": 5.453097843541614, "set_robot_commands": 0.009725379943847656, "deviation-center-line": 0.28665447981951625, "driven_lanedir_consec": -0.0005547867508171045, "sim_compute_sim_state": 0.005045426686604818, "sim_compute_performance-ego": 0.007305744489034017, "sim_compute_robot_state-ego": 0.009330426851908366}}
set_robot_commands_max0.009725379943847656
set_robot_commands_mean0.0091887313524882
set_robot_commands_median0.00916763385136922
set_robot_commands_min0.008896360397338867
sim_compute_performance-ego_max0.007392756144205729
sim_compute_performance-ego_mean0.007202048619588217
sim_compute_performance-ego_median0.00726104736328125
sim_compute_performance-ego_min0.0069377930959065755
sim_compute_robot_state-ego_max0.009330426851908366
sim_compute_robot_state-ego_mean0.009059545675913492
sim_compute_robot_state-ego_median0.009163438479105631
sim_compute_robot_state-ego_min0.008689845403035482
sim_compute_sim_state_max0.005045426686604818
sim_compute_sim_state_mean0.0049602414766947425
sim_compute_sim_state_median0.004959495862325033
sim_compute_sim_state_min0.00487368106842041
sim_physics_max0.11514880975087484
sim_physics_mean0.10630700063705444
sim_physics_median0.11031422773996988
sim_physics_min0.09346266190210978
sim_render-ego_max0.012358726660410563
sim_render-ego_mean0.012148104667663575
sim_render-ego_median0.012264773845672608
sim_render-ego_min0.011716043949127198
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
249934140Andrea Censi 🇨🇭rotationaido3-LFVI-sim-validationstep1-simulationsuccessnoip-172-31-46-148-104910:22:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.00030504467687686443
survival_time_median14.950000000000076
deviation-center-line_median0.4891046565937845
in-drivable-lane_median7.600000000000043


other stats
agent_compute-ego_max0.023066624800364175
agent_compute-ego_mean0.022727931181589764
agent_compute-ego_median0.02287506659825643
agent_compute-ego_min0.0220133376121521
deviation-center-line_max0.6113246250082045
deviation-center-line_mean0.4225460275157091
deviation-center-line_min0.020737627062439232
deviation-heading_max5.830135897709591
deviation-heading_mean5.6845705507076865
deviation-heading_median5.657649106857696
deviation-heading_min5.53666299685486
driven_any_max0.026478672834052443
driven_any_mean0.02486423959225232
driven_any_median0.02593019554120905
driven_any_min0.022165452661382787
driven_lanedir_consec_max0.002162616049345756
driven_lanedir_consec_mean-0.0004886462276807446
driven_lanedir_consec_min-0.0036402347654349576
driven_lanedir_max0.002162616049345756
driven_lanedir_mean-0.0004886462276807446
driven_lanedir_median0.00030504467687686443
driven_lanedir_min-0.0036402347654349576
in-drivable-lane_max7.650000000000043
in-drivable-lane_mean7.600000000000042
in-drivable-lane_min7.550000000000044
per-episodes
details{"udem1-0-0": {"driven_any": 0.026478672834052443, "sim_physics": 0.18475377241770427, "survival_time": 14.950000000000076, "driven_lanedir": 0.002162616049345756, "sim_render-ego": 0.01184730847676595, "in-drivable-lane": 7.600000000000043, "agent_compute-ego": 0.023066624800364175, "deviation-heading": 5.657649106857696, "set_robot_commands": 0.008987046877543132, "deviation-center-line": 0.561012181201902, "driven_lanedir_consec": 0.002162616049345756, "sim_compute_sim_state": 0.004900589783986409, "sim_compute_performance-ego": 0.007231624921162924, "sim_compute_robot_state-ego": 0.008895676930745442, "sim_compute_robot_state-npc0": 0.008355889320373535, "sim_compute_robot_state-npc1": 0.008076038360595703, "sim_compute_robot_state-npc2": 0.008253127733866373, "sim_compute_robot_state-npc3": 0.008012163639068603}, "udem1-1-0": {"driven_any": 0.02593019554120905, "sim_physics": 0.19165331761042276, "survival_time": 14.950000000000076, "driven_lanedir": -0.0036402347654349576, "sim_render-ego": 0.012417867183685302, "in-drivable-lane": 7.650000000000043, "agent_compute-ego": 0.02299348751703898, "deviation-heading": 5.79531637039674, "set_robot_commands": 0.00918852170308431, "deviation-center-line": 0.6113246250082045, "driven_lanedir_consec": -0.0036402347654349576, "sim_compute_sim_state": 0.004861683050791423, "sim_compute_performance-ego": 0.00712154229482015, "sim_compute_robot_state-ego": 0.009026634693145751, "sim_compute_robot_state-npc0": 0.008376406828562418, "sim_compute_robot_state-npc1": 0.008098715941111247, "sim_compute_robot_state-npc2": 0.007796008586883545, "sim_compute_robot_state-npc3": 0.007920705477396647}, "udem1-2-0": {"driven_any": 0.023804404672430148, "sim_physics": 0.18834747076034547, "survival_time": 14.950000000000076, "driven_lanedir": 0.0017436411031614971, "sim_render-ego": 0.012414127985636392, "in-drivable-lane": 7.550000000000044, "agent_compute-ego": 0.02287506659825643, "deviation-heading": 5.603088381719548, "set_robot_commands": 0.009194196859995524, "deviation-center-line": 0.4891046565937845, "driven_lanedir_consec": 0.0017436411031614971, "sim_compute_sim_state": 0.004846236705780029, "sim_compute_performance-ego": 0.007320585250854492, "sim_compute_robot_state-ego": 0.009036441644032797, "sim_compute_robot_state-npc0": 0.008228889306386312, "sim_compute_robot_state-npc1": 0.008055080572764078, "sim_compute_robot_state-npc2": 0.007901748816172283, "sim_compute_robot_state-npc3": 0.008189899921417236}, "udem1-3-0": {"driven_any": 0.022165452661382787, "sim_physics": 0.16748192071914672, "survival_time": 14.950000000000076, "driven_lanedir": -0.003014298202352883, "sim_render-ego": 0.010887529850006104, "in-drivable-lane": 7.600000000000044, "agent_compute-ego": 0.0220133376121521, "deviation-heading": 5.53666299685486, "set_robot_commands": 0.008028364181518555, "deviation-center-line": 0.4305510477122152, "driven_lanedir_consec": -0.003014298202352883, "sim_compute_sim_state": 0.004296416441599528, "sim_compute_performance-ego": 0.006180969079335531, "sim_compute_robot_state-ego": 0.007876338164011638, "sim_compute_robot_state-npc0": 0.0073238976796468095, "sim_compute_robot_state-npc1": 0.007054825623830159, "sim_compute_robot_state-npc2": 0.007079644997914632, "sim_compute_robot_state-npc3": 0.007272063891092936}, "udem1-4-0": {"driven_any": 0.02594247225218716, "sim_physics": 0.18078513145446776, "survival_time": 14.950000000000076, "driven_lanedir": 0.00030504467687686443, "sim_render-ego": 0.01203542153040568, "in-drivable-lane": 7.600000000000042, "agent_compute-ego": 0.02269113938013713, "deviation-heading": 5.830135897709591, "set_robot_commands": 0.008852940400441487, "deviation-center-line": 0.020737627062439232, "driven_lanedir_consec": 0.00030504467687686443, "sim_compute_sim_state": 0.00476535161336263, "sim_compute_performance-ego": 0.007224415938059489, "sim_compute_robot_state-ego": 0.008948921362559, "sim_compute_robot_state-npc0": 0.008064843813578288, "sim_compute_robot_state-npc1": 0.007807010809580485, "sim_compute_robot_state-npc2": 0.008239996433258057, "sim_compute_robot_state-npc3": 0.007766455014546712}}
set_robot_commands_max0.009194196859995524
set_robot_commands_mean0.008850214004516601
set_robot_commands_median0.008987046877543132
set_robot_commands_min0.008028364181518555
sim_compute_performance-ego_max0.007320585250854492
sim_compute_performance-ego_mean0.007015827496846518
sim_compute_performance-ego_median0.007224415938059489
sim_compute_performance-ego_min0.006180969079335531
sim_compute_robot_state-ego_max0.009036441644032797
sim_compute_robot_state-ego_mean0.008756802558898925
sim_compute_robot_state-ego_median0.008948921362559
sim_compute_robot_state-ego_min0.007876338164011638
sim_compute_robot_state-npc0_max0.008376406828562418
sim_compute_robot_state-npc0_mean0.008069985389709473
sim_compute_robot_state-npc0_median0.008228889306386312
sim_compute_robot_state-npc0_min0.0073238976796468095
sim_compute_robot_state-npc1_max0.008098715941111247
sim_compute_robot_state-npc1_mean0.007818334261576335
sim_compute_robot_state-npc1_median0.008055080572764078
sim_compute_robot_state-npc1_min0.007054825623830159
sim_compute_robot_state-npc2_max0.008253127733866373
sim_compute_robot_state-npc2_mean0.007854105313618978
sim_compute_robot_state-npc2_median0.007901748816172283
sim_compute_robot_state-npc2_min0.007079644997914632
sim_compute_robot_state-npc3_max0.008189899921417236
sim_compute_robot_state-npc3_mean0.007832257588704426
sim_compute_robot_state-npc3_median0.007920705477396647
sim_compute_robot_state-npc3_min0.007272063891092936
sim_compute_sim_state_max0.004900589783986409
sim_compute_sim_state_mean0.004734055519104004
sim_compute_sim_state_median0.004846236705780029
sim_compute_sim_state_min0.004296416441599528
sim_physics_max0.19165331761042276
sim_physics_mean0.18260432259241735
sim_physics_median0.18475377241770427
sim_physics_min0.16748192071914672
sim_render-ego_max0.012417867183685302
sim_render-ego_mean0.011920451005299888
sim_render-ego_median0.01203542153040568
sim_render-ego_min0.010887529850006104
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
249864156Andrea Censi 🇨🇭minimal_agent_python2 (Python 2)aido3-LFV-sim-validationstep1-simulationsuccessnoip-172-31-46-148-104910:07:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6707441932890994
survival_time_median3.649999999999995
deviation-center-line_median0.14435558631904788
in-drivable-lane_median0.9499999999999966


other stats
agent_compute-ego_max0.010830087321145194
agent_compute-ego_mean0.009868699064973912
agent_compute-ego_median0.009481642697308517
agent_compute-ego_min0.009403161704540253
deviation-center-line_max0.2085822874042474
deviation-center-line_mean0.1589542518908908
deviation-center-line_min0.10686452122645386
deviation-heading_max0.6191860504142408
deviation-heading_mean0.523857550127959
deviation-heading_median0.5156622477370731
deviation-heading_min0.42218734391092383
driven_any_max1.5798273185720382
driven_any_mean0.9978358087547315
driven_any_median0.9948273195001778
driven_any_min0.3798605253583843
driven_lanedir_consec_max1.2017887891468408
driven_lanedir_consec_mean0.7740609626522831
driven_lanedir_consec_min0.366714522485295
driven_lanedir_max1.2017887891468408
driven_lanedir_mean0.7740609626522831
driven_lanedir_median0.6707441932890994
driven_lanedir_min0.366714522485295
in-drivable-lane_max1.1999999999999955
in-drivable-lane_mean0.6699999999999976
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9948273195001778, "sim_physics": 0.1917030125448149, "survival_time": 3.649999999999995, "driven_lanedir": 0.6707441932890994, "sim_render-ego": 0.012857035414813317, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.00947938226673701, "deviation-heading": 0.5590533399929921, "set_robot_commands": 0.009043079532989084, "deviation-center-line": 0.14435558631904788, "driven_lanedir_consec": 0.6707441932890994, "sim_compute_sim_state": 0.004945343487883267, "sim_compute_performance-ego": 0.00743250650902317, "sim_compute_robot_state-ego": 0.009787582371332874, "sim_compute_robot_state-npc0": 0.008273369645419186, "sim_compute_robot_state-npc1": 0.008232168955345676, "sim_compute_robot_state-npc2": 0.008300392595055986, "sim_compute_robot_state-npc3": 0.008399502871787711}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4548365617710475, "sim_physics": 0.17105668299906962, "survival_time": 1.850000000000001, "driven_lanedir": 0.4295977528045558, "sim_render-ego": 0.012502812050484322, "in-drivable-lane": 0, "agent_compute-ego": 0.009481642697308517, "deviation-heading": 0.6191860504142408, "set_robot_commands": 0.008826339567029799, "deviation-center-line": 0.13177328856984177, "driven_lanedir_consec": 0.4295977528045558, "sim_compute_sim_state": 0.004496323095785605, "sim_compute_performance-ego": 0.007030184204513962, "sim_compute_robot_state-ego": 0.009610659367329364, "sim_compute_robot_state-npc0": 0.008095154891142974, "sim_compute_robot_state-npc1": 0.007810599095112568, "sim_compute_robot_state-npc2": 0.007577303293588999, "sim_compute_robot_state-npc3": 0.007379029248211835}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.57982731857201, "sim_physics": 0.2092039712837764, "survival_time": 5.599999999999988, "driven_lanedir": 1.2014595555356244, "sim_render-ego": 0.012817808559962682, "in-drivable-lane": 1.1999999999999955, "agent_compute-ego": 0.010830087321145194, "deviation-heading": 0.5156622477370731, "set_robot_commands": 0.010245757443564278, "deviation-center-line": 0.2085822874042474, "driven_lanedir_consec": 1.2014595555356244, "sim_compute_sim_state": 0.0051609788622174945, "sim_compute_performance-ego": 0.007906809449195862, "sim_compute_robot_state-ego": 0.010025814175605774, "sim_compute_robot_state-npc0": 0.009253908480916704, "sim_compute_robot_state-npc1": 0.009363608700888496, "sim_compute_robot_state-npc2": 0.008856162428855896, "sim_compute_robot_state-npc3": 0.00875312089920044}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.5798273185720382, "sim_physics": 0.17896118547235215, "survival_time": 5.599999999999988, "driven_lanedir": 1.2017887891468408, "sim_render-ego": 0.011714424405779158, "in-drivable-lane": 1.1999999999999955, "agent_compute-ego": 0.010149221335138594, "deviation-heading": 0.5031987685845647, "set_robot_commands": 0.009142062493732997, "deviation-center-line": 0.2031955759348632, "driven_lanedir_consec": 1.2017887891468408, "sim_compute_sim_state": 0.004735812544822693, "sim_compute_performance-ego": 0.006921299866267613, "sim_compute_robot_state-ego": 0.009220229727881297, "sim_compute_robot_state-npc0": 0.008542967694146293, "sim_compute_robot_state-npc1": 0.007861013923372542, "sim_compute_robot_state-npc2": 0.007927211267607552, "sim_compute_robot_state-npc3": 0.00802978447505406}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.3798605253583843, "sim_physics": 0.16683577001094818, "survival_time": 1.6000000000000008, "driven_lanedir": 0.366714522485295, "sim_render-ego": 0.01266661286354065, "in-drivable-lane": 0, "agent_compute-ego": 0.009403161704540253, "deviation-heading": 0.42218734391092383, "set_robot_commands": 0.008715681731700897, "deviation-center-line": 0.10686452122645386, "driven_lanedir_consec": 0.366714522485295, "sim_compute_sim_state": 0.005689740180969238, "sim_compute_performance-ego": 0.007246255874633789, "sim_compute_robot_state-ego": 0.010112248361110687, "sim_compute_robot_state-npc0": 0.00817977637052536, "sim_compute_robot_state-npc1": 0.00784861296415329, "sim_compute_robot_state-npc2": 0.007977932691574097, "sim_compute_robot_state-npc3": 0.008733831346035004}}
set_robot_commands_max0.010245757443564278
set_robot_commands_mean0.009194584153803411
set_robot_commands_median0.009043079532989084
set_robot_commands_min0.008715681731700897
sim_compute_performance-ego_max0.007906809449195862
sim_compute_performance-ego_mean0.007307411180726879
sim_compute_performance-ego_median0.007246255874633789
sim_compute_performance-ego_min0.006921299866267613
sim_compute_robot_state-ego_max0.010112248361110687
sim_compute_robot_state-ego_mean0.009751306800652
sim_compute_robot_state-ego_median0.009787582371332874
sim_compute_robot_state-ego_min0.009220229727881297
sim_compute_robot_state-npc0_max0.009253908480916704
sim_compute_robot_state-npc0_mean0.008469035416430102
sim_compute_robot_state-npc0_median0.008273369645419186
sim_compute_robot_state-npc0_min0.008095154891142974
sim_compute_robot_state-npc1_max0.009363608700888496
sim_compute_robot_state-npc1_mean0.008223200727774516
sim_compute_robot_state-npc1_median0.007861013923372542
sim_compute_robot_state-npc1_min0.007810599095112568
sim_compute_robot_state-npc2_max0.008856162428855896
sim_compute_robot_state-npc2_mean0.008127800455336506
sim_compute_robot_state-npc2_median0.007977932691574097
sim_compute_robot_state-npc2_min0.007577303293588999
sim_compute_robot_state-npc3_max0.00875312089920044
sim_compute_robot_state-npc3_mean0.00825905376805781
sim_compute_robot_state-npc3_median0.008399502871787711
sim_compute_robot_state-npc3_min0.007379029248211835
sim_compute_sim_state_max0.005689740180969238
sim_compute_sim_state_mean0.00500563963433566
sim_compute_sim_state_median0.004945343487883267
sim_compute_sim_state_min0.004496323095785605
sim_physics_max0.2092039712837764
sim_physics_mean0.18355212446219224
sim_physics_median0.17896118547235215
sim_physics_min0.16683577001094818
sim_render-ego_max0.012857035414813317
sim_render-ego_mean0.012511738658916024
sim_render-ego_median0.01266661286354065
sim_render-ego_min0.011714424405779158
simulation-passed1
survival_time_max5.599999999999988
survival_time_mean3.6599999999999944
survival_time_min1.6000000000000008
No reset possible
249754168Tanli Chou 🇹🇼challenge-aido_LF-template-randomaido3-LFVI-sim-validationstep1-simulationerrornoip-172-31-46-148-104910:02:54
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249694174Bhairav Mehtachallenge-aido_LF-template-pytorchaido3-LFV-sim-validationstep1-simulationfailednoip-172-31-46-148-104910:01:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 298, in run_episode
    agent.write_topic_and_expect_zero('observations', recv.data.observations)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 105, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "observations".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "solution.py", line 36, in on_received_observations
               ||     self.current_image = self.preprocessor.preprocess(obs)
               ||   File "/workspace/wrappers.py", line 9, in preprocess
               ||     from scipy.misc import imresize
               || ImportError: cannot import name 'imresize' from 'scipy.misc' (/usr/local/lib/python3.7/site-packages/scipy/misc/__init__.py)
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249604187Liam Paull 🇨🇦challenge-aido_LF-template-ros - Template solution using ROSaido3-LF-sim-validationstep1-simulationsuccessnoip-172-31-46-148-104910:03:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.36592094708219225
survival_time_median2.3999999999999995
deviation-center-line_median0.11667593935453972
in-drivable-lane_median0.29999999999999893


other stats
agent_compute-ego_max0.02766881783803304
agent_compute-ego_mean0.022961857809207096
agent_compute-ego_median0.022891662337563255
agent_compute-ego_min0.01907997735789124
deviation-center-line_max0.16987818035809085
deviation-center-line_mean0.13199870851217008
deviation-center-line_min0.1011892720306074
deviation-heading_max0.5809378480871985
deviation-heading_mean0.4194004386292104
deviation-heading_median0.34882560565471876
deviation-heading_min0.2596285487829773
driven_any_max0.9226822529128632
driven_any_mean0.6324710235085376
driven_any_median0.6091462024047467
driven_any_min0.2364518949491652
driven_lanedir_consec_max0.6632135184572033
driven_lanedir_consec_mean0.4245071231796043
driven_lanedir_consec_min0.2221238695940002
driven_lanedir_max0.6632135184572033
driven_lanedir_mean0.4245071231796043
driven_lanedir_median0.36592094708219225
driven_lanedir_min0.2221238695940002
in-drivable-lane_max1.8999999999999944
in-drivable-lane_mean0.6299999999999986
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9226822529128632, "sim_physics": 0.07479687140021525, "survival_time": 3.5499999999999954, "driven_lanedir": 0.36592094708219225, "sim_render-ego": 0.009580081617328482, "in-drivable-lane": 1.8999999999999944, "agent_compute-ego": 0.01907997735789124, "deviation-heading": 0.2596285487829773, "set_robot_commands": 0.00691832958812445, "deviation-center-line": 0.16987818035809085, "driven_lanedir_consec": 0.36592094708219225, "sim_compute_sim_state": 0.003872643054371149, "sim_compute_performance-ego": 0.005722113058600627, "sim_compute_robot_state-ego": 0.006953568525717292}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.7993934895811183, "sim_physics": 0.1408007542292277, "survival_time": 2.9999999999999973, "driven_lanedir": 0.6632135184572033, "sim_render-ego": 0.01570660670598348, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.02766881783803304, "deviation-heading": 0.5809378480871985, "set_robot_commands": 0.01179632345835368, "deviation-center-line": 0.11667593935453972, "driven_lanedir_consec": 0.6632135184572033, "sim_compute_sim_state": 0.006482998530069987, "sim_compute_performance-ego": 0.007949002583821614, "sim_compute_robot_state-ego": 0.011090771357218424}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.2364518949491652, "sim_physics": 0.09637846729972144, "survival_time": 1.1000000000000003, "driven_lanedir": 0.2221238695940002, "sim_render-ego": 0.010653810067610308, "in-drivable-lane": 0, "agent_compute-ego": 0.022891662337563255, "deviation-heading": 0.34882560565471876, "set_robot_commands": 0.007601174441250888, "deviation-center-line": 0.11316665887359229, "driven_lanedir_consec": 0.2221238695940002, "sim_compute_sim_state": 0.004239613359624689, "sim_compute_performance-ego": 0.005928072062405673, "sim_compute_robot_state-ego": 0.007556936957619407}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.5946812776947944, "sim_physics": 0.1278414527575175, "survival_time": 2.3999999999999995, "driven_lanedir": 0.5792955579972934, "sim_render-ego": 0.016697416702906292, "in-drivable-lane": 0, "agent_compute-ego": 0.025807703534762062, "deviation-heading": 0.568524016017798, "set_robot_commands": 0.01014581819375356, "deviation-center-line": 0.15908349194402024, "driven_lanedir_consec": 0.5792955579972934, "sim_compute_sim_state": 0.005566492676734924, "sim_compute_performance-ego": 0.007656072576840718, "sim_compute_robot_state-ego": 0.009580299258232117}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.6091462024047467, "sim_physics": 0.07573500405187192, "survival_time": 2.3, "driven_lanedir": 0.2919817227673325, "sim_render-ego": 0.009681354398312777, "in-drivable-lane": 0.9499999999999992, "agent_compute-ego": 0.019361127977785855, "deviation-heading": 0.33908617460335944, "set_robot_commands": 0.006451254305632218, "deviation-center-line": 0.1011892720306074, "driven_lanedir_consec": 0.2919817227673325, "sim_compute_sim_state": 0.004101437071095343, "sim_compute_performance-ego": 0.005415646926216457, "sim_compute_robot_state-ego": 0.006625424260678498}}
set_robot_commands_max0.01179632345835368
set_robot_commands_mean0.00858257999742296
set_robot_commands_median0.007601174441250888
set_robot_commands_min0.006451254305632218
sim_compute_performance-ego_max0.007949002583821614
sim_compute_performance-ego_mean0.006534181441577018
sim_compute_performance-ego_median0.005928072062405673
sim_compute_performance-ego_min0.005415646926216457
sim_compute_robot_state-ego_max0.011090771357218424
sim_compute_robot_state-ego_mean0.008361400071893147
sim_compute_robot_state-ego_median0.007556936957619407
sim_compute_robot_state-ego_min0.006625424260678498
sim_compute_sim_state_max0.006482998530069987
sim_compute_sim_state_mean0.004852636938379219
sim_compute_sim_state_median0.004239613359624689
sim_compute_sim_state_min0.003872643054371149
sim_physics_max0.1408007542292277
sim_physics_mean0.10311050994771076
sim_physics_median0.09637846729972144
sim_physics_min0.07479687140021525
sim_render-ego_max0.016697416702906292
sim_render-ego_mean0.012463853898428268
sim_render-ego_median0.010653810067610308
sim_render-ego_min0.009580081617328482
simulation-passed1
survival_time_max3.5499999999999954
survival_time_mean2.469999999999999
survival_time_min1.1000000000000003
No reset possible
249514194Liam Paull 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LF-sim-testingstep1-simulationerrornoip-172-31-46-148-104910:00:49
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249474201Manfred Diazchallenge-aido_LF-template-tensorflowaido3-LFV-sim-testingstep1-simulationerrornoip-172-31-46-148-104910:00:41
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249434203Manfred Diazchallenge-aido_LF-template-tensorflowaido3-LFVI-sim-testingstep1-simulationerrornoip-172-31-46-148-104910:02:00
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249354217Manfred DiazBaseline-IL-logs-tensorflowaido3-LFV-sim-testingstep1-simulationfailednoip-172-31-46-148-104910:01:35
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[{{node prefix/ConvNet/conv_layer_1/kernel/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 69, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 58, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/conv_layer_1/kernel/read', defined at:
               ||   File "imitation_agent.py", line 134, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 130, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 69, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 41, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249294220Manfred DiazBaseline-IL-logs-tensorflowaido3-LFVI-sim-validationstep1-simulationfailednoip-172-31-46-148-104910:00:55
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[{{node prefix/ConvNet/conv_layer_1/kernel/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 69, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 58, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/conv_layer_1/kernel/read', defined at:
               ||   File "imitation_agent.py", line 134, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 130, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 69, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 41, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249214227Manfred DiazBaseline-IL-logs-tensorflowaido3-LFVI-sim-testingstep1-simulationfailednoip-172-31-46-148-104910:02:51
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[{{node prefix/ConvNet/conv_layer_1/kernel/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 60, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/conv_layer_1/kernel/read', defined at:
               ||   File "imitation_agent.py", line 136, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 132, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 43, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
249184231Andrea Censi 🇨🇭challenge-aido_LF-template-pytorchaido3-LF-sim-validationstep1-simulationerrornoip-172-31-46-148-104910:00:41
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
248914261Gianmarco Bernasconichallenge-aido_LF-baseline-duckietownaido3-LF-sim-testingstep1-simulationsuccessnoip-172-31-46-148-104910:38:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9877157612172136
survival_time_median9.800000000000004
deviation-center-line_median0.560801746349332
in-drivable-lane_median1.0500000000000016


other stats
agent_compute-ego_max0.06172322247126331
agent_compute-ego_mean0.05478342117456272
agent_compute-ego_median0.05534021384051057
agent_compute-ego_min0.042545937761968494
deviation-center-line_max1.3652391647990982
deviation-center-line_mean0.6307154943614144
deviation-center-line_min0.23697101400436504
deviation-heading_max4.726102828454712
deviation-heading_mean2.8219259880271865
deviation-heading_median2.942215210088024
deviation-heading_min0.6023684295151841
driven_any_max2.3491302755517376
driven_any_mean1.5319734336783954
driven_any_median1.5135331765299278
driven_any_min0.4365263712552368
driven_lanedir_consec_max2.290391782200912
driven_lanedir_consec_mean1.0474147121379658
driven_lanedir_consec_min0.08206782941699886
driven_lanedir_max2.290391782200912
driven_lanedir_mean1.1684633418309072
driven_lanedir_median1.1548138859084167
driven_lanedir_min0.2297977381805658
in-drivable-lane_max5.650000000000037
in-drivable-lane_mean1.4033333333333418
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.5135331765299278, "sim_physics": 0.12493983093573124, "survival_time": 9.800000000000004, "driven_lanedir": 1.3569134796871396, "sim_render-ego": 0.013854526743596914, "in-drivable-lane": 0, "agent_compute-ego": 0.042545937761968494, "deviation-heading": 3.587025442793564, "set_robot_commands": 0.01240842561332547, "deviation-center-line": 0.6440631905984852, "driven_lanedir_consec": 1.3569134796871396, "sim_compute_sim_state": 0.005848883366098209, "sim_compute_performance-ego": 0.00844933792036407, "sim_compute_robot_state-ego": 0.010358226542570154}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.5569368387712803, "sim_physics": 0.1360667065570229, "survival_time": 3.7999999999999945, "driven_lanedir": 0.2297977381805658, "sim_render-ego": 0.014446123650199488, "in-drivable-lane": 0.7499999999999997, "agent_compute-ego": 0.05518963148719386, "deviation-heading": 2.4876886008759396, "set_robot_commands": 0.011082774714419716, "deviation-center-line": 0.30088669984335503, "driven_lanedir_consec": 0.10200934310813368, "sim_compute_sim_state": 0.005775523813147294, "sim_compute_performance-ego": 0.008182710722873085, "sim_compute_robot_state-ego": 0.01062356484563727}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.3488503581151305, "sim_physics": 0.1412779672940572, "survival_time": 14.950000000000076, "driven_lanedir": 1.6432629862351975, "sim_render-ego": 0.014495846430460611, "in-drivable-lane": 3.4000000000000483, "agent_compute-ego": 0.057208122412363685, "deviation-heading": 3.620166324852328, "set_robot_commands": 0.011330535411834717, "deviation-center-line": 0.8424817638607123, "driven_lanedir_consec": 1.483033168464281, "sim_compute_sim_state": 0.00614825169245402, "sim_compute_performance-ego": 0.008500122229258219, "sim_compute_robot_state-ego": 0.010931654771169029}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.348563814476779, "sim_physics": 0.13807825247446695, "survival_time": 14.950000000000076, "driven_lanedir": 1.1548138859084167, "sim_render-ego": 0.014173417886098228, "in-drivable-lane": 5.650000000000037, "agent_compute-ego": 0.05532611052195231, "deviation-heading": 4.726102828454712, "set_robot_commands": 0.011206588745117188, "deviation-center-line": 0.6849022300274012, "driven_lanedir_consec": 0.8432428717673908, "sim_compute_sim_state": 0.005744022528330485, "sim_compute_performance-ego": 0.0084064515431722, "sim_compute_robot_state-ego": 0.01078283389409383}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.3487823912223305, "sim_physics": 0.13674669106801352, "survival_time": 14.950000000000076, "driven_lanedir": 1.7786337537427317, "sim_render-ego": 0.013798298835754397, "in-drivable-lane": 2.150000000000012, "agent_compute-ego": 0.05363793134689331, "deviation-heading": 4.712801082506641, "set_robot_commands": 0.011276258627573648, "deviation-center-line": 1.3652391647990982, "driven_lanedir_consec": 1.0952369036746763, "sim_compute_sim_state": 0.005693031152089437, "sim_compute_performance-ego": 0.0082782514890035, "sim_compute_robot_state-ego": 0.010653705596923828}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 2.348838140864166, "sim_physics": 0.12956636746724445, "survival_time": 14.950000000000076, "driven_lanedir": 1.8009517086564204, "sim_render-ego": 0.013897231419881185, "in-drivable-lane": 2.350000000000019, "agent_compute-ego": 0.055050406455993656, "deviation-heading": 4.123756486236227, "set_robot_commands": 0.01127261479695638, "deviation-center-line": 1.0063821513157902, "driven_lanedir_consec": 1.8009517086564204, "sim_compute_sim_state": 0.005737531185150147, "sim_compute_performance-ego": 0.008198107878367106, "sim_compute_robot_state-ego": 0.010606845219930014}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.4927656932510965, "sim_physics": 0.11533673370585722, "survival_time": 3.399999999999996, "driven_lanedir": 0.4443453865500677, "sim_render-ego": 0.01296993564156925, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.0563316730891957, "deviation-heading": 0.6754861188082931, "set_robot_commands": 0.009985012166640338, "deviation-center-line": 0.2530648837213237, "driven_lanedir_consec": 0.4443453865500677, "sim_compute_sim_state": 0.005600820569431081, "sim_compute_performance-ego": 0.007902632741367115, "sim_compute_robot_state-ego": 0.010052165564368751}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 1.320364700370299, "sim_physics": 0.1302205080177352, "survival_time": 8.549999999999986, "driven_lanedir": 0.9877157612172136, "sim_render-ego": 0.01377005326120477, "in-drivable-lane": 1.0500000000000016, "agent_compute-ego": 0.05910644196627433, "deviation-heading": 2.963737634498175, "set_robot_commands": 0.011327937332510252, "deviation-center-line": 0.5523737616876101, "driven_lanedir_consec": 0.9877157612172136, "sim_compute_sim_state": 0.005448860034608005, "sim_compute_performance-ego": 0.00802991961875157, "sim_compute_robot_state-ego": 0.01041754226238407}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 2.3491302755517376, "sim_physics": 0.1119143303235372, "survival_time": 14.950000000000076, "driven_lanedir": 2.290391782200912, "sim_render-ego": 0.013111793200174969, "in-drivable-lane": 0, "agent_compute-ego": 0.05195237239201864, "deviation-heading": 2.6845325878899864, "set_robot_commands": 0.010432971318562826, "deviation-center-line": 1.0182389405472094, "driven_lanedir_consec": 2.290391782200912, "sim_compute_sim_state": 0.005330619812011719, "sim_compute_performance-ego": 0.008001096248626709, "sim_compute_robot_state-ego": 0.009814192454020182}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.4365263712552368, "sim_physics": 0.12063977757438284, "survival_time": 3.049999999999997, "driven_lanedir": 0.4128912424386524, "sim_render-ego": 0.013071529200819672, "in-drivable-lane": 0, "agent_compute-ego": 0.055939811174986795, "deviation-heading": 0.6023684295151841, "set_robot_commands": 0.01087371247713683, "deviation-center-line": 0.23697101400436504, "driven_lanedir_consec": 0.4128912424386524, "sim_compute_sim_state": 0.005367865327928887, "sim_compute_performance-ego": 0.0077957481634421425, "sim_compute_robot_state-ego": 0.009690910089211384}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.1274481733446238, "sim_physics": 0.1308461046543251, "survival_time": 7.349999999999982, "driven_lanedir": 0.5741690844384523, "sim_render-ego": 0.01334922978667175, "in-drivable-lane": 2.499999999999991, "agent_compute-ego": 0.05534021384051057, "deviation-heading": 2.2723314548599567, "set_robot_commands": 0.010617765439610904, "deviation-center-line": 0.3808251708234354, "driven_lanedir_consec": 0.35224954999072455, "sim_compute_sim_state": 0.005493353824226224, "sim_compute_performance-ego": 0.007962408519926526, "sim_compute_robot_state-ego": 0.010280325299217588}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.5329471629740984, "sim_physics": 0.13322192022245224, "survival_time": 3.649999999999995, "driven_lanedir": 0.4530716531806259, "sim_render-ego": 0.0141086317088506, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.06172322247126331, "deviation-heading": 0.789709534894425, "set_robot_commands": 0.010647202191287524, "deviation-center-line": 0.2420679738650172, "driven_lanedir_consec": 0.4530716531806259, "sim_compute_sim_state": 0.005469560623168945, "sim_compute_performance-ego": 0.008741493094457339, "sim_compute_robot_state-ego": 0.010751978991782828}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.5569206245359362, "sim_physics": 0.1299416924777784, "survival_time": 3.7999999999999945, "driven_lanedir": 0.27197375353801756, "sim_render-ego": 0.013742553560357343, "in-drivable-lane": 1.0999999999999972, "agent_compute-ego": 0.05553372910148219, "deviation-heading": 1.5143662825010529, "set_robot_commands": 0.011051156018909656, "deviation-center-line": 0.2588843247029263, "driven_lanedir_consec": 0.08206782941699886, "sim_compute_sim_state": 0.0053539715315166275, "sim_compute_performance-ego": 0.008249298522346899, "sim_compute_robot_state-ego": 0.010714044696406314}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 2.3489519202237608, "sim_physics": 0.12593108177185058, "survival_time": 14.950000000000076, "driven_lanedir": 2.12609834184759, "sim_render-ego": 0.013548635641733805, "in-drivable-lane": 0.20000000000000284, "agent_compute-ego": 0.05590847094853719, "deviation-heading": 4.6266018016332815, "set_robot_commands": 0.011256778240203855, "deviation-center-line": 0.560801746349332, "driven_lanedir_consec": 2.046689731652523, "sim_compute_sim_state": 0.005510522524515788, "sim_compute_performance-ego": 0.007852428754170736, "sim_compute_robot_state-ego": 0.010129101276397705}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.3490418636895245, "sim_physics": 0.11175414641698202, "survival_time": 14.950000000000076, "driven_lanedir": 2.001919569641603, "sim_render-ego": 0.01319756825764974, "in-drivable-lane": 1.4500000000000206, "agent_compute-ego": 0.0509572426478068, "deviation-heading": 2.942215210088024, "set_robot_commands": 0.01056382417678833, "deviation-center-line": 1.1135493992751542, "driven_lanedir_consec": 1.9604102700637236, "sim_compute_sim_state": 0.005343918005625407, "sim_compute_performance-ego": 0.007582077980041504, "sim_compute_robot_state-ego": 0.009594943523406984}}
set_robot_commands_max0.01240842561332547
set_robot_commands_mean0.011022237151391842
set_robot_commands_median0.011082774714419716
set_robot_commands_min0.009985012166640338
sim_compute_performance-ego_max0.008741493094457339
sim_compute_performance-ego_mean0.008142139028411247
sim_compute_performance-ego_median0.008182710722873085
sim_compute_performance-ego_min0.007582077980041504
sim_compute_robot_state-ego_max0.010931654771169029
sim_compute_robot_state-ego_mean0.01036013566850133
sim_compute_robot_state-ego_median0.01041754226238407
sim_compute_robot_state-ego_min0.009594943523406984
sim_compute_sim_state_max0.00614825169245402
sim_compute_sim_state_mean0.005591115732686817
sim_compute_sim_state_median0.005510522524515788
sim_compute_sim_state_min0.005330619812011719
sim_physics_max0.1412779672940572
sim_physics_mean0.1277654740640958
sim_physics_median0.1299416924777784
sim_physics_min0.11175414641698202
sim_render-ego_max0.014495846430460611
sim_render-ego_mean0.013702358348334848
sim_render-ego_median0.01377005326120477
sim_render-ego_min0.01296993564156925
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean9.870000000000031
survival_time_min3.049999999999997
No reset possible
248824266Gianmarco Bernasconichallenge-aido_LF-baseline-duckietownaido3-LFVI-sim-testingstep1-simulationerrornoip-172-31-46-148-104910:02:32
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
248374341Soroush Saryazdi 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LFV-sim-testingstep1-simulationsuccessnoip-172-31-46-148-104910:56:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8862840843448361
survival_time_median10.000000000000009
deviation-center-line_median0.5846364835738153
in-drivable-lane_median1.1000000000000003


other stats
agent_compute-ego_max0.06170825277056013
agent_compute-ego_mean0.05531375885102038
agent_compute-ego_median0.05546312282482783
agent_compute-ego_min0.04794940233230591
deviation-center-line_max1.0932634329365245
deviation-center-line_mean0.6223568244549624
deviation-center-line_min0.16129474716966818
deviation-heading_max7.813658978546615
deviation-heading_mean3.590404412575387
deviation-heading_median3.122100892750523
deviation-heading_min0.9098999041225946
driven_any_max2.3491185282991975
driven_any_mean1.612258089812763
driven_any_median1.5533869952443986
driven_any_min0.5088212213405976
driven_lanedir_consec_max2.2774614743122026
driven_lanedir_consec_mean0.9474811904746612
driven_lanedir_consec_min0.07951488839818444
driven_lanedir_max2.2774614743122026
driven_lanedir_mean1.1328911764545633
driven_lanedir_median1.079754839825923
driven_lanedir_min0.11817818508807364
in-drivable-lane_max5.600000000000042
in-drivable-lane_mean1.7533333333333414
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.5654636502311637, "sim_physics": 0.24719092479118937, "survival_time": 3.899999999999994, "driven_lanedir": 0.4626682213264473, "sim_render-ego": 0.014546039776924327, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.05347887369302603, "deviation-heading": 1.3831301794402842, "set_robot_commands": 0.012565307128123747, "deviation-center-line": 0.16172222880498255, "driven_lanedir_consec": 0.4626682213264473, "sim_compute_sim_state": 0.006326039632161458, "sim_compute_performance-ego": 0.008534847161708733, "sim_compute_robot_state-ego": 0.011475550822722606, "sim_compute_robot_state-npc0": 0.010916795486058945, "sim_compute_robot_state-npc1": 0.010072720356476612, "sim_compute_robot_state-npc2": 0.010093150994716546, "sim_compute_robot_state-npc3": 0.009657942331754245}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.3491185282991975, "sim_physics": 0.24813685417175296, "survival_time": 14.950000000000076, "driven_lanedir": 2.2774614743122026, "sim_render-ego": 0.014144864082336426, "in-drivable-lane": 0, "agent_compute-ego": 0.05669626951217652, "deviation-heading": 3.122100892750523, "set_robot_commands": 0.011812502543131513, "deviation-center-line": 0.5846364835738153, "driven_lanedir_consec": 2.2774614743122026, "sim_compute_sim_state": 0.005822263558705648, "sim_compute_performance-ego": 0.00868233601252238, "sim_compute_robot_state-ego": 0.010819030602773031, "sim_compute_robot_state-npc0": 0.010555925369262696, "sim_compute_robot_state-npc1": 0.01040256182352702, "sim_compute_robot_state-npc2": 0.01026532808939616, "sim_compute_robot_state-npc3": 0.010331791241963704}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.1757956809971395, "sim_physics": 0.2254659007577335, "survival_time": 7.649999999999981, "driven_lanedir": 1.079754839825923, "sim_render-ego": 0.013995650547002657, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.058679284613116896, "deviation-heading": 1.2569770476660935, "set_robot_commands": 0.011559154473099052, "deviation-center-line": 0.6749393222605576, "driven_lanedir_consec": 1.079754839825923, "sim_compute_sim_state": 0.005826167810976115, "sim_compute_performance-ego": 0.00867899258931478, "sim_compute_robot_state-ego": 0.010803567038642036, "sim_compute_robot_state-npc0": 0.010356621025434509, "sim_compute_robot_state-npc1": 0.010203215031842, "sim_compute_robot_state-npc2": 0.010033312965841854, "sim_compute_robot_state-npc3": 0.010071385140512505}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.2602972453287236, "sim_physics": 0.23283531268437704, "survival_time": 14.40000000000007, "driven_lanedir": 1.2071441059084431, "sim_render-ego": 0.013834358917342292, "in-drivable-lane": 3.900000000000008, "agent_compute-ego": 0.05546312282482783, "deviation-heading": 6.3966555569584305, "set_robot_commands": 0.011008638474676345, "deviation-center-line": 0.8758498268713399, "driven_lanedir_consec": 0.8258055860853735, "sim_compute_sim_state": 0.005766547388500637, "sim_compute_performance-ego": 0.008198791080050997, "sim_compute_robot_state-ego": 0.010561871031920116, "sim_compute_robot_state-npc0": 0.010166915754477182, "sim_compute_robot_state-npc1": 0.010079901251528, "sim_compute_robot_state-npc2": 0.00989053895076116, "sim_compute_robot_state-npc3": 0.009866720272435082}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.3083876371059624, "sim_physics": 0.1976914576121739, "survival_time": 14.700000000000074, "driven_lanedir": 1.060874247915598, "sim_render-ego": 0.013156144797396498, "in-drivable-lane": 5.600000000000042, "agent_compute-ego": 0.05480160842947408, "deviation-heading": 5.111625386654195, "set_robot_commands": 0.010316347589298173, "deviation-center-line": 0.5728713687824695, "driven_lanedir_consec": 0.7702507300146403, "sim_compute_sim_state": 0.005452929710855289, "sim_compute_performance-ego": 0.008000209217979795, "sim_compute_robot_state-ego": 0.01053562456247758, "sim_compute_robot_state-npc0": 0.009728883399444372, "sim_compute_robot_state-npc1": 0.009186004294830116, "sim_compute_robot_state-npc2": 0.009138447897774833, "sim_compute_robot_state-npc3": 0.009201537995111374}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 2.3486795797575533, "sim_physics": 0.21314777612686156, "survival_time": 14.950000000000076, "driven_lanedir": 1.1688544607054707, "sim_render-ego": 0.014201470216115316, "in-drivable-lane": 5.350000000000059, "agent_compute-ego": 0.05667517423629761, "deviation-heading": 5.5040216281124295, "set_robot_commands": 0.010769485632578532, "deviation-center-line": 0.955863796138068, "driven_lanedir_consec": 0.8862840843448361, "sim_compute_sim_state": 0.00553653081258138, "sim_compute_performance-ego": 0.008321096897125244, "sim_compute_robot_state-ego": 0.010646138985951742, "sim_compute_robot_state-npc0": 0.009902897675832112, "sim_compute_robot_state-npc1": 0.009808812936147054, "sim_compute_robot_state-npc2": 0.00940454880396525, "sim_compute_robot_state-npc3": 0.00955575704574585}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.5088212213405976, "sim_physics": 0.2233640330178397, "survival_time": 3.4999999999999956, "driven_lanedir": 0.4588915743644485, "sim_render-ego": 0.014714997155325753, "in-drivable-lane": 0, "agent_compute-ego": 0.06170825277056013, "deviation-heading": 0.9098999041225946, "set_robot_commands": 0.011013620240347726, "deviation-center-line": 0.19884634689416153, "driven_lanedir_consec": 0.4588915743644485, "sim_compute_sim_state": 0.006045917102268764, "sim_compute_performance-ego": 0.008567234448024205, "sim_compute_robot_state-ego": 0.011192454610552106, "sim_compute_robot_state-npc0": 0.010503510066441128, "sim_compute_robot_state-npc1": 0.01009678840637207, "sim_compute_robot_state-npc2": 0.00986675534929548, "sim_compute_robot_state-npc3": 0.010643236977713449}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.5729768530985893, "sim_physics": 0.20411387162330824, "survival_time": 3.899999999999994, "driven_lanedir": 0.11817818508807364, "sim_render-ego": 0.01460096163627429, "in-drivable-lane": 1.999999999999997, "agent_compute-ego": 0.05649585601611015, "deviation-heading": 1.6962864229229284, "set_robot_commands": 0.010910419317392202, "deviation-center-line": 0.16129474716966818, "driven_lanedir_consec": 0.07951488839818444, "sim_compute_sim_state": 0.00602885087331136, "sim_compute_performance-ego": 0.008522969025831956, "sim_compute_robot_state-ego": 0.01054244775038499, "sim_compute_robot_state-npc0": 0.010267214897351388, "sim_compute_robot_state-npc1": 0.010114012620387932, "sim_compute_robot_state-npc2": 0.010130851696699094, "sim_compute_robot_state-npc3": 0.010012327096401116}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.119348559481373, "sim_physics": 0.2120178284710401, "survival_time": 7.299999999999982, "driven_lanedir": 0.6647596977761276, "sim_render-ego": 0.014397866105380124, "in-drivable-lane": 1.4499999999999964, "agent_compute-ego": 0.05612941310830312, "deviation-heading": 3.693824240704581, "set_robot_commands": 0.011878372871712464, "deviation-center-line": 0.5424336415781857, "driven_lanedir_consec": 0.34590668245447587, "sim_compute_sim_state": 0.006311575027361308, "sim_compute_performance-ego": 0.00831956732763003, "sim_compute_robot_state-ego": 0.010872210541816607, "sim_compute_robot_state-npc0": 0.010649367554547037, "sim_compute_robot_state-npc1": 0.010747305334430851, "sim_compute_robot_state-npc2": 0.010533143396246922, "sim_compute_robot_state-npc3": 0.010415247041885166}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 2.3490677817863643, "sim_physics": 0.2138987890879313, "survival_time": 14.950000000000076, "driven_lanedir": 2.275973439510677, "sim_render-ego": 0.013921972115834551, "in-drivable-lane": 0, "agent_compute-ego": 0.05368628819783529, "deviation-heading": 3.15639607115372, "set_robot_commands": 0.010951786835988363, "deviation-center-line": 0.8446895024806419, "driven_lanedir_consec": 2.275973439510677, "sim_compute_sim_state": 0.005579359531402588, "sim_compute_performance-ego": 0.008107173442840575, "sim_compute_robot_state-ego": 0.010573218663533527, "sim_compute_robot_state-npc0": 0.009820166428883873, "sim_compute_robot_state-npc1": 0.009685720602671304, "sim_compute_robot_state-npc2": 0.009613954226175944, "sim_compute_robot_state-npc3": 0.009388131300608315}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.3123418350146188, "sim_physics": 0.19524587743422564, "survival_time": 8.499999999999986, "driven_lanedir": 1.0640037284226251, "sim_render-ego": 0.013734775431015912, "in-drivable-lane": 0.7500000000000027, "agent_compute-ego": 0.059015324536491845, "deviation-heading": 2.482462980315683, "set_robot_commands": 0.011534967141992904, "deviation-center-line": 0.4624908291334815, "driven_lanedir_consec": 1.0640037284226251, "sim_compute_sim_state": 0.005579043837154613, "sim_compute_performance-ego": 0.007927500500398523, "sim_compute_robot_state-ego": 0.010459320685442756, "sim_compute_robot_state-npc0": 0.009894239201265222, "sim_compute_robot_state-npc1": 0.009554223453297334, "sim_compute_robot_state-npc2": 0.009584255779490751, "sim_compute_robot_state-npc3": 0.009941395591287053}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.5533869952443986, "sim_physics": 0.1828316307067871, "survival_time": 10.000000000000009, "driven_lanedir": 1.2451871267876893, "sim_render-ego": 0.013712304830551149, "in-drivable-lane": 1.1000000000000003, "agent_compute-ego": 0.05542068958282471, "deviation-heading": 2.8525986938832197, "set_robot_commands": 0.01037518858909607, "deviation-center-line": 0.7371704244270239, "driven_lanedir_consec": 1.0623082759244626, "sim_compute_sim_state": 0.005490734577178955, "sim_compute_performance-ego": 0.008119803667068482, "sim_compute_robot_state-ego": 0.010213865041732788, "sim_compute_robot_state-npc0": 0.009332643747329712, "sim_compute_robot_state-npc1": 0.009268442392349243, "sim_compute_robot_state-npc2": 0.00921963095664978, "sim_compute_robot_state-npc3": 0.00949203372001648}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 2.348377115115607, "sim_physics": 0.1772332771619161, "survival_time": 14.950000000000076, "driven_lanedir": 1.3677627958088296, "sim_render-ego": 0.011567265192667642, "in-drivable-lane": 3.1000000000000103, "agent_compute-ego": 0.05049855470657349, "deviation-heading": 7.813658978546615, "set_robot_commands": 0.009043556849161784, "deviation-center-line": 1.0932634329365245, "driven_lanedir_consec": 0.6008182388479578, "sim_compute_sim_state": 0.005052131017049153, "sim_compute_performance-ego": 0.006926422119140625, "sim_compute_robot_state-ego": 0.00863553524017334, "sim_compute_robot_state-npc0": 0.008517820040384929, "sim_compute_robot_state-npc1": 0.008518420855204264, "sim_compute_robot_state-npc2": 0.008218708833058676, "sim_compute_robot_state-npc3": 0.008230535984039307}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 1.0633387468557975, "sim_physics": 0.18259823922630697, "survival_time": 6.949999999999983, "driven_lanedir": 1.0101220943871332, "sim_render-ego": 0.013122267002682032, "in-drivable-lane": 0, "agent_compute-ego": 0.053008268205381986, "deviation-heading": 1.3443047906016146, "set_robot_commands": 0.010180553943990804, "deviation-center-line": 0.5384555250616788, "driven_lanedir_consec": 1.0101220943871332, "sim_compute_sim_state": 0.005330835314963362, "sim_compute_performance-ego": 0.007957038261907563, "sim_compute_robot_state-ego": 0.00981274440134172, "sim_compute_robot_state-npc0": 0.00963459598074714, "sim_compute_robot_state-npc1": 0.009197585016703437, "sim_compute_robot_state-npc2": 0.009484498620890885, "sim_compute_robot_state-npc3": 0.009429501114989356}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.3484699175343597, "sim_physics": 0.1758361045519511, "survival_time": 14.950000000000076, "driven_lanedir": 1.5317316546787576, "sim_render-ego": 0.011862157185872396, "in-drivable-lane": 2.500000000000004, "agent_compute-ego": 0.04794940233230591, "deviation-heading": 7.132123414797906, "set_robot_commands": 0.009221571286519369, "deviation-center-line": 0.930824890711836, "driven_lanedir_consec": 1.0124539989005306, "sim_compute_sim_state": 0.004961798191070557, "sim_compute_performance-ego": 0.006913255055745443, "sim_compute_robot_state-ego": 0.008867402076721192, "sim_compute_robot_state-npc0": 0.00858306884765625, "sim_compute_robot_state-npc1": 0.008603248596191406, "sim_compute_robot_state-npc2": 0.008552733262379965, "sim_compute_robot_state-npc3": 0.008571681181589762}}
set_robot_commands_max0.012565307128123747
set_robot_commands_mean0.010876098194473938
set_robot_commands_median0.010951786835988363
set_robot_commands_min0.009043556849161784
sim_compute_performance-ego_max0.00868233601252238
sim_compute_performance-ego_mean0.00811848245381929
sim_compute_performance-ego_median0.008198791080050997
sim_compute_performance-ego_min0.006913255055745443
sim_compute_robot_state-ego_max0.011475550822722606
sim_compute_robot_state-ego_mean0.010400732137079077
sim_compute_robot_state-ego_median0.010561871031920116
sim_compute_robot_state-ego_min0.00863553524017334
sim_compute_robot_state-npc0_max0.010916795486058945
sim_compute_robot_state-npc0_mean0.009922044365007768
sim_compute_robot_state-npc0_median0.009902897675832112
sim_compute_robot_state-npc0_min0.008517820040384929
sim_compute_robot_state-npc1_max0.010747305334430851
sim_compute_robot_state-npc1_mean0.00970259753146391
sim_compute_robot_state-npc1_median0.009808812936147054
sim_compute_robot_state-npc1_min0.008518420855204264
sim_compute_robot_state-npc2_max0.010533143396246922
sim_compute_robot_state-npc2_mean0.009601990654889554
sim_compute_robot_state-npc2_median0.009613954226175944
sim_compute_robot_state-npc2_min0.008218708833058676
sim_compute_robot_state-npc3_max0.010643236977713449
sim_compute_robot_state-npc3_mean0.009653948269070184
sim_compute_robot_state-npc3_median0.009657942331754245
sim_compute_robot_state-npc3_min0.008230535984039307
sim_compute_sim_state_max0.006326039632161458
sim_compute_sim_state_mean0.005674048292369412
sim_compute_sim_state_median0.005579359531402588
sim_compute_sim_state_min0.004961798191070557
sim_physics_max0.24813685417175296
sim_physics_mean0.2087738584950263
sim_physics_median0.2120178284710401
sim_physics_min0.1758361045519511
sim_render-ego_max0.014714997155325753
sim_render-ego_mean0.013700872999514757
sim_render-ego_median0.013921972115834551
sim_render-ego_min0.011567265192667642
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean10.37000000000003
survival_time_min3.4999999999999956
No reset possible
248224330Soroush Saryazdi 🇨🇦challenge-aido_LF-template-randomaido3-LF-sim-validationstep1-simulationerrornoip-172-31-46-148-104910:05:16
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
248084342Soroush Saryazdi 🇨🇦challenge-aido_LF-baseline-duckietownaido3-LFV-sim-validationstep1-simulationsuccessnoip-172-31-46-148-104910:16:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.7593261167653771
survival_time_median7.549999999999981
deviation-center-line_median0.5463154156144918
in-drivable-lane_median0.2499999999999991


other stats
agent_compute-ego_max0.0605040294783456
agent_compute-ego_mean0.05671622381714331
agent_compute-ego_median0.05522258658158152
agent_compute-ego_min0.053847319040543
deviation-center-line_max0.7864876086459238
deviation-center-line_mean0.536127262635569
deviation-center-line_min0.20815115690801525
deviation-heading_max4.596894351686314
deviation-heading_mean2.474944702937073
deviation-heading_median1.471783980681353
deviation-heading_min1.1237703608212988
driven_any_max1.8265075936005488
driven_any_mean1.246479810570127
driven_any_median1.1597612947540112
driven_any_min0.5493824397617497
driven_lanedir_consec_max1.0696157695934745
driven_lanedir_consec_mean0.7657979406612869
driven_lanedir_consec_min0.48196995009635746
driven_lanedir_max1.4563858828377056
driven_lanedir_mean1.0334842344198336
driven_lanedir_median1.0696157695934745
driven_lanedir_min0.48196995009635746
in-drivable-lane_max1.450000000000001
in-drivable-lane_mean0.5700000000000006
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.5493824397617497, "sim_physics": 0.20993374523363617, "survival_time": 3.7999999999999945, "driven_lanedir": 0.48196995009635746, "sim_render-ego": 0.014536424687034205, "in-drivable-lane": 0.04999999999999982, "agent_compute-ego": 0.059494504803105405, "deviation-heading": 1.1299343881298602, "set_robot_commands": 0.010821420895425897, "deviation-center-line": 0.20815115690801525, "driven_lanedir_consec": 0.48196995009635746, "sim_compute_sim_state": 0.005924359748238011, "sim_compute_performance-ego": 0.008672927555285002, "sim_compute_robot_state-ego": 0.010601523675416646, "sim_compute_robot_state-npc0": 0.010255722623122366, "sim_compute_robot_state-npc1": 0.01007600834495143, "sim_compute_robot_state-npc2": 0.009656821426592375, "sim_compute_robot_state-npc3": 0.0095996103788677}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.8265075936005488, "sim_physics": 0.18745794663062465, "survival_time": 11.700000000000031, "driven_lanedir": 1.4563858828377056, "sim_render-ego": 0.013502309465000773, "in-drivable-lane": 1.1000000000000032, "agent_compute-ego": 0.053847319040543, "deviation-heading": 4.052340433366537, "set_robot_commands": 0.010204280543531109, "deviation-center-line": 0.7864876086459238, "driven_lanedir_consec": 0.7593261167653771, "sim_compute_sim_state": 0.005466660882672693, "sim_compute_performance-ego": 0.007496406889369345, "sim_compute_robot_state-ego": 0.010120425468836076, "sim_compute_robot_state-npc0": 0.009507923044710076, "sim_compute_robot_state-npc1": 0.009224828491863022, "sim_compute_robot_state-npc2": 0.009348649245042069, "sim_compute_robot_state-npc3": 0.00917966854877961}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.1597612947540112, "sim_physics": 0.17215626760823838, "survival_time": 7.549999999999981, "driven_lanedir": 1.0696157695934745, "sim_render-ego": 0.01304667043370127, "in-drivable-lane": 0.2499999999999991, "agent_compute-ego": 0.05451267918214103, "deviation-heading": 1.471783980681353, "set_robot_commands": 0.009288606264733322, "deviation-center-line": 0.5051037045616807, "driven_lanedir_consec": 1.0696157695934745, "sim_compute_sim_state": 0.005516050667162762, "sim_compute_performance-ego": 0.007689090754022661, "sim_compute_robot_state-ego": 0.00988398798254152, "sim_compute_robot_state-npc0": 0.00902128377497591, "sim_compute_robot_state-npc1": 0.009337557862136536, "sim_compute_robot_state-npc2": 0.008834110980002296, "sim_compute_robot_state-npc3": 0.008889915137890949}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.0713533110821691, "sim_physics": 0.19249282905033657, "survival_time": 6.999999999999983, "driven_lanedir": 1.032967878645207, "sim_render-ego": 0.014295532022203718, "in-drivable-lane": 0, "agent_compute-ego": 0.0605040294783456, "deviation-heading": 1.1237703608212988, "set_robot_commands": 0.010619556903839112, "deviation-center-line": 0.5463154156144918, "driven_lanedir_consec": 1.032967878645207, "sim_compute_sim_state": 0.005684895174843925, "sim_compute_performance-ego": 0.008151052679334367, "sim_compute_robot_state-ego": 0.011007937363215856, "sim_compute_robot_state-npc0": 0.009675499371119908, "sim_compute_robot_state-npc1": 0.00979950087411063, "sim_compute_robot_state-npc2": 0.009279189790998186, "sim_compute_robot_state-npc3": 0.009499178613935198}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.625394413652156, "sim_physics": 0.1810205747065932, "survival_time": 10.450000000000014, "driven_lanedir": 1.126481690926424, "sim_render-ego": 0.013461695903796329, "in-drivable-lane": 1.450000000000001, "agent_compute-ego": 0.05522258658158152, "deviation-heading": 4.596894351686314, "set_robot_commands": 0.009680089768040124, "deviation-center-line": 0.6345784274477331, "driven_lanedir_consec": 0.4851099882060189, "sim_compute_sim_state": 0.005444915671097605, "sim_compute_performance-ego": 0.007505657570213793, "sim_compute_robot_state-ego": 0.010268084740524656, "sim_compute_robot_state-npc0": 0.009355189127214787, "sim_compute_robot_state-npc1": 0.009295785255979691, "sim_compute_robot_state-npc2": 0.009136982511675528, "sim_compute_robot_state-npc3": 0.009089102585349926}}
set_robot_commands_max0.010821420895425897
set_robot_commands_mean0.010122790875113911
set_robot_commands_median0.010204280543531109
set_robot_commands_min0.009288606264733322
sim_compute_performance-ego_max0.008672927555285002
sim_compute_performance-ego_mean0.007903027089645033
sim_compute_performance-ego_median0.007689090754022661
sim_compute_performance-ego_min0.007496406889369345
sim_compute_robot_state-ego_max0.011007937363215856
sim_compute_robot_state-ego_mean0.01037639184610695
sim_compute_robot_state-ego_median0.010268084740524656
sim_compute_robot_state-ego_min0.00988398798254152
sim_compute_robot_state-npc0_max0.010255722623122366
sim_compute_robot_state-npc0_mean0.00956312358822861
sim_compute_robot_state-npc0_median0.009507923044710076
sim_compute_robot_state-npc0_min0.00902128377497591
sim_compute_robot_state-npc1_max0.01007600834495143
sim_compute_robot_state-npc1_mean0.009546736165808265
sim_compute_robot_state-npc1_median0.009337557862136536
sim_compute_robot_state-npc1_min0.009224828491863022
sim_compute_robot_state-npc2_max0.009656821426592375
sim_compute_robot_state-npc2_mean0.00925115079086209
sim_compute_robot_state-npc2_median0.009279189790998186
sim_compute_robot_state-npc2_min0.008834110980002296
sim_compute_robot_state-npc3_max0.0095996103788677
sim_compute_robot_state-npc3_mean0.009251495052964676
sim_compute_robot_state-npc3_median0.00917966854877961
sim_compute_robot_state-npc3_min0.008889915137890949
sim_compute_sim_state_max0.005924359748238011
sim_compute_sim_state_mean0.005607376428802999
sim_compute_sim_state_median0.005516050667162762
sim_compute_sim_state_min0.005444915671097605
sim_physics_max0.20993374523363617
sim_physics_mean0.1886122726458858
sim_physics_median0.18745794663062465
sim_physics_min0.17215626760823838
sim_render-ego_max0.014536424687034205
sim_render-ego_mean0.01376852650234726
sim_render-ego_median0.013502309465000773
sim_render-ego_min0.01304667043370127
simulation-passed1
survival_time_max11.700000000000031
survival_time_mean8.1
survival_time_min3.7999999999999945
No reset possible
247904329Soroush Saryazdi 🇨🇦challenge-aido_LF-template-randomaido3-LF-sim-testingstep1-simulationsuccessnoip-172-31-46-148-104910:12:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.10099507669212504
in-drivable-lane_median0.4499999999999984


other stats
agent_compute-ego_max0.026175750626458064
agent_compute-ego_mean0.022791043983084205
agent_compute-ego_median0.022655055636451357
agent_compute-ego_min0.02120142512851291
deviation-center-line_max0.28386357003337986
deviation-center-line_mean0.12786165368054708
deviation-center-line_min0.07112529346797256
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3391682685652288
deviation-heading_median0.3292838510353387
deviation-heading_min0.19401296908406307
driven_any_max2.1706243860917107
driven_any_mean0.975949037827615
driven_any_median1.0117660515775695
driven_any_min0.23626560261560617
driven_lanedir_consec_max1.919400421063861
driven_lanedir_consec_mean0.6896171461819268
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max1.919400421063861
driven_lanedir_mean0.6896171461819268
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.5899999999999982
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429704, "sim_physics": 0.1155761789392542, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.012640268714339644, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.026175750626458064, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.010071524867305047, "deviation-center-line": 0.15907928712407876, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.005957802136739095, "sim_compute_performance-ego": 0.007475384959468135, "sim_compute_robot_state-ego": 0.009637969511526602}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.10241924547681622, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.011756602455587947, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.02289233020707673, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.008918635985430549, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.0048766089420692596, "sim_compute_performance-ego": 0.006721978094063553, "sim_compute_robot_state-ego": 0.008389351414699181}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.09921431541442872, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.012467286166022806, "in-drivable-lane": 0, "agent_compute-ego": 0.02301439116982853, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.00915241241455078, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.005224213880651137, "sim_compute_performance-ego": 0.006980685626759249, "sim_compute_robot_state-ego": 0.00991675432990579}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.11972984313964843, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.013465576171875, "in-drivable-lane": 0, "agent_compute-ego": 0.02389580726623535, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.00962005615234375, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.005139980316162109, "sim_compute_performance-ego": 0.007070713043212891, "sim_compute_robot_state-ego": 0.009925365447998049}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.6182144577009443, "sim_physics": 0.10059669047971316, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.011829288699958898, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.022798677034015898, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.008103992365583589, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.00484091722512547, "sim_compute_performance-ego": 0.006514679027509086, "sim_compute_robot_state-ego": 0.008877265302440788}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.003851495683299, "sim_physics": 0.1074211548785774, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7531703123735949, "sim_render-ego": 0.012750246086899114, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.0237325162303691, "deviation-heading": 0.4596437246381204, "set_robot_commands": 0.00920540459302007, "deviation-center-line": 0.08968283443958694, "driven_lanedir_consec": 0.7531703123735949, "sim_compute_sim_state": 0.005666401921486368, "sim_compute_performance-ego": 0.007560569412854253, "sim_compute_robot_state-ego": 0.009999576880007374}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0542898001504386, "sim_physics": 0.09628665021487644, "survival_time": 2.799999999999998, "driven_lanedir": 0.5389398655375865, "sim_render-ego": 0.012328846114022391, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.02198027287210737, "deviation-heading": 0.4109275195546752, "set_robot_commands": 0.008823573589324951, "deviation-center-line": 0.10786249131209884, "driven_lanedir_consec": 0.5389398655375865, "sim_compute_sim_state": 0.004952358348029, "sim_compute_performance-ego": 0.006680050066539219, "sim_compute_robot_state-ego": 0.009026697703770228}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3164475681403097, "sim_physics": 0.11143716176350912, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3093025166124813, "sim_render-ego": 0.012141931624639602, "in-drivable-lane": 0, "agent_compute-ego": 0.022655055636451357, "deviation-heading": 0.2242052814634115, "set_robot_commands": 0.00958887736002604, "deviation-center-line": 0.09758615464743424, "driven_lanedir_consec": 0.3093025166124813, "sim_compute_sim_state": 0.005600622722080776, "sim_compute_performance-ego": 0.006771859668550037, "sim_compute_robot_state-ego": 0.008905717304774694}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2018528489510794, "sim_physics": 0.08604759685063766, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6922532395203, "sim_render-ego": 0.010717189918130131, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.02232668763500149, "deviation-heading": 0.3292838510353387, "set_robot_commands": 0.00837284831677453, "deviation-center-line": 0.07112529346797256, "driven_lanedir_consec": 0.6922532395203, "sim_compute_sim_state": 0.004643698870125464, "sim_compute_performance-ego": 0.006080150604248047, "sim_compute_robot_state-ego": 0.008012654417652195}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.043548155325732, "sim_physics": 0.09552513469349254, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5204405548974873, "sim_render-ego": 0.011804485321044922, "in-drivable-lane": 1.1499999999999977, "agent_compute-ego": 0.02318804047324441, "deviation-heading": 0.33296447766287096, "set_robot_commands": 0.008219099044799805, "deviation-center-line": 0.0950267943453044, "driven_lanedir_consec": 0.5204405548974873, "sim_compute_sim_state": 0.004843898253007369, "sim_compute_performance-ego": 0.006495969945734197, "sim_compute_robot_state-ego": 0.008456880396062677}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5954548672193285, "sim_physics": 0.08774019526196765, "survival_time": 3.8499999999999943, "driven_lanedir": 1.1042463557848785, "sim_render-ego": 0.012377485052331702, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.02253342294073724, "deviation-heading": 0.35668107106767694, "set_robot_commands": 0.008126200019539176, "deviation-center-line": 0.2251391443608223, "driven_lanedir_consec": 1.1042463557848785, "sim_compute_sim_state": 0.004858964449399477, "sim_compute_performance-ego": 0.006709331041806704, "sim_compute_robot_state-ego": 0.009081917923766298}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.49030923107489305, "sim_physics": 0.1092690910611834, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4826393189617785, "sim_render-ego": 0.011940394129071916, "in-drivable-lane": 0, "agent_compute-ego": 0.021990605763026645, "deviation-heading": 0.23322336656472228, "set_robot_commands": 0.008105618613106864, "deviation-center-line": 0.10099507669212504, "driven_lanedir_consec": 0.4826393189617785, "sim_compute_sim_state": 0.004514387675694057, "sim_compute_performance-ego": 0.007100113800593785, "sim_compute_robot_state-ego": 0.009091292108808244}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0117660515775695, "sim_physics": 0.08514023294635847, "survival_time": 2.549999999999999, "driven_lanedir": 0.791717039833255, "sim_render-ego": 0.01154938866110409, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.022213117749083276, "deviation-heading": 0.320449356593834, "set_robot_commands": 0.00779868574703441, "deviation-center-line": 0.1017166571540705, "driven_lanedir_consec": 0.791717039833255, "sim_compute_sim_state": 0.004885374331006817, "sim_compute_performance-ego": 0.006470194049910003, "sim_compute_robot_state-ego": 0.008458436704149433}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.44419823353351984, "sim_physics": 0.08914061828895851, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4333246859278659, "sim_render-ego": 0.011211015560008862, "in-drivable-lane": 0, "agent_compute-ego": 0.02120142512851291, "deviation-heading": 0.2988926743481924, "set_robot_commands": 0.00813797668174461, "deviation-center-line": 0.08545702330592322, "driven_lanedir_consec": 0.4333246859278659, "sim_compute_sim_state": 0.004635501790929724, "sim_compute_performance-ego": 0.0065794432604754415, "sim_compute_robot_state-ego": 0.008198861722592954}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1706243860917107, "sim_physics": 0.07919084090812534, "survival_time": 5.09999999999999, "driven_lanedir": 1.919400421063861, "sim_render-ego": 0.010721248738905963, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.02126755901411468, "deviation-heading": 0.5178939525116448, "set_robot_commands": 0.007756511370340983, "deviation-center-line": 0.28386357003337986, "driven_lanedir_consec": 1.919400421063861, "sim_compute_sim_state": 0.004482129040886374, "sim_compute_performance-ego": 0.006209333737691243, "sim_compute_robot_state-ego": 0.007740518626044779}}
set_robot_commands_max0.010071524867305047
set_robot_commands_mean0.00866676114139501
set_robot_commands_median0.00837284831677453
set_robot_commands_min0.007756511370340983
sim_compute_performance-ego_max0.007560569412854253
sim_compute_performance-ego_mean0.006761363755961056
sim_compute_performance-ego_median0.006709331041806704
sim_compute_performance-ego_min0.006080150604248047
sim_compute_robot_state-ego_max0.009999576880007374
sim_compute_robot_state-ego_mean0.008914617319613285
sim_compute_robot_state-ego_median0.008905717304774694
sim_compute_robot_state-ego_min0.007740518626044779
sim_compute_sim_state_max0.005957802136739095
sim_compute_sim_state_mean0.005008190660226167
sim_compute_sim_state_median0.0048766089420692596
sim_compute_sim_state_min0.004482129040886374
sim_physics_max0.11972984313964843
sim_physics_mean0.0989823300211698
sim_physics_median0.09921431541442872
sim_physics_min0.07919084090812534
sim_render-ego_max0.013465576171875
sim_render-ego_mean0.011980083560929533
sim_render-ego_median0.011940394129071916
sim_render-ego_min0.010717189918130131
simulation-passed1
survival_time_max5.09999999999999
survival_time_mean2.503333333333331
survival_time_min0.8500000000000002
No reset possible
247533257Liu Sam 🇹🇼PredictorLastaido2-PREDstep1-simulationhost-erroryesip-172-31-46-148-104910:00:05
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
247523257Liu Sam 🇹🇼PredictorLastaido2-PREDstep1-simulationhost-erroryesip-172-31-46-148-104910:00:06
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
246664299Liam Paull 🇨🇦challenge-aido_LF-template-randomaido3-LFVI-sim-validationstep1-simulationsuccessnoip-172-31-46-148-104910:08:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.493517638549009
survival_time_median2.3999999999999995
deviation-center-line_median0.09791133281082
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.022341673887228665
agent_compute-ego_mean0.017918703100046555
agent_compute-ego_median0.01805200179417928
agent_compute-ego_min0.013383277824946811
deviation-center-line_max0.2348196785686717
deviation-center-line_mean0.1354748199226606
deviation-center-line_min0.06216029860587694
deviation-heading_max0.8115272458314912
deviation-heading_mean0.49959867397150814
deviation-heading_median0.46504862964833654
deviation-heading_min0.30044835691402494
driven_any_max2.1864003300000703
driven_any_mean1.1068208197848537
driven_any_median0.9410513078477728
driven_any_min0.34749292376309954
driven_lanedir_consec_max1.8262319453635671
driven_lanedir_consec_mean0.8047440134491037
driven_lanedir_consec_min0.3325249708009643
driven_lanedir_max1.8262319453635671
driven_lanedir_mean0.8048281473204828
driven_lanedir_median0.493517638549009
driven_lanedir_min0.3325249708009643
in-drivable-lane_max1.3499999999999952
in-drivable-lane_mean0.5599999999999984
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.1626796149000337, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.01110610479040991, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.022341673887228665, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.0075968036168738255, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.004591096805620797, "sim_compute_performance-ego": 0.006092358239089386, "sim_compute_robot_state-ego": 0.00796548324295237, "sim_compute_robot_state-npc0": 0.007516501825067062, "sim_compute_robot_state-npc1": 0.00717734988731674, "sim_compute_robot_state-npc2": 0.00711935381346111, "sim_compute_robot_state-npc3": 0.006911105747464337}, "udem1-1-0": {"driven_any": 0.47798123795353264, "sim_physics": 0.09634350027356828, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.007027481283460345, "in-drivable-lane": 0, "agent_compute-ego": 0.013383277824946811, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.004375100135803223, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.0026745200157165527, "sim_compute_performance-ego": 0.0036622200693402977, "sim_compute_robot_state-ego": 0.004616473402295794, "sim_compute_robot_state-npc0": 0.004345996039254325, "sim_compute_robot_state-npc1": 0.0043084536279950824, "sim_compute_robot_state-npc2": 0.004296158041272845, "sim_compute_robot_state-npc3": 0.004295093672616141}, "udem1-2-0": {"driven_any": 2.1864003300000703, "sim_physics": 0.13677824816657502, "survival_time": 5.14999999999999, "driven_lanedir": 1.8262319453635671, "sim_render-ego": 0.00958830407522257, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.017669876802314834, "deviation-heading": 0.8115272458314912, "set_robot_commands": 0.006204350480755556, "deviation-center-line": 0.19273160416094068, "driven_lanedir_consec": 1.8262319453635671, "sim_compute_sim_state": 0.003853247003647888, "sim_compute_performance-ego": 0.0050676586558517905, "sim_compute_robot_state-ego": 0.006399089850268318, "sim_compute_robot_state-npc0": 0.006097427849630708, "sim_compute_robot_state-npc1": 0.005852345124031734, "sim_compute_robot_state-npc2": 0.005771405488541982, "sim_compute_robot_state-npc3": 0.005751447770202044}, "udem1-3-0": {"driven_any": 0.9410513078477728, "sim_physics": 0.1367644717295965, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.009243478377660116, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.01805200179417928, "deviation-heading": 0.35154027506548413, "set_robot_commands": 0.006208707888921102, "deviation-center-line": 0.08975118546699376, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.0036755849917729697, "sim_compute_performance-ego": 0.004990677038828532, "sim_compute_robot_state-ego": 0.006487647692362468, "sim_compute_robot_state-npc0": 0.005936379233996074, "sim_compute_robot_state-npc1": 0.005948270360628764, "sim_compute_robot_state-npc2": 0.0060404737790425616, "sim_compute_robot_state-npc3": 0.006065239508946736}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.1451423168182373, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.010071323031470888, "in-drivable-lane": 0, "agent_compute-ego": 0.0181466851915632, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.00690482911609468, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.004002162388392857, "sim_compute_performance-ego": 0.00557633808680943, "sim_compute_robot_state-ego": 0.007209073929559617, "sim_compute_robot_state-npc0": 0.006470282872517903, "sim_compute_robot_state-npc1": 0.006167582103184291, "sim_compute_robot_state-npc2": 0.006332283928280785, "sim_compute_robot_state-npc3": 0.006472053981962658}}
set_robot_commands_max0.0075968036168738255
set_robot_commands_mean0.006257958247689677
set_robot_commands_median0.006208707888921102
set_robot_commands_min0.004375100135803223
sim_compute_performance-ego_max0.006092358239089386
sim_compute_performance-ego_mean0.005077850417983887
sim_compute_performance-ego_median0.0050676586558517905
sim_compute_performance-ego_min0.0036622200693402977
sim_compute_robot_state-ego_max0.00796548324295237
sim_compute_robot_state-ego_mean0.006535553623487714
sim_compute_robot_state-ego_median0.006487647692362468
sim_compute_robot_state-ego_min0.004616473402295794
sim_compute_robot_state-npc0_max0.007516501825067062
sim_compute_robot_state-npc0_mean0.006073317564093214
sim_compute_robot_state-npc0_median0.006097427849630708
sim_compute_robot_state-npc0_min0.004345996039254325
sim_compute_robot_state-npc1_max0.00717734988731674
sim_compute_robot_state-npc1_mean0.005890800220631322
sim_compute_robot_state-npc1_median0.005948270360628764
sim_compute_robot_state-npc1_min0.0043084536279950824
sim_compute_robot_state-npc2_max0.00711935381346111
sim_compute_robot_state-npc2_mean0.005911935010119856
sim_compute_robot_state-npc2_median0.0060404737790425616
sim_compute_robot_state-npc2_min0.004296158041272845
sim_compute_robot_state-npc3_max0.006911105747464337
sim_compute_robot_state-npc3_mean0.005898988136238383
sim_compute_robot_state-npc3_median0.006065239508946736
sim_compute_robot_state-npc3_min0.004295093672616141
sim_compute_sim_state_max0.004591096805620797
sim_compute_sim_state_mean0.003759322241030213
sim_compute_sim_state_median0.003853247003647888
sim_compute_sim_state_min0.0026745200157165527
sim_physics_max0.1626796149000337
sim_physics_mean0.13554163037760217
sim_physics_median0.13677824816657502
sim_physics_min0.09634350027356828
sim_render-ego_max0.01110610479040991
sim_render-ego_mean0.009407338311644763
sim_render-ego_median0.00958830407522257
sim_render-ego_min0.007027481283460345
simulation-passed1
survival_time_max5.14999999999999
survival_time_mean2.789999999999997
survival_time_min1.0500000000000005
No reset possible
246563264Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-46-148-104910:00:31
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
246433282Daniel HuangPredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.12320427505570893
error_L20.023163305460310996


No reset possible
246353265Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-46-148-104910:00:04
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
246283283Jing-Lin Chen 🇹🇼PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.10478495893466604
error_L20.01819770672777608


No reset possible
246213583Karthikk Subramanian 🇸🇬PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.0936910669502488
error_L20.014108443128222818


No reset possible
246144024Scott HixsonPredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.12274547922812055
error_L20.02170352734491878


No reset possible
246104029Frank (Chude) Qian 🇨🇦PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.1109797389415236
error_L20.01902076986722897


No reset possible
246014043Riza Velioglu 🇩🇪PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.10521260251099904
error_L20.01733017234136434


No reset possible
245954065c reddy 🇮🇳PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.1164062917733812
error_L20.021106336201491292


No reset possible
245894081Sebastian Nicolas Giles 🇮🇹PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.12499550980661878
error_L20.02468145151873354


No reset possible
245834081Sebastian Nicolas Giles 🇮🇹PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.10559229636377956
error_L20.018224635299106097


No reset possible
245784085Marc-Philippe FreyPredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.12046784985927408
error_L20.021233952822984316


No reset possible
245734089Stefan Lionar 🇨🇭PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.10470119509491634
error_L20.018026650301010947


No reset possible
245674097Yannick StrümplerPredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.11707910360891524
error_L20.02117622107364736


No reset possible
245584108Nikolaj Witting 🇩🇰PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:00:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.11749782335210374
error_L20.02134793494675946


No reset possible
245374252Kobe Leenders 🇧🇪PredictorLastaido2-PREDstep1-simulationsuccessyesip-172-31-46-148-104910:01:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.1029890632058805
error_L20.016731795024027357


No reset possible
245244281Liam Paull 🇨🇦challenge-aido_LF-template-randomaido3-LF-sim-testingstep1-simulationsuccessnoip-172-31-46-148-104910:09:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.10099507669212504
in-drivable-lane_median0.4499999999999984


other stats
agent_compute-ego_max0.020841792777732567
agent_compute-ego_mean0.01818980995601947
agent_compute-ego_median0.01818275451660156
agent_compute-ego_min0.014119241197230452
deviation-center-line_max0.28386357003337986
deviation-center-line_mean0.12786165368054708
deviation-center-line_min0.07112529346797256
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3391682685652288
deviation-heading_median0.3292838510353387
deviation-heading_min0.19401296908406307
driven_any_max2.1706243860917107
driven_any_mean0.975949037827615
driven_any_median1.0117660515775695
driven_any_min0.23626560261560617
driven_lanedir_consec_max1.919400421063861
driven_lanedir_consec_mean0.6896171461819268
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max1.919400421063861
driven_lanedir_mean0.6896171461819268
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.5899999999999982
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429704, "sim_physics": 0.07735687715035898, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.010954132786503545, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.020841792777732567, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.00715423513341833, "deviation-center-line": 0.15907928712407876, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.004054246125397859, "sim_compute_performance-ego": 0.006123410330878364, "sim_compute_robot_state-ego": 0.007204762211552373}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.05642235045339547, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.007336826885447782, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.018879109737919827, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.0050676570219152116, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.002966904172710344, "sim_compute_performance-ego": 0.003896727281458238, "sim_compute_robot_state-ego": 0.005285777297674441}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.06386228168711942, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.010338965584250057, "in-drivable-lane": 0, "agent_compute-ego": 0.01729237332063563, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.007512569427490234, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.00350482323590447, "sim_compute_performance-ego": 0.0055643390206729666, "sim_compute_robot_state-ego": 0.007188572603113511}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.06689751625061036, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.00980900764465332, "in-drivable-lane": 0, "agent_compute-ego": 0.019067344665527345, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.006158838272094727, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.003751773834228515, "sim_compute_performance-ego": 0.005199213027954102, "sim_compute_robot_state-ego": 0.007197809219360351}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.6182144577009443, "sim_physics": 0.06871504723271236, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.01047008852415447, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.020051666452914858, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.007607541506803489, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.00411473044866248, "sim_compute_performance-ego": 0.005697337886955165, "sim_compute_robot_state-ego": 0.007161339627036565}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.003851495683299, "sim_physics": 0.05742124635346082, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7531703123735949, "sim_render-ego": 0.0096098500855115, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.017647986509362047, "deviation-heading": 0.4596437246381204, "set_robot_commands": 0.006322529851173868, "deviation-center-line": 0.08968283443958694, "driven_lanedir_consec": 0.7531703123735949, "sim_compute_sim_state": 0.0035078379572654255, "sim_compute_performance-ego": 0.005240776100937201, "sim_compute_robot_state-ego": 0.006216540628550004}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0542898001504386, "sim_physics": 0.07115124804633004, "survival_time": 2.799999999999998, "driven_lanedir": 0.5389398655375865, "sim_render-ego": 0.009669538055147444, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.018046638795307705, "deviation-heading": 0.4109275195546752, "set_robot_commands": 0.006777205637523106, "deviation-center-line": 0.10786249131209884, "driven_lanedir_consec": 0.5389398655375865, "sim_compute_sim_state": 0.003728985786437988, "sim_compute_performance-ego": 0.0052347225802285334, "sim_compute_robot_state-ego": 0.0066165753773280555}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3164475681403097, "sim_physics": 0.07399220693679083, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3093025166124813, "sim_render-ego": 0.00950380734034947, "in-drivable-lane": 0, "agent_compute-ego": 0.01818275451660156, "deviation-heading": 0.2242052814634115, "set_robot_commands": 0.006197725023542132, "deviation-center-line": 0.09758615464743424, "driven_lanedir_consec": 0.3093025166124813, "sim_compute_sim_state": 0.004172075362432571, "sim_compute_performance-ego": 0.005222967692783901, "sim_compute_robot_state-ego": 0.006613538378760928}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2018528489510794, "sim_physics": 0.056738465519274695, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6922532395203, "sim_render-ego": 0.0075503809977385955, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.014119241197230452, "deviation-heading": 0.3292838510353387, "set_robot_commands": 0.005137035402200989, "deviation-center-line": 0.07112529346797256, "driven_lanedir_consec": 0.6922532395203, "sim_compute_sim_state": 0.003081172199572547, "sim_compute_performance-ego": 0.00402309126773123, "sim_compute_robot_state-ego": 0.005333128622022725}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.043548155325732, "sim_physics": 0.07641397389498623, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5204405548974873, "sim_render-ego": 0.009630164233121003, "in-drivable-lane": 1.1499999999999977, "agent_compute-ego": 0.018878091465343128, "deviation-heading": 0.33296447766287096, "set_robot_commands": 0.006910475817593661, "deviation-center-line": 0.0950267943453044, "driven_lanedir_consec": 0.5204405548974873, "sim_compute_sim_state": 0.003785024989734996, "sim_compute_performance-ego": 0.005214920910921963, "sim_compute_robot_state-ego": 0.006815160404552113}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5954548672193285, "sim_physics": 0.06391137915772277, "survival_time": 3.8499999999999943, "driven_lanedir": 1.1042463557848785, "sim_render-ego": 0.009461049909715528, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.01762662615094866, "deviation-heading": 0.35668107106767694, "set_robot_commands": 0.005869732274637594, "deviation-center-line": 0.2251391443608223, "driven_lanedir_consec": 1.1042463557848785, "sim_compute_sim_state": 0.003642069828974736, "sim_compute_performance-ego": 0.00526886172108836, "sim_compute_robot_state-ego": 0.006484753125673765}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.49030923107489305, "sim_physics": 0.06002889360700335, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4826393189617785, "sim_render-ego": 0.007968451295580183, "in-drivable-lane": 0, "agent_compute-ego": 0.01844104698726109, "deviation-heading": 0.23322336656472228, "set_robot_commands": 0.0054547446114676336, "deviation-center-line": 0.10099507669212504, "driven_lanedir_consec": 0.4826393189617785, "sim_compute_sim_state": 0.003139027527400426, "sim_compute_performance-ego": 0.0041743772370474675, "sim_compute_robot_state-ego": 0.005417295864650181}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0117660515775695, "sim_physics": 0.062096263848099055, "survival_time": 2.549999999999999, "driven_lanedir": 0.791717039833255, "sim_render-ego": 0.01048892152075674, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.018947554569618375, "deviation-heading": 0.320449356593834, "set_robot_commands": 0.008048286624983246, "deviation-center-line": 0.1017166571540705, "driven_lanedir_consec": 0.791717039833255, "sim_compute_sim_state": 0.003688919777963676, "sim_compute_performance-ego": 0.005872693716310987, "sim_compute_robot_state-ego": 0.0072464335198495905}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.44419823353351984, "sim_physics": 0.06634237148143628, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4333246859278659, "sim_render-ego": 0.0091891641970034, "in-drivable-lane": 0, "agent_compute-ego": 0.017698738310072158, "deviation-heading": 0.2988926743481924, "set_robot_commands": 0.006365431679619683, "deviation-center-line": 0.08545702330592322, "driven_lanedir_consec": 0.4333246859278659, "sim_compute_sim_state": 0.003679602234451859, "sim_compute_performance-ego": 0.004951079686482747, "sim_compute_robot_state-ego": 0.00638625356886122}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1706243860917107, "sim_physics": 0.05556614258710075, "survival_time": 5.09999999999999, "driven_lanedir": 1.919400421063861, "sim_render-ego": 0.007628681612949746, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.01712618388381659, "deviation-heading": 0.5178939525116448, "set_robot_commands": 0.005322580244026932, "deviation-center-line": 0.28386357003337986, "driven_lanedir_consec": 1.919400421063861, "sim_compute_sim_state": 0.0030289304022695506, "sim_compute_performance-ego": 0.004158134553946701, "sim_compute_robot_state-ego": 0.0053669027253693226}}
set_robot_commands_max0.008048286624983246
set_robot_commands_mean0.006393772568566057
set_robot_commands_median0.006322529851173868
set_robot_commands_min0.0050676570219152116
sim_compute_performance-ego_max0.006123410330878364
sim_compute_performance-ego_mean0.005056176867693194
sim_compute_performance-ego_median0.005222967692783901
sim_compute_performance-ego_min0.003896727281458238
sim_compute_robot_state-ego_max0.0072464335198495905
sim_compute_robot_state-ego_mean0.006435656211623677
sim_compute_robot_state-ego_median0.006613538378760928
sim_compute_robot_state-ego_min0.005285777297674441
sim_compute_sim_state_max0.004172075362432571
sim_compute_sim_state_mean0.0035897415922271627
sim_compute_sim_state_median0.003679602234451859
sim_compute_sim_state_min0.002966904172710344
sim_physics_max0.07735687715035898
sim_physics_mean0.06512775094709343
sim_physics_median0.06391137915772277
sim_physics_min0.05556614258710075
sim_render-ego_max0.010954132786503545
sim_render-ego_mean0.00930726871152552
sim_render-ego_median0.0096098500855115
sim_render-ego_min0.007336826885447782
simulation-passed1
survival_time_max5.09999999999999
survival_time_mean2.503333333333331
survival_time_min0.8500000000000002
No reset possible
244584252Kobe Leenders 🇧🇪PredictorLastaido2-PREDstep1-simulationsuccessnoip-172-31-46-148-104910:01:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.11481497971959664
error_L20.020765203959546272


No reset possible
244464243Manfred DiazBaseline-IL-logs-tensorflowaido3-LFV-sim-validationstep1-simulationerrornoip-172-31-46-148-104910:04:48
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
244304234Andrea Censi 🇨🇭challenge-aido_LF-template-pytorchaido3-LFV-sim-validationstep1-simulationerrornoip-172-31-46-148-104910:03:59
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 150, in _write
    self.fpin.write(j)
BrokenPipeError: [Errno 32] Broken pipe

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 77, in main
    agent_ci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 61, in _get_node_protocol
    self._write(j)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 162, in _write
    raise RemoteNodeAborted(msg) from e
zuper_nodes.structures.RemoteNodeAborted: While attempting to write to node "agent", I reckon that the pipe is closed and the node exited.

I could not read any aborted message: The remote node "agent" aborted with the following error:

error in agent |Unexpected error:
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper.py", line 214, in loop
               ||     node_name=node_name, tout=tout)
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper.py", line 39, in __init__
               ||     self.pc = LanguageChecker(protocol.interaction)
               ||   File "/usr/local/lib/python3.7/site-packages/zuper_nodes/language_recognize.py", line 123, in __init__
               ||     self.g.node[n]['label'] = 'start'
               || AttributeError: 'MultiDiGraph' object has no attribute 'node'
               ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
244204227Manfred DiazBaseline-IL-logs-tensorflowaido3-LFVI-sim-testingstep1-simulationfailednoip-172-31-46-148-104910:00:42
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[{{node prefix/ConvNet/conv_layer_1/kernel/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 60, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/conv_layer_1/kernel/read', defined at:
               ||   File "imitation_agent.py", line 136, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 132, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 43, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
244114219Manfred DiazBaseline-IL-logs-tensorflowaido3-LFVI-sim-testingstep1-simulationfailednoip-172-31-46-148-104910:03:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[{{node prefix/ConvNet/fc_layer_2/bias/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 69, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 58, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[node prefix/ConvNet/fc_layer_2/bias/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/fc_layer_2/bias/read', defined at:
               ||   File "imitation_agent.py", line 134, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 130, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 69, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 41, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[node prefix/ConvNet/fc_layer_2/bias/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
243964198Manfred Diazchallenge-aido_LF-template-tensorflowaido3-LF-sim-testingstep1-simulationerrornoip-172-31-46-148-104910:03:32
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 150, in _write
    self.fpin.write(j)
BrokenPipeError: [Errno 32] Broken pipe

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 77, in main
    agent_ci._get_node_protocol(timeout=config.timeout_initialization)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 61, in _get_node_protocol
    self._write(j)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 162, in _write
    raise RemoteNodeAborted(msg) from e
zuper_nodes.structures.RemoteNodeAborted: While attempting to write to node "agent", I reckon that the pipe is closed and the node exited.

I could not read any aborted message: The remote node "agent" aborted with the following error:

error in agent |Unexpected error:
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 214, in loop
               ||     node_name=node_name, tout=tout)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 39, in __init__
               ||     self.pc = LanguageChecker(protocol.interaction)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes/language_recognize.py", line 123, in __init__
               ||     self.g.node[n]['label'] = 'start'
               || AttributeError: 'MultiDiGraph' object has no attribute 'node'
               ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible