AI Driving Olympics Home Challenges Submissions Jobs

Evaluator 841

evaluator841
ownerI don't have one 😀
machineip-172-31-38-104
processip-172-31-38-104-5317
versiond-c:4.0.19;d-c-r:4.0.30;d-s:4.0.21
first heard
last heard
statusinactive
# evaluating
# success19
# timeout
# failed5
# error6
# aborted
# host-error17
arm0
x86_641
Mac0
gpu available1
Number of processors4
Processor frequency (MHz)2.9 GHz
Free % of processors37%
RAM total (MB)60.0 GB
RAM free (MB)55.9 GB
Disk (MB)96.9 GB
Disk available (MB)56.0 GB
Docker Hub
P11
P2
PI Camera0
# Duckiebots0
Map 3x3 avaiable0
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
215483212AmaurXchallenge-aido_LF-template-randomaido2-LF-sim-testingstep1-simulationsuccessno8410:15:58(hidden)
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.10099507669212504
in-drivable-lane_median0.4499999999999984


other stats
agent_compute-ego_max0.17283551781265824
agent_compute-ego_mean0.14685230374237035
agent_compute-ego_median0.15364584752491542
agent_compute-ego_min0.11306130444561992
deviation-center-line_max0.2838635700333795
deviation-center-line_mean0.12786165368054708
deviation-center-line_min0.07112529346797256
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3391682685652287
deviation-heading_median0.32928385103533875
deviation-heading_min0.19401296908406307
driven_any_max2.1706243860917107
driven_any_mean0.9759490378276146
driven_any_median1.0117660515775693
driven_any_min0.23626560261560617
driven_lanedir_consec_max1.919400421063832
driven_lanedir_consec_mean0.6896171461819246
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max1.919400421063832
driven_lanedir_mean0.6896171461819246
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.5899999999999982
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.07694900918889928, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.04448299496262162, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.11306130444561992, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.07291378798308196, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.02946715443222611, "sim_compute_performance-ego": 0.0488894780476888, "sim_compute_robot_state-ego": 0.05416336765995732}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.08832050304786832, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.05172170377245136, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.12601449442844764, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.07832631877824373, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.03205240006540336, "sim_compute_performance-ego": 0.05571260639265472, "sim_compute_robot_state-ego": 0.05876307393990311}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.09119563944199506, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.04686131196863511, "in-drivable-lane": 0, "agent_compute-ego": 0.1184089604546042, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.07564769071691177, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.028042260338278377, "sim_compute_performance-ego": 0.05115600193248076, "sim_compute_robot_state-ego": 0.057176674113554114}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.0994598388671875, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.05095966339111328, "in-drivable-lane": 0, "agent_compute-ego": 0.1270763111114502, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.07916634559631347, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.03101029396057129, "sim_compute_performance-ego": 0.05687820434570313, "sim_compute_robot_state-ego": 0.06007928848266601}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.618214457700942, "sim_physics": 0.09704500512231756, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.054547822928126856, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.1347445928597752, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.08671852606761304, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.03494580184357076, "sim_compute_performance-ego": 0.05906927736499642, "sim_compute_robot_state-ego": 0.06966662406921387}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.0038514956833002, "sim_physics": 0.0944764906046342, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7531703123735944, "sim_render-ego": 0.055332052464387854, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.14607184273856028, "deviation-heading": 0.4596437246381204, "set_robot_commands": 0.08153205988358478, "deviation-center-line": 0.08968283443958727, "driven_lanedir_consec": 0.7531703123735944, "sim_compute_sim_state": 0.0358240896341752, "sim_compute_performance-ego": 0.059232828568439096, "sim_compute_robot_state-ego": 0.06470302173069545}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0542898001504375, "sim_physics": 0.11768468362944466, "survival_time": 2.799999999999998, "driven_lanedir": 0.5389398655375865, "sim_render-ego": 0.0614052414894104, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.15057166985103063, "deviation-heading": 0.4109275195546752, "set_robot_commands": 0.09363992725099836, "deviation-center-line": 0.10786249131209884, "driven_lanedir_consec": 0.5389398655375865, "sim_compute_sim_state": 0.039036567722048075, "sim_compute_performance-ego": 0.06643964563097272, "sim_compute_robot_state-ego": 0.07148829528263637}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3164475681403097, "sim_physics": 0.1286184106554304, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3093025166124813, "sim_render-ego": 0.0650051094236828, "in-drivable-lane": 0, "agent_compute-ego": 0.15869386990865073, "deviation-heading": 0.2242052814634115, "set_robot_commands": 0.10376991544451032, "deviation-center-line": 0.09758615464743424, "driven_lanedir_consec": 0.3093025166124813, "sim_compute_sim_state": 0.04194273267473493, "sim_compute_performance-ego": 0.07140874862670898, "sim_compute_robot_state-ego": 0.08292719296046666}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2018528489510765, "sim_physics": 0.12577224990068855, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6922532395203, "sim_render-ego": 0.06334178730592889, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.1585680347377971, "deviation-heading": 0.32928385103533875, "set_robot_commands": 0.1006429639913268, "deviation-center-line": 0.07112529346797256, "driven_lanedir_consec": 0.6922532395203, "sim_compute_sim_state": 0.03960977166385974, "sim_compute_performance-ego": 0.06886681863817118, "sim_compute_robot_state-ego": 0.07352276575767387}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.0435481553257333, "sim_physics": 0.1246747927232222, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5204405548974873, "sim_render-ego": 0.06502399878068403, "in-drivable-lane": 1.1499999999999977, "agent_compute-ego": 0.15701844041997737, "deviation-heading": 0.33296447766287096, "set_robot_commands": 0.09902071952819824, "deviation-center-line": 0.0950267943453044, "driven_lanedir_consec": 0.5204405548974873, "sim_compute_sim_state": 0.040138175270774144, "sim_compute_performance-ego": 0.070794339613481, "sim_compute_robot_state-ego": 0.07547157027504661}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5954548672193265, "sim_physics": 0.11671760175135228, "survival_time": 3.8499999999999943, "driven_lanedir": 1.1042463557848754, "sim_render-ego": 0.0642052223156025, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.15395906683686492, "deviation-heading": 0.3566810710676767, "set_robot_commands": 0.10285306596136712, "deviation-center-line": 0.22513914436082189, "driven_lanedir_consec": 1.1042463557848754, "sim_compute_sim_state": 0.03996596088657131, "sim_compute_performance-ego": 0.06881062705795486, "sim_compute_robot_state-ego": 0.07369843396273526}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.49030923107489305, "sim_physics": 0.13111884253365652, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4826393189617785, "sim_render-ego": 0.06337494509560722, "in-drivable-lane": 0, "agent_compute-ego": 0.15364584752491542, "deviation-heading": 0.23322336656472228, "set_robot_commands": 0.09650949920926775, "deviation-center-line": 0.10099507669212504, "driven_lanedir_consec": 0.4826393189617785, "sim_compute_sim_state": 0.04091102736336844, "sim_compute_performance-ego": 0.069071250302451, "sim_compute_robot_state-ego": 0.07187164681298393}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0117660515775693, "sim_physics": 0.1298967155755735, "survival_time": 2.549999999999999, "driven_lanedir": 0.791717039833255, "sim_render-ego": 0.06756304292117848, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.16947663064096488, "deviation-heading": 0.320449356593834, "set_robot_commands": 0.10609895575280284, "deviation-center-line": 0.1017166571540705, "driven_lanedir_consec": 0.791717039833255, "sim_compute_sim_state": 0.04201848366681267, "sim_compute_performance-ego": 0.07570878664652507, "sim_compute_robot_state-ego": 0.08867417597303204}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.44419823353351984, "sim_physics": 0.1354428573890969, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4333246859278659, "sim_render-ego": 0.06709299264130769, "in-drivable-lane": 0, "agent_compute-ego": 0.17283551781265824, "deviation-heading": 0.2988926743481924, "set_robot_commands": 0.11175387877005118, "deviation-center-line": 0.08545702330592322, "driven_lanedir_consec": 0.4333246859278659, "sim_compute_sim_state": 0.04137200779385037, "sim_compute_performance-ego": 0.07742914447077999, "sim_compute_robot_state-ego": 0.09407503516585738}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1706243860917107, "sim_physics": 0.12239383716209262, "survival_time": 5.09999999999999, "driven_lanedir": 1.919400421063832, "sim_render-ego": 0.06524128072402056, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.16263797236423866, "deviation-heading": 0.5178939525116442, "set_robot_commands": 0.1015252482657339, "deviation-center-line": 0.2838635700333795, "driven_lanedir_consec": 1.919400421063832, "sim_compute_sim_state": 0.04159950742534563, "sim_compute_performance-ego": 0.07220048296685312, "sim_compute_robot_state-ego": 0.07772504114637188}}
set_robot_commands_max0.11175387877005118
set_robot_commands_mean0.09267459354666702
set_robot_commands_median0.09650949920926775
set_robot_commands_min0.07291378798308196
sim_compute_performance-ego_max0.07742914447077999
sim_compute_performance-ego_mean0.06477788270705738
sim_compute_performance-ego_median0.06881062705795486
sim_compute_performance-ego_min0.0488894780476888
sim_compute_robot_state-ego_max0.09407503516585738
sim_compute_robot_state-ego_mean0.07160041382218625
sim_compute_robot_state-ego_median0.07187164681298393
sim_compute_robot_state-ego_min0.05416336765995732
sim_compute_sim_state_max0.04201848366681267
sim_compute_sim_state_mean0.03719574898277269
sim_compute_sim_state_median0.03960977166385974
sim_compute_sim_state_min0.028042260338278377
sim_physics_max0.1354428573890969
sim_physics_mean0.11198443183956396
sim_physics_median0.11768468362944466
sim_physics_min0.07694900918889928
sim_render-ego_max0.06756304292117848
sim_render-ego_mean0.059077278012317255
sim_render-ego_median0.06334178730592889
sim_render-ego_min0.04448299496262162
simulation-passed1
survival_time_max5.09999999999999
survival_time_mean2.503333333333331
survival_time_min0.8500000000000002
215323209JBRRussiaSSegmaido2-LF-sim-validationstep1-simulationhost-errorno8410:09:56
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-validation/submission3209/step1-simulation-ip-172-31-38-104-5317-job21532/logs/challenges-runner/stdout.log'
(hidden)
215203207MYFtestaido2-LF-sim-validationstep1-simulationsuccessno8410:10:53(hidden)
driven_lanedir_consec_median1.796630439099885
survival_time_median13.350000000000056
deviation-center-line_median0.7218006464507031
in-drivable-lane_median0.5999999999999979


other stats
agent_compute-ego_max0.0951028245218684
agent_compute-ego_mean0.09447411113675194
agent_compute-ego_median0.09492066860198976
agent_compute-ego_min0.09359569638688987
deviation-center-line_max1.2130527144248868
deviation-center-line_mean0.7122318008514024
deviation-center-line_min0.43318527907464666
deviation-heading_max3.509359422642182
deviation-heading_mean2.160025291194262
deviation-heading_median2.7232625218869715
deviation-heading_min0.6320246874729665
driven_any_max3.3900387859506496
driven_any_mean2.4023120469710557
driven_any_median2.652694734523318
driven_any_min1.12916067822082
driven_lanedir_consec_max3.200438999505815
driven_lanedir_consec_mean1.9896354891255303
driven_lanedir_consec_min1.0071942904779951
driven_lanedir_max3.200438999505815
driven_lanedir_mean1.9896354891255303
driven_lanedir_median1.796630439099885
driven_lanedir_min1.0071942904779951
in-drivable-lane_max3.8500000000000543
in-drivable-lane_mean1.760000000000019
in-drivable-lane_min0.14999999999999947
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.652694734523318, "sim_physics": 0.05948283699121368, "survival_time": 13.350000000000056, "driven_lanedir": 1.796630439099885, "sim_render-ego": 0.03759119215975987, "in-drivable-lane": 3.6500000000000465, "agent_compute-ego": 0.0951028245218684, "deviation-heading": 3.509359422642182, "set_robot_commands": 0.05772056651026122, "deviation-center-line": 0.7218006464507031, "driven_lanedir_consec": 1.796630439099885, "sim_compute_sim_state": 0.0238784636451064, "sim_compute_performance-ego": 0.04045443231246891, "sim_compute_robot_state-ego": 0.04242673170254025}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.12916067822082, "sim_physics": 0.0602223226957232, "survival_time": 5.349999999999989, "driven_lanedir": 1.0071942904779951, "sim_render-ego": 0.03614598791175914, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.09359569638688987, "deviation-heading": 0.6320246874729665, "set_robot_commands": 0.05706502566827792, "deviation-center-line": 0.43318527907464666, "driven_lanedir_consec": 1.0071942904779951, "sim_compute_sim_state": 0.024392181467787127, "sim_compute_performance-ego": 0.04058667878124202, "sim_compute_robot_state-ego": 0.04226044850928761}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.7345654981378695, "sim_physics": 0.06001417878745259, "survival_time": 6.8999999999999835, "driven_lanedir": 1.682894526200014, "sim_render-ego": 0.03770724414051443, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.09374067230501036, "deviation-heading": 0.9550658387050416, "set_robot_commands": 0.058535677799280136, "deviation-center-line": 0.4559453593711875, "driven_lanedir_consec": 1.682894526200014, "sim_compute_sim_state": 0.024155067360919456, "sim_compute_performance-ego": 0.04047635327214781, "sim_compute_robot_state-ego": 0.04244376092717267}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 3.1051005380226204, "sim_physics": 0.05927089770634969, "survival_time": 14.950000000000076, "driven_lanedir": 2.2610191903439443, "sim_render-ego": 0.03783372243245443, "in-drivable-lane": 3.8500000000000543, "agent_compute-ego": 0.09492066860198976, "deviation-heading": 2.98041398526415, "set_robot_commands": 0.05727486928304037, "deviation-center-line": 0.7371750049355881, "driven_lanedir_consec": 2.2610191903439443, "sim_compute_sim_state": 0.023863322734832763, "sim_compute_performance-ego": 0.04025878111521403, "sim_compute_robot_state-ego": 0.04277834018071493}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 3.3900387859506496, "sim_physics": 0.057869317531585696, "survival_time": 14.950000000000076, "driven_lanedir": 3.200438999505815, "sim_render-ego": 0.03702797412872314, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.0950106938680013, "deviation-heading": 2.7232625218869715, "set_robot_commands": 0.05717933177947998, "deviation-center-line": 1.2130527144248868, "driven_lanedir_consec": 3.200438999505815, "sim_compute_sim_state": 0.024082117080688477, "sim_compute_performance-ego": 0.04106000820795695, "sim_compute_robot_state-ego": 0.042467629114786784}}
set_robot_commands_max0.058535677799280136
set_robot_commands_mean0.057555094208067925
set_robot_commands_median0.05727486928304037
set_robot_commands_min0.05706502566827792
sim_compute_performance-ego_max0.04106000820795695
sim_compute_performance-ego_mean0.04056725073780594
sim_compute_performance-ego_median0.04047635327214781
sim_compute_performance-ego_min0.04025878111521403
sim_compute_robot_state-ego_max0.04277834018071493
sim_compute_robot_state-ego_mean0.04247538208690045
sim_compute_robot_state-ego_median0.04244376092717267
sim_compute_robot_state-ego_min0.04226044850928761
sim_compute_sim_state_max0.024392181467787127
sim_compute_sim_state_mean0.024074230457866847
sim_compute_sim_state_median0.024082117080688477
sim_compute_sim_state_min0.023863322734832763
sim_physics_max0.0602223226957232
sim_physics_mean0.05937191074246497
sim_physics_median0.05948283699121368
sim_physics_min0.057869317531585696
sim_render-ego_max0.03783372243245443
sim_render-ego_mean0.0372612241546422
sim_render-ego_median0.03759119215975987
sim_render-ego_min0.03614598791175914
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean11.100000000000035
survival_time_min5.349999999999989
214653160aklebanPython templateaido2-AMOD-service_qualitystep2-scoringsuccessyes8410:00:16(hidden)
service_quality-32.368985668200835


other stats
efficiency-63.36354267280057
fleet_size-1000000000
214643161aklebanPython templateaido2-AMOD-fleet_sizestep2-scoringsuccessyes8410:00:10(hidden)
fleet_size-1000000000


other stats
efficiency-63.38101369598091
service_quality-32.38285342399595
214603161aklebanPython templateaido2-AMOD-fleet_sizestep1-simulationsuccessyes8410:03:02(hidden)
other stats
passedtrue
214553155awoo424After debuggingaido2-LF-sim-validationstep1-simulationsuccessno8410:04:27(hidden)
driven_lanedir_consec_median0.09767651915054731
survival_time_median1.5000000000000009
deviation-center-line_median0.06746458910260299
in-drivable-lane_median0.15000000000000013


other stats
agent_compute-ego_max0.10139440201424264
agent_compute-ego_mean0.09997749820179
agent_compute-ego_median0.09967160224914552
agent_compute-ego_min0.09860289664495558
deviation-center-line_max0.09546468255289656
deviation-center-line_mean0.07097998846551419
deviation-center-line_min0.05403746052209888
deviation-heading_max0.940474645478361
deviation-heading_mean0.8474245868165939
deviation-heading_median0.8691364881504393
deviation-heading_min0.7235560393646222
driven_any_max0.2402612205613981
driven_any_mean0.17480627267678658
driven_any_median0.18223126386803423
driven_any_min0.0938015414032502
driven_lanedir_consec_max0.11556915042381856
driven_lanedir_consec_mean0.09014075691016768
driven_lanedir_consec_min0.05451872405854763
driven_lanedir_max0.11556915042381856
driven_lanedir_mean0.09014075691016768
driven_lanedir_median0.09767651915054731
driven_lanedir_min0.05451872405854763
in-drivable-lane_max0.4500000000000004
in-drivable-lane_mean0.17000000000000015
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.2402612205613981, "sim_physics": 0.06137783462936814, "survival_time": 1.850000000000001, "driven_lanedir": 0.11556915042381856, "sim_render-ego": 0.03683892456260887, "in-drivable-lane": 0.4500000000000004, "agent_compute-ego": 0.10139440201424264, "deviation-heading": 0.8691364881504393, "set_robot_commands": 0.055251695014335016, "deviation-center-line": 0.08035687909954502, "driven_lanedir_consec": 0.11556915042381856, "sim_compute_sim_state": 0.02307059958174422, "sim_compute_performance-ego": 0.039959598231959985, "sim_compute_robot_state-ego": 0.04147984530474689}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.18223126386803423, "sim_physics": 0.06246485710144043, "survival_time": 1.5000000000000009, "driven_lanedir": 0.1134200136466368, "sim_render-ego": 0.036124054590861, "in-drivable-lane": 0, "agent_compute-ego": 0.09943314393361408, "deviation-heading": 0.9153545566021856, "set_robot_commands": 0.05650804837544759, "deviation-center-line": 0.06746458910260299, "driven_lanedir_consec": 0.1134200136466368, "sim_compute_sim_state": 0.023970301946004233, "sim_compute_performance-ego": 0.03979717095692952, "sim_compute_robot_state-ego": 0.0415942112604777}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.0938015414032502, "sim_physics": 0.062101034891037715, "survival_time": 1.0500000000000005, "driven_lanedir": 0.05451872405854763, "sim_render-ego": 0.035424357368832544, "in-drivable-lane": 0, "agent_compute-ego": 0.09860289664495558, "deviation-heading": 0.7235560393646222, "set_robot_commands": 0.055933986391339986, "deviation-center-line": 0.09546468255289656, "driven_lanedir_consec": 0.05451872405854763, "sim_compute_sim_state": 0.023391087849934895, "sim_compute_performance-ego": 0.0398577508472261, "sim_compute_robot_state-ego": 0.04191470146179199}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.1730110361658628, "sim_physics": 0.06704127377477186, "survival_time": 1.4500000000000006, "driven_lanedir": 0.0695193772712881, "sim_render-ego": 0.03763628828114477, "in-drivable-lane": 0.2500000000000002, "agent_compute-ego": 0.09967160224914552, "deviation-heading": 0.940474645478361, "set_robot_commands": 0.0562840905682794, "deviation-center-line": 0.05757633105042749, "driven_lanedir_consec": 0.0695193772712881, "sim_compute_sim_state": 0.024447350666440767, "sim_compute_performance-ego": 0.04020804372327081, "sim_compute_robot_state-ego": 0.042031649885506464}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.18472630138538773, "sim_physics": 0.06333325703938802, "survival_time": 1.5000000000000009, "driven_lanedir": 0.09767651915054731, "sim_render-ego": 0.03667583465576172, "in-drivable-lane": 0.15000000000000013, "agent_compute-ego": 0.10078544616699218, "deviation-heading": 0.7886012044873617, "set_robot_commands": 0.05662806828816732, "deviation-center-line": 0.05403746052209888, "driven_lanedir_consec": 0.09767651915054731, "sim_compute_sim_state": 0.02332471211751302, "sim_compute_performance-ego": 0.039987492561340335, "sim_compute_robot_state-ego": 0.04247764746348063}}
set_robot_commands_max0.05662806828816732
set_robot_commands_mean0.05612117772751386
set_robot_commands_median0.0562840905682794
set_robot_commands_min0.055251695014335016
sim_compute_performance-ego_max0.04020804372327081
sim_compute_performance-ego_mean0.03996201126414535
sim_compute_performance-ego_median0.039959598231959985
sim_compute_performance-ego_min0.03979717095692952
sim_compute_robot_state-ego_max0.04247764746348063
sim_compute_robot_state-ego_mean0.04189961107520073
sim_compute_robot_state-ego_median0.04191470146179199
sim_compute_robot_state-ego_min0.04147984530474689
sim_compute_sim_state_max0.024447350666440767
sim_compute_sim_state_mean0.02364081043232743
sim_compute_sim_state_median0.023391087849934895
sim_compute_sim_state_min0.02307059958174422
sim_physics_max0.06704127377477186
sim_physics_mean0.06326365148720123
sim_physics_median0.06246485710144043
sim_physics_min0.06137783462936814
sim_render-ego_max0.03763628828114477
sim_render-ego_mean0.03653989189184178
sim_render-ego_median0.03667583465576172
sim_render-ego_min0.035424357368832544
simulation-passed1
survival_time_max1.850000000000001
survival_time_mean1.4700000000000009
survival_time_min1.0500000000000005
214523154Andy Serchallenge-aido_LF-baseline-duckietownaido2-LF-sim-testingstep1-simulationhost-errorno8410:07:04
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-testing/submission3154/step1-simulation-ip-172-31-38-104-5317-job21452/logs/challenges-runner/stdout.log'
(hidden)
214513153Andy Serchallenge-aido_LF-baseline-duckietownaido2-LF-sim-validationstep1-simulationsuccessno8410:07:40(hidden)
driven_lanedir_consec_median0.6447460901444462
survival_time_median5.04999999999999
deviation-center-line_median0.4878536500805453
in-drivable-lane_median0.2499999999999991


other stats
agent_compute-ego_max0.05862967566688462
agent_compute-ego_mean0.05761735591457192
agent_compute-ego_median0.05766307830810547
agent_compute-ego_min0.05659904199488023
deviation-center-line_max1.2674440785135146
deviation-center-line_mean0.6408106479376643
deviation-center-line_min0.31637763969121124
deviation-heading_max6.970832601775145
deviation-heading_mean2.72659098184398
deviation-heading_median2.2749401473357027
deviation-heading_min0.4890732970682755
driven_any_max2.3484939608403383
driven_any_mean1.167722539275266
driven_any_median0.7502499985425195
driven_any_min0.6212441632030568
driven_lanedir_consec_max1.4265017493395105
driven_lanedir_consec_mean0.755282285260406
driven_lanedir_consec_min0.16829731736440065
driven_lanedir_max1.5226063103102452
driven_lanedir_mean0.9110891812257236
driven_lanedir_median0.6447460901444462
driven_lanedir_min0.3369775191475766
in-drivable-lane_max2.3000000000000176
in-drivable-lane_mean0.6500000000000032
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.7502499985425195, "sim_physics": 0.05961423817247447, "survival_time": 5.04999999999999, "driven_lanedir": 0.6447460901444462, "sim_render-ego": 0.03798912067224484, "in-drivable-lane": 0.2499999999999991, "agent_compute-ego": 0.05862967566688462, "deviation-heading": 1.5358830938207364, "set_robot_commands": 0.05854982904868551, "deviation-center-line": 0.349389635727287, "driven_lanedir_consec": 0.6447460901444462, "sim_compute_sim_state": 0.024376654388880967, "sim_compute_performance-ego": 0.04145583775964114, "sim_compute_robot_state-ego": 0.04357653325146968}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.6212441632030568, "sim_physics": 0.07333580369041079, "survival_time": 4.199999999999993, "driven_lanedir": 0.3369775191475766, "sim_render-ego": 0.03787226052511306, "in-drivable-lane": 0.6999999999999993, "agent_compute-ego": 0.05778259890420096, "deviation-heading": 2.362225769220041, "set_robot_commands": 0.0589448128427778, "deviation-center-line": 0.31637763969121124, "driven_lanedir_consec": 0.16829731736440065, "sim_compute_sim_state": 0.02478179477509998, "sim_compute_performance-ego": 0.0407659269514538, "sim_compute_robot_state-ego": 0.04451125008719308}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.6294169712159847, "sim_physics": 0.06147060394287109, "survival_time": 4.249999999999993, "driven_lanedir": 0.6246142371868397, "sim_render-ego": 0.03759559743544635, "in-drivable-lane": 0, "agent_compute-ego": 0.05659904199488023, "deviation-heading": 0.4890732970682755, "set_robot_commands": 0.05847250153036679, "deviation-center-line": 0.4878536500805453, "driven_lanedir_consec": 0.6246142371868397, "sim_compute_sim_state": 0.02496778263765223, "sim_compute_performance-ego": 0.04135873738457175, "sim_compute_robot_state-ego": 0.04379530794480268}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.4892076025744312, "sim_physics": 0.06551499540607135, "survival_time": 9.6, "driven_lanedir": 1.4265017493395105, "sim_render-ego": 0.03782959282398224, "in-drivable-lane": 0, "agent_compute-ego": 0.05741238469878832, "deviation-heading": 2.2749401473357027, "set_robot_commands": 0.05835782612363497, "deviation-center-line": 0.7829882356757638, "driven_lanedir_consec": 1.4265017493395105, "sim_compute_sim_state": 0.024698310842116673, "sim_compute_performance-ego": 0.041568029671907425, "sim_compute_robot_state-ego": 0.044113972534736}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.3484939608403383, "sim_physics": 0.06383905092875163, "survival_time": 14.950000000000076, "driven_lanedir": 1.5226063103102452, "sim_render-ego": 0.037866265773773194, "in-drivable-lane": 2.3000000000000176, "agent_compute-ego": 0.05766307830810547, "deviation-heading": 6.970832601775145, "set_robot_commands": 0.05752587636311849, "deviation-center-line": 1.2674440785135146, "driven_lanedir_consec": 0.9122520322668324, "sim_compute_sim_state": 0.024532779852549234, "sim_compute_performance-ego": 0.041352808475494385, "sim_compute_robot_state-ego": 0.04489694277445475}}
set_robot_commands_max0.0589448128427778
set_robot_commands_mean0.05837016918171671
set_robot_commands_median0.05847250153036679
set_robot_commands_min0.05752587636311849
sim_compute_performance-ego_max0.041568029671907425
sim_compute_performance-ego_mean0.0413002680486137
sim_compute_performance-ego_median0.04135873738457175
sim_compute_performance-ego_min0.0407659269514538
sim_compute_robot_state-ego_max0.04489694277445475
sim_compute_robot_state-ego_mean0.04417880131853124
sim_compute_robot_state-ego_median0.044113972534736
sim_compute_robot_state-ego_min0.04357653325146968
sim_compute_sim_state_max0.02496778263765223
sim_compute_sim_state_mean0.024671464499259815
sim_compute_sim_state_median0.024698310842116673
sim_compute_sim_state_min0.024376654388880967
sim_physics_max0.07333580369041079
sim_physics_mean0.06475493842811587
sim_physics_median0.06383905092875163
sim_physics_min0.05961423817247447
sim_render-ego_max0.03798912067224484
sim_render-ego_mean0.037830567446111935
sim_render-ego_median0.037866265773773194
sim_render-ego_min0.03759559743544635
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean7.610000000000011
survival_time_min4.199999999999993
214343137AmaurXchallenge-aido_LF-template-randomaido2-LF-sim-testingstep1-simulationsuccessno8410:09:33(hidden)
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.10099507669212504
in-drivable-lane_median0.4499999999999984


other stats
agent_compute-ego_max0.09519691467285156
agent_compute-ego_mean0.09124149026256692
agent_compute-ego_median0.09123796224594116
agent_compute-ego_min0.08959858930563624
deviation-center-line_max0.2838635700333795
deviation-center-line_mean0.12786165368054708
deviation-center-line_min0.07112529346797256
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3391682685652287
deviation-heading_median0.32928385103533875
deviation-heading_min0.19401296908406307
driven_any_max2.1706243860917107
driven_any_mean0.9759490378276146
driven_any_median1.0117660515775693
driven_any_min0.23626560261560617
driven_lanedir_consec_max1.919400421063832
driven_lanedir_consec_mean0.6896171461819246
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max1.919400421063832
driven_lanedir_mean0.6896171461819246
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.5899999999999982
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.05803173118167453, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.03593413476590757, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.09153798774436668, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.05678857255865027, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.0239145005190814, "sim_compute_performance-ego": 0.04032548710151955, "sim_compute_robot_state-ego": 0.04164164596133762}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.057816692427092906, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.0363772289425719, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.08986071044323492, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.05669876173430798, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.023313522338867188, "sim_compute_performance-ego": 0.03952288160137102, "sim_compute_robot_state-ego": 0.04197985518212412}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.06463961040272433, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.03798828405492446, "in-drivable-lane": 0, "agent_compute-ego": 0.0918271261103013, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.05878012320574592, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.024057738921221566, "sim_compute_performance-ego": 0.04237852377050063, "sim_compute_robot_state-ego": 0.04294587584102855}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.06698253631591797, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.03748958587646485, "in-drivable-lane": 0, "agent_compute-ego": 0.09519691467285156, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.05643953323364258, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.02368396759033203, "sim_compute_performance-ego": 0.04026776313781738, "sim_compute_robot_state-ego": 0.04244614601135254}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.618214457700942, "sim_physics": 0.059177377555943746, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.03652544564838651, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.08959858930563624, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.05678789525092402, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.023513151120535935, "sim_compute_performance-ego": 0.039263462718529034, "sim_compute_robot_state-ego": 0.04240555099294155}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.0038514956833002, "sim_physics": 0.057137440661994776, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7531703123735944, "sim_render-ego": 0.03710350211785765, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.09215355892570652, "deviation-heading": 0.4596437246381204, "set_robot_commands": 0.05657156632871044, "deviation-center-line": 0.08968283443958727, "driven_lanedir_consec": 0.7531703123735944, "sim_compute_sim_state": 0.023618683523061325, "sim_compute_performance-ego": 0.04004560684671207, "sim_compute_robot_state-ego": 0.04201109068734305}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0542898001504375, "sim_physics": 0.05942629916327341, "survival_time": 2.799999999999998, "driven_lanedir": 0.5389398655375865, "sim_render-ego": 0.03653946944645473, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.08981100150517055, "deviation-heading": 0.4109275195546752, "set_robot_commands": 0.05765630943434579, "deviation-center-line": 0.10786249131209884, "driven_lanedir_consec": 0.5389398655375865, "sim_compute_sim_state": 0.024116171257836477, "sim_compute_performance-ego": 0.039994614464896064, "sim_compute_robot_state-ego": 0.04204210638999939}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3164475681403097, "sim_physics": 0.06423784437633696, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3093025166124813, "sim_render-ego": 0.036638100941975914, "in-drivable-lane": 0, "agent_compute-ego": 0.09003724370683942, "deviation-heading": 0.2242052814634115, "set_robot_commands": 0.05693160919916062, "deviation-center-line": 0.09758615464743424, "driven_lanedir_consec": 0.3093025166124813, "sim_compute_sim_state": 0.024527867635091145, "sim_compute_performance-ego": 0.040167592820667085, "sim_compute_robot_state-ego": 0.04139791216169085}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2018528489510765, "sim_physics": 0.06107036946183544, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6922532395203, "sim_render-ego": 0.03722540806915801, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.09095726983021882, "deviation-heading": 0.32928385103533875, "set_robot_commands": 0.05580844313411389, "deviation-center-line": 0.07112529346797256, "driven_lanedir_consec": 0.6922532395203, "sim_compute_sim_state": 0.023466449672892943, "sim_compute_performance-ego": 0.03987202805987859, "sim_compute_robot_state-ego": 0.0424952385789257}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.0435481553257333, "sim_physics": 0.06039589535106312, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5204405548974873, "sim_render-ego": 0.03678337443958629, "in-drivable-lane": 1.1499999999999977, "agent_compute-ego": 0.09156940200112083, "deviation-heading": 0.33296447766287096, "set_robot_commands": 0.05570063591003418, "deviation-center-line": 0.0950267943453044, "driven_lanedir_consec": 0.5204405548974873, "sim_compute_sim_state": 0.023684063824740323, "sim_compute_performance-ego": 0.04014189026572488, "sim_compute_robot_state-ego": 0.04225086298855868}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5954548672193265, "sim_physics": 0.06235774461325113, "survival_time": 3.8499999999999943, "driven_lanedir": 1.1042463557848754, "sim_render-ego": 0.037240845816476006, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.09172238622392928, "deviation-heading": 0.3566810710676767, "set_robot_commands": 0.056578973671058554, "deviation-center-line": 0.22513914436082189, "driven_lanedir_consec": 1.1042463557848754, "sim_compute_sim_state": 0.02382237260991877, "sim_compute_performance-ego": 0.04022931742977787, "sim_compute_robot_state-ego": 0.04198691442415312}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.49030923107489305, "sim_physics": 0.06444948060171944, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4826393189617785, "sim_render-ego": 0.03784318481172834, "in-drivable-lane": 0, "agent_compute-ego": 0.09123796224594116, "deviation-heading": 0.23322336656472228, "set_robot_commands": 0.056195667811802456, "deviation-center-line": 0.10099507669212504, "driven_lanedir_consec": 0.4826393189617785, "sim_compute_sim_state": 0.023493809359414235, "sim_compute_performance-ego": 0.04015740326472691, "sim_compute_robot_state-ego": 0.04256591626576015}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0117660515775693, "sim_physics": 0.05940617767034793, "survival_time": 2.549999999999999, "driven_lanedir": 0.791717039833255, "sim_render-ego": 0.03598795217626235, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.09198716575024175, "deviation-heading": 0.320449356593834, "set_robot_commands": 0.05635667314716414, "deviation-center-line": 0.1017166571540705, "driven_lanedir_consec": 0.791717039833255, "sim_compute_sim_state": 0.02366352548786238, "sim_compute_performance-ego": 0.0399477014354631, "sim_compute_robot_state-ego": 0.0420227892258588}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.44419823353351984, "sim_physics": 0.06628084182739258, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4333246859278659, "sim_render-ego": 0.03749325540330675, "in-drivable-lane": 0, "agent_compute-ego": 0.09002304077148438, "deviation-heading": 0.2988926743481924, "set_robot_commands": 0.05574711163838705, "deviation-center-line": 0.08545702330592322, "driven_lanedir_consec": 0.4333246859278659, "sim_compute_sim_state": 0.02390163916128653, "sim_compute_performance-ego": 0.03911564085218641, "sim_compute_robot_state-ego": 0.04256902800665961}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1706243860917107, "sim_physics": 0.06019136952418907, "survival_time": 5.09999999999999, "driven_lanedir": 1.919400421063832, "sim_render-ego": 0.03674910349004409, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.0911019947014603, "deviation-heading": 0.5178939525116442, "set_robot_commands": 0.056477864583333336, "deviation-center-line": 0.2838635700333795, "driven_lanedir_consec": 1.919400421063832, "sim_compute_sim_state": 0.02368451333513447, "sim_compute_performance-ego": 0.04006088481229894, "sim_compute_robot_state-ego": 0.0422734770120359}}
set_robot_commands_max0.05878012320574592
set_robot_commands_mean0.05663464938942542
set_robot_commands_median0.05657156632871044
set_robot_commands_min0.05570063591003418
sim_compute_performance-ego_max0.04237852377050063
sim_compute_performance-ego_mean0.04009938657213797
sim_compute_performance-ego_median0.04006088481229894
sim_compute_performance-ego_min0.03911564085218641
sim_compute_robot_state-ego_max0.04294587584102855
sim_compute_robot_state-ego_mean0.042202293981984634
sim_compute_robot_state-ego_median0.04225086298855868
sim_compute_robot_state-ego_min0.04139791216169085
sim_compute_sim_state_max0.024527867635091145
sim_compute_sim_state_mean0.023764131757151783
sim_compute_sim_state_median0.023684063824740323
sim_compute_sim_state_min0.023313522338867188
sim_physics_max0.06698253631591797
sim_physics_mean0.06144009407565049
sim_physics_median0.06039589535106312
sim_physics_min0.057137440661994776
sim_render-ego_max0.03798828405492446
sim_render-ego_mean0.03692792506674037
sim_render-ego_median0.03678337443958629
sim_render-ego_min0.03593413476590757
simulation-passed1
survival_time_max5.09999999999999
survival_time_mean2.503333333333331
survival_time_min0.8500000000000002
214313132MYFtestaido2-LF-sim-testingstep1-simulationsuccessno8410:36:41(hidden)
driven_lanedir_consec_median2.6592140764351235
survival_time_median14.950000000000076
deviation-center-line_median0.5321148478878259
in-drivable-lane_median0.550000000000006


other stats
agent_compute-ego_max0.13535574754079183
agent_compute-ego_mean0.102140043515189
agent_compute-ego_median0.09212418467597625
agent_compute-ego_min0.09089086373647054
deviation-center-line_max0.920425395615501
deviation-center-line_mean0.5707830121753137
deviation-center-line_min0.25627065397421517
deviation-heading_max3.5071794332756028
deviation-heading_mean1.970794762044987
deviation-heading_median1.9720559742053665
deviation-heading_min0.5735423589840908
driven_any_max3.9258720003669567
driven_any_mean3.002858955485701
driven_any_median3.693201150069478
driven_any_min1.1379084615565582
driven_lanedir_consec_max3.899728383464023
driven_lanedir_consec_mean2.6341494064984152
driven_lanedir_consec_min0.968451206440034
driven_lanedir_max3.899728383464023
driven_lanedir_mean2.6341494064984152
driven_lanedir_median2.6592140764351235
driven_lanedir_min0.968451206440034
in-drivable-lane_max5.850000000000045
in-drivable-lane_mean1.1666666666666703
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 3.6950367738237153, "sim_physics": 0.08792593965562291, "survival_time": 14.950000000000076, "driven_lanedir": 3.105612199406713, "sim_render-ego": 0.052401159120642624, "in-drivable-lane": 1.6999999999999966, "agent_compute-ego": 0.1306753860269502, "deviation-heading": 3.5071794332756028, "set_robot_commands": 0.0769577385191136, "deviation-center-line": 0.920425395615501, "driven_lanedir_consec": 3.105612199406713, "sim_compute_sim_state": 0.03294058308553536, "sim_compute_performance-ego": 0.05599239518411184, "sim_compute_robot_state-ego": 0.0598168859513707}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 3.863561074609414, "sim_physics": 0.086118057568868, "survival_time": 14.950000000000076, "driven_lanedir": 3.830657428837964, "sim_render-ego": 0.05120306491851807, "in-drivable-lane": 0, "agent_compute-ego": 0.13535574754079183, "deviation-heading": 1.669146690256994, "set_robot_commands": 0.07668242851893108, "deviation-center-line": 0.4453364246536755, "driven_lanedir_consec": 3.830657428837964, "sim_compute_sim_state": 0.032163257598876956, "sim_compute_performance-ego": 0.05620741128921509, "sim_compute_robot_state-ego": 0.060709201494852705}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 3.720750592779589, "sim_physics": 0.08698561668395996, "survival_time": 14.950000000000076, "driven_lanedir": 3.610559375085419, "sim_render-ego": 0.051105304559071855, "in-drivable-lane": 0.15000000000000213, "agent_compute-ego": 0.1315937678019206, "deviation-heading": 2.436080143009035, "set_robot_commands": 0.08002414226531983, "deviation-center-line": 0.7479694906149612, "driven_lanedir_consec": 3.610559375085419, "sim_compute_sim_state": 0.03358116149902344, "sim_compute_performance-ego": 0.05640427033106486, "sim_compute_robot_state-ego": 0.060608108838399254}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 3.693201150069478, "sim_physics": 0.08318196376164755, "survival_time": 14.950000000000076, "driven_lanedir": 2.1300728768772155, "sim_render-ego": 0.048281150658925374, "in-drivable-lane": 5.850000000000045, "agent_compute-ego": 0.12423850297927856, "deviation-heading": 2.104427189670663, "set_robot_commands": 0.07360259850819906, "deviation-center-line": 0.5223882667414983, "driven_lanedir_consec": 2.1300728768772155, "sim_compute_sim_state": 0.030844624837239584, "sim_compute_performance-ego": 0.055269261995951335, "sim_compute_robot_state-ego": 0.05902582804361979}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.099614676558197, "sim_physics": 0.05859820757593427, "survival_time": 8.399999999999984, "driven_lanedir": 1.8250751705307224, "sim_render-ego": 0.03682209054629008, "in-drivable-lane": 0.6999999999999975, "agent_compute-ego": 0.0938074787457784, "deviation-heading": 1.8529116674053097, "set_robot_commands": 0.05622407368251255, "deviation-center-line": 0.43001657659248926, "driven_lanedir_consec": 1.8250751705307224, "sim_compute_sim_state": 0.024098113888785952, "sim_compute_performance-ego": 0.04059825340906779, "sim_compute_robot_state-ego": 0.04140742619832357}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.1379084615565582, "sim_physics": 0.05806867778301239, "survival_time": 4.799999999999991, "driven_lanedir": 1.08510848352438, "sim_render-ego": 0.03830044468243917, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.09106538693110149, "deviation-heading": 1.0371668833660934, "set_robot_commands": 0.05586991459131241, "deviation-center-line": 0.30052495831858606, "driven_lanedir_consec": 1.08510848352438, "sim_compute_sim_state": 0.02361246198415756, "sim_compute_performance-ego": 0.039837293326854706, "sim_compute_robot_state-ego": 0.04206309715906779}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.4747832939595609, "sim_physics": 0.060132843017578125, "survival_time": 6.249999999999986, "driven_lanedir": 1.2719188225501032, "sim_render-ego": 0.03690357780456543, "in-drivable-lane": 0.34999999999999876, "agent_compute-ego": 0.09228813552856444, "deviation-heading": 2.041867054225896, "set_robot_commands": 0.05604265022277832, "deviation-center-line": 0.5157963631298015, "driven_lanedir_consec": 1.2719188225501032, "sim_compute_sim_state": 0.023964378356933595, "sim_compute_performance-ego": 0.04015589141845703, "sim_compute_robot_state-ego": 0.04188192367553711}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 2.7846391335914027, "sim_physics": 0.059743280959340325, "survival_time": 11.300000000000026, "driven_lanedir": 2.221087814256407, "sim_render-ego": 0.036703110796160406, "in-drivable-lane": 1.75, "agent_compute-ego": 0.09212418467597625, "deviation-heading": 2.4595072836786462, "set_robot_commands": 0.05530956466641046, "deviation-center-line": 0.5321148478878259, "driven_lanedir_consec": 2.221087814256407, "sim_compute_sim_state": 0.025329616217486625, "sim_compute_performance-ego": 0.04046081652683494, "sim_compute_robot_state-ego": 0.04249839339636068}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 3.816227713030613, "sim_physics": 0.057058283487955726, "survival_time": 14.950000000000076, "driven_lanedir": 3.767510823782679, "sim_render-ego": 0.03675215323766073, "in-drivable-lane": 0, "agent_compute-ego": 0.09219433625539145, "deviation-heading": 2.0909852629385868, "set_robot_commands": 0.05504979689915975, "deviation-center-line": 0.7191951178518158, "driven_lanedir_consec": 3.767510823782679, "sim_compute_sim_state": 0.02375837484995524, "sim_compute_performance-ego": 0.03940680185953776, "sim_compute_robot_state-ego": 0.041674700578053794}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 3.69567824569167, "sim_physics": 0.05982586622238159, "survival_time": 14.950000000000076, "driven_lanedir": 3.2544242861963144, "sim_render-ego": 0.03675049940745036, "in-drivable-lane": 1.3500000000000192, "agent_compute-ego": 0.09132476011912027, "deviation-heading": 2.8129782992673373, "set_robot_commands": 0.05501183907190959, "deviation-center-line": 0.7526842070216211, "driven_lanedir_consec": 3.2544242861963144, "sim_compute_sim_state": 0.023533851305643717, "sim_compute_performance-ego": 0.039330168565114336, "sim_compute_robot_state-ego": 0.041963300704956054}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 3.0780347437714752, "sim_physics": 0.06177167360447655, "survival_time": 12.100000000000035, "driven_lanedir": 2.6592140764351235, "sim_render-ego": 0.03660714527792182, "in-drivable-lane": 1.3500000000000032, "agent_compute-ego": 0.09172896215738344, "deviation-heading": 1.9157620223787355, "set_robot_commands": 0.05475284737988937, "deviation-center-line": 0.8066770911118064, "driven_lanedir_consec": 2.6592140764351235, "sim_compute_sim_state": 0.02369692010327804, "sim_compute_performance-ego": 0.0392171105077444, "sim_compute_robot_state-ego": 0.041684527042483495}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.9882258361456095, "sim_physics": 0.058495063492746066, "survival_time": 8.249999999999982, "driven_lanedir": 0.968451206440034, "sim_render-ego": 0.03652605143460361, "in-drivable-lane": 3.6499999999999906, "agent_compute-ego": 0.09139564976547704, "deviation-heading": 1.6561291125340212, "set_robot_commands": 0.05476552067380963, "deviation-center-line": 0.25627065397421517, "driven_lanedir_consec": 0.968451206440034, "sim_compute_sim_state": 0.02363174611871893, "sim_compute_performance-ego": 0.03917803619847153, "sim_compute_robot_state-ego": 0.04125054243839149}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 3.8033808753644, "sim_physics": 0.05901103576024373, "survival_time": 14.950000000000076, "driven_lanedir": 3.7551924173698272, "sim_render-ego": 0.03647242546081543, "in-drivable-lane": 0, "agent_compute-ego": 0.09089086373647054, "deviation-heading": 1.9720559742053665, "set_robot_commands": 0.05506151835123698, "deviation-center-line": 0.6445640548050576, "driven_lanedir_consec": 3.7551924173698272, "sim_compute_sim_state": 0.02372180461883545, "sim_compute_performance-ego": 0.03936395486195882, "sim_compute_robot_state-ego": 0.042156432469685874}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 3.9258720003669567, "sim_physics": 0.0569548233350118, "survival_time": 14.950000000000076, "driven_lanedir": 3.899728383464023, "sim_render-ego": 0.036693987051645914, "in-drivable-lane": 0, "agent_compute-ego": 0.09167080958684286, "deviation-heading": 1.4321820554784217, "set_robot_commands": 0.05486665884653727, "deviation-center-line": 0.5586225233373633, "driven_lanedir_consec": 3.899728383464023, "sim_compute_sim_state": 0.02376578172047933, "sim_compute_performance-ego": 0.03944907347361247, "sim_compute_robot_state-ego": 0.04233535925547282}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.265969760966878, "sim_physics": 0.05597585229312672, "survival_time": 8.499999999999986, "driven_lanedir": 2.1276277327193047, "sim_render-ego": 0.03626457663143382, "in-drivable-lane": 0.550000000000006, "agent_compute-ego": 0.09174668087678796, "deviation-heading": 0.5735423589840908, "set_robot_commands": 0.05621168473187615, "deviation-center-line": 0.4091592109734886, "driven_lanedir_consec": 2.1276277327193047, "sim_compute_sim_state": 0.0233014653710758, "sim_compute_performance-ego": 0.039255226359647864, "sim_compute_robot_state-ego": 0.04164112595950856}}
set_robot_commands_max0.08002414226531983
set_robot_commands_mean0.0610955317952664
set_robot_commands_median0.05586991459131241
set_robot_commands_min0.05475284737988937
sim_compute_performance-ego_max0.05640427033106486
sim_compute_performance-ego_mean0.04400839768717632
sim_compute_performance-ego_median0.039837293326854706
sim_compute_performance-ego_min0.03917803619847153
sim_compute_robot_state-ego_max0.060709201494852705
sim_compute_robot_state-ego_mean0.046714456880405576
sim_compute_robot_state-ego_median0.04206309715906779
sim_compute_robot_state-ego_min0.04125054243839149
sim_compute_sim_state_max0.03358116149902344
sim_compute_sim_state_mean0.026129609437068373
sim_compute_sim_state_median0.02376578172047933
sim_compute_sim_state_min0.0233014653710758
sim_physics_max0.08792593965562291
sim_physics_mean0.06598981234679371
sim_physics_median0.059743280959340325
sim_physics_min0.05597585229312672
sim_render-ego_max0.052401159120642624
sim_render-ego_mean0.04051911610587631
sim_render-ego_median0.03675215323766073
sim_render-ego_min0.03626457663143382
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean11.946666666666705
survival_time_min4.799999999999991
213942770Bhairav Mehtachallenge-aido_LF-baseline-duckietownaido2-LFV-sim-testingstep1-simulationsuccessno8410:43:43(hidden)
driven_lanedir_consec_median0.9123243787763884
survival_time_median10.750000000000018
deviation-center-line_median0.8401391641801251
in-drivable-lane_median0.8000000000000114


other stats
agent_compute-ego_max0.05965746800104777
agent_compute-ego_mean0.05827673562800323
agent_compute-ego_median0.05860993349663565
agent_compute-ego_min0.05588133378071828
deviation-center-line_max1.3254902774697102
deviation-center-line_mean0.7836387520005754
deviation-center-line_min0.3462463700095799
deviation-heading_max8.203948820291735
deviation-heading_mean3.5893528998076336
deviation-heading_median3.1543430543398783
deviation-heading_min0.41738420828219047
driven_any_max2.3489092971560903
driven_any_mean1.625678921201286
driven_any_median1.673919820227727
driven_any_min0.4767286908213942
driven_lanedir_consec_max1.68686700980409
driven_lanedir_consec_mean0.9579607438723936
driven_lanedir_consec_min0.4721079162306081
driven_lanedir_max2.007573748040173
driven_lanedir_mean1.1762180823864623
driven_lanedir_median1.222842858408336
driven_lanedir_min0.4721079162306081
in-drivable-lane_max5.300000000000022
in-drivable-lane_mean1.6166666666666758
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.6823803407689155, "sim_physics": 0.10670253758056922, "survival_time": 10.85000000000002, "driven_lanedir": 0.6692713238692961, "sim_render-ego": 0.03857763242062336, "in-drivable-lane": 5.300000000000022, "agent_compute-ego": 0.05885137061369584, "deviation-heading": 2.5719761098890785, "set_robot_commands": 0.05859662640479303, "deviation-center-line": 0.4517891584153808, "driven_lanedir_consec": 0.6692713238692961, "sim_compute_sim_state": 0.02376609024364278, "sim_compute_performance-ego": 0.04203039819743776, "sim_compute_robot_state-ego": 0.04459167735367876, "sim_compute_robot_state-npc0": 0.04377930065453877, "sim_compute_robot_state-npc1": 0.0425926100823187, "sim_compute_robot_state-npc2": 0.04212952099637502, "sim_compute_robot_state-npc3": 0.042212854332638224}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.7303392770947092, "sim_physics": 0.10721149100913656, "survival_time": 11.100000000000025, "driven_lanedir": 1.68686700980409, "sim_render-ego": 0.037771296930742694, "in-drivable-lane": 0, "agent_compute-ego": 0.05588133378071828, "deviation-heading": 2.151915050028158, "set_robot_commands": 0.058009638442649496, "deviation-center-line": 1.0665400638166649, "driven_lanedir_consec": 1.68686700980409, "sim_compute_sim_state": 0.024374690141763772, "sim_compute_performance-ego": 0.0423021971642434, "sim_compute_robot_state-ego": 0.044218394133421754, "sim_compute_robot_state-npc0": 0.0435061197023134, "sim_compute_robot_state-npc1": 0.04274556658289454, "sim_compute_robot_state-npc2": 0.04214249323080252, "sim_compute_robot_state-npc3": 0.0424664374944326}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.3489092971560903, "sim_physics": 0.10245172103246052, "survival_time": 14.950000000000076, "driven_lanedir": 2.007573748040173, "sim_render-ego": 0.03924939235051473, "in-drivable-lane": 0.8000000000000114, "agent_compute-ego": 0.05794275760650635, "deviation-heading": 5.00262059526022, "set_robot_commands": 0.05944721857706706, "deviation-center-line": 0.993389498649148, "driven_lanedir_consec": 1.2887588538875463, "sim_compute_sim_state": 0.0244544514020284, "sim_compute_performance-ego": 0.04210793654123942, "sim_compute_robot_state-ego": 0.04421951691309611, "sim_compute_robot_state-npc0": 0.04374608596165975, "sim_compute_robot_state-npc1": 0.04250725666681925, "sim_compute_robot_state-npc2": 0.042092595100402835, "sim_compute_robot_state-npc3": 0.04233385801315308}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.3445374699102204, "sim_physics": 0.10624858154647653, "survival_time": 8.699999999999989, "driven_lanedir": 1.222842858408336, "sim_render-ego": 0.037906511076565445, "in-drivable-lane": 0.25000000000000355, "agent_compute-ego": 0.05808652132407002, "deviation-heading": 2.2346349488593984, "set_robot_commands": 0.05997782603077505, "deviation-center-line": 0.5897058680103656, "driven_lanedir_consec": 1.222842858408336, "sim_compute_sim_state": 0.024044949432899212, "sim_compute_performance-ego": 0.04200324518927213, "sim_compute_robot_state-ego": 0.04419016427007215, "sim_compute_robot_state-npc0": 0.04352433517061431, "sim_compute_robot_state-npc1": 0.042585159170216526, "sim_compute_robot_state-npc2": 0.042214833456894446, "sim_compute_robot_state-npc3": 0.04223454820698705}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.9749456721303108, "sim_physics": 0.1006455272436142, "survival_time": 6.399999999999985, "driven_lanedir": 0.9522686146248938, "sim_render-ego": 0.03873235359787941, "in-drivable-lane": 0, "agent_compute-ego": 0.05793498456478119, "deviation-heading": 1.2403648517800263, "set_robot_commands": 0.05861794576048851, "deviation-center-line": 0.5907748752576674, "driven_lanedir_consec": 0.9522686146248938, "sim_compute_sim_state": 0.024279598146677017, "sim_compute_performance-ego": 0.04383927583694458, "sim_compute_robot_state-ego": 0.04457597807049751, "sim_compute_robot_state-npc0": 0.043398091569542885, "sim_compute_robot_state-npc1": 0.04305802658200264, "sim_compute_robot_state-npc2": 0.04246487468481064, "sim_compute_robot_state-npc3": 0.04252323880791664}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 2.3486256381593633, "sim_physics": 0.10647994438807168, "survival_time": 14.950000000000076, "driven_lanedir": 1.359455946446225, "sim_render-ego": 0.03813084920247395, "in-drivable-lane": 3.700000000000011, "agent_compute-ego": 0.05965746800104777, "deviation-heading": 6.462302564727685, "set_robot_commands": 0.05837711811065674, "deviation-center-line": 0.9776247798493112, "driven_lanedir_consec": 0.9123243787763884, "sim_compute_sim_state": 0.02412455399831136, "sim_compute_performance-ego": 0.04250506321589152, "sim_compute_robot_state-ego": 0.044108311335245766, "sim_compute_robot_state-npc0": 0.04346088965733846, "sim_compute_robot_state-npc1": 0.04235200643539429, "sim_compute_robot_state-npc2": 0.04234007358551026, "sim_compute_robot_state-npc3": 0.042103786468505856}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.7819776689573992, "sim_physics": 0.1016849485727457, "survival_time": 5.1999999999999895, "driven_lanedir": 0.608911598041536, "sim_render-ego": 0.03842656658245967, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.05880511953280522, "deviation-heading": 2.143576631576528, "set_robot_commands": 0.05821317205062279, "deviation-center-line": 0.3462463700095799, "driven_lanedir_consec": 0.608911598041536, "sim_compute_sim_state": 0.023937649451769315, "sim_compute_performance-ego": 0.04218316078186035, "sim_compute_robot_state-ego": 0.04417281196667598, "sim_compute_robot_state-npc0": 0.04313109471247746, "sim_compute_robot_state-npc1": 0.04207759637099046, "sim_compute_robot_state-npc2": 0.04205070550625141, "sim_compute_robot_state-npc3": 0.042204561141821056}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 2.3487044919545426, "sim_physics": 0.10701319694519044, "survival_time": 14.950000000000076, "driven_lanedir": 1.770218300173262, "sim_render-ego": 0.03831885019938151, "in-drivable-lane": 1.9000000000000137, "agent_compute-ego": 0.059323450724283855, "deviation-heading": 5.059320566391891, "set_robot_commands": 0.058325248559316, "deviation-center-line": 1.3254902774697102, "driven_lanedir_consec": 1.285522685518215, "sim_compute_sim_state": 0.024126062393188475, "sim_compute_performance-ego": 0.042224965890248614, "sim_compute_robot_state-ego": 0.04427064100901286, "sim_compute_robot_state-npc0": 0.043597626686096194, "sim_compute_robot_state-npc1": 0.042778188387552894, "sim_compute_robot_state-npc2": 0.04242791175842285, "sim_compute_robot_state-npc3": 0.042549590269724526}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.673919820227727, "sim_physics": 0.10338190544483274, "survival_time": 10.750000000000018, "driven_lanedir": 0.7854647256721043, "sim_render-ego": 0.03826361589653547, "in-drivable-lane": 4.200000000000032, "agent_compute-ego": 0.0579537125520928, "deviation-heading": 3.661899820892573, "set_robot_commands": 0.058208207196967544, "deviation-center-line": 0.587745358244814, "driven_lanedir_consec": 0.5388222300952101, "sim_compute_sim_state": 0.024407846983089003, "sim_compute_performance-ego": 0.04223684044771416, "sim_compute_robot_state-ego": 0.04396756416143373, "sim_compute_robot_state-npc0": 0.04346896992173306, "sim_compute_robot_state-npc1": 0.042305130182310594, "sim_compute_robot_state-npc2": 0.04233479943386344, "sim_compute_robot_state-npc3": 0.0422666494236436}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 2.3484388721492984, "sim_physics": 0.10972349405288696, "survival_time": 14.950000000000076, "driven_lanedir": 1.5009477697778264, "sim_render-ego": 0.038226777712504066, "in-drivable-lane": 3.0500000000000203, "agent_compute-ego": 0.05900700171788534, "deviation-heading": 5.9373248506020415, "set_robot_commands": 0.05832138538360595, "deviation-center-line": 0.9438979256741552, "driven_lanedir_consec": 1.242104107343191, "sim_compute_sim_state": 0.02416032552719116, "sim_compute_performance-ego": 0.0420933723449707, "sim_compute_robot_state-ego": 0.04443740924199422, "sim_compute_robot_state-npc0": 0.04357576608657837, "sim_compute_robot_state-npc1": 0.04275109132130941, "sim_compute_robot_state-npc2": 0.04267366568247477, "sim_compute_robot_state-npc3": 0.042327348391215006}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.585654888418435, "sim_physics": 0.10237487624673282, "survival_time": 10.20000000000001, "driven_lanedir": 1.4744352450610392, "sim_render-ego": 0.038897115810244694, "in-drivable-lane": 0, "agent_compute-ego": 0.058198296556285785, "deviation-heading": 3.1543430543398783, "set_robot_commands": 0.05867948485355751, "deviation-center-line": 0.8528092854057571, "driven_lanedir_consec": 1.4744352450610392, "sim_compute_sim_state": 0.02410762450274299, "sim_compute_performance-ego": 0.04210492208892224, "sim_compute_robot_state-ego": 0.044658005237579346, "sim_compute_robot_state-npc0": 0.04349825662725112, "sim_compute_robot_state-npc1": 0.04428115428662768, "sim_compute_robot_state-npc2": 0.04215071598688761, "sim_compute_robot_state-npc3": 0.042440929833580464}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.4767286908213942, "sim_physics": 0.09802233811580774, "survival_time": 3.2999999999999963, "driven_lanedir": 0.4721079162306081, "sim_render-ego": 0.04268543286757036, "in-drivable-lane": 0, "agent_compute-ego": 0.05596309719663678, "deviation-heading": 0.41738420828219047, "set_robot_commands": 0.0580617008787213, "deviation-center-line": 0.3623109698312442, "driven_lanedir_consec": 0.4721079162306081, "sim_compute_sim_state": 0.024418722499500625, "sim_compute_performance-ego": 0.04206787817405932, "sim_compute_robot_state-ego": 0.044606411095821495, "sim_compute_robot_state-npc0": 0.044089953104654946, "sim_compute_robot_state-npc1": 0.04284774895870325, "sim_compute_robot_state-npc2": 0.04267680644989014, "sim_compute_robot_state-npc3": 0.04275186856587728}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.585310103991549, "sim_physics": 0.10312151908874512, "survival_time": 10.20000000000001, "driven_lanedir": 0.8599828863234054, "sim_render-ego": 0.03838019978766348, "in-drivable-lane": 2.6000000000000165, "agent_compute-ego": 0.05879516227572572, "deviation-heading": 4.792983305001075, "set_robot_commands": 0.05890029902551688, "deviation-center-line": 0.8401391641801251, "driven_lanedir_consec": 0.4736367642844863, "sim_compute_sim_state": 0.024420679784288593, "sim_compute_performance-ego": 0.04226536727419086, "sim_compute_robot_state-ego": 0.04449867734722063, "sim_compute_robot_state-npc0": 0.043783422778634465, "sim_compute_robot_state-npc1": 0.042890646878410786, "sim_compute_robot_state-npc2": 0.04237688756456562, "sim_compute_robot_state-npc3": 0.042351955292271634}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.8061964306101271, "sim_physics": 0.10281381651619884, "survival_time": 5.349999999999989, "driven_lanedir": 0.7936619156701687, "sim_render-ego": 0.03890626898435789, "in-drivable-lane": 0, "agent_compute-ego": 0.05860993349663565, "deviation-heading": 0.8056981191920325, "set_robot_commands": 0.05820601231583925, "deviation-center-line": 0.5854820816443561, "driven_lanedir_consec": 0.7936619156701687, "sim_compute_sim_state": 0.024405593069914345, "sim_compute_performance-ego": 0.042238986380746434, "sim_compute_robot_state-ego": 0.04482251907063422, "sim_compute_robot_state-npc0": 0.042800627022146064, "sim_compute_robot_state-npc1": 0.04411203170491156, "sim_compute_robot_state-npc2": 0.04208070763917727, "sim_compute_robot_state-npc3": 0.042092501560104224}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.3485151556692037, "sim_physics": 0.10703759272893268, "survival_time": 14.950000000000076, "driven_lanedir": 1.479261377653978, "sim_render-ego": 0.0385875391960144, "in-drivable-lane": 2.1500000000000083, "agent_compute-ego": 0.05914082447687785, "deviation-heading": 8.203948820291735, "set_robot_commands": 0.05879268805185954, "deviation-center-line": 1.2406356035503538, "driven_lanedir_consec": 0.7478756564708982, "sim_compute_sim_state": 0.024153931140899657, "sim_compute_performance-ego": 0.042215809027353925, "sim_compute_robot_state-ego": 0.044626909097035726, "sim_compute_robot_state-npc0": 0.04403238852818807, "sim_compute_robot_state-npc1": 0.042760533491770426, "sim_compute_robot_state-npc2": 0.0427531886100769, "sim_compute_robot_state-npc3": 0.042637540499369304}}
set_robot_commands_max0.05997782603077505
set_robot_commands_mean0.05858230477616244
set_robot_commands_median0.05837711811065674
set_robot_commands_min0.058009638442649496
sim_compute_performance-ego_max0.04383927583694458
sim_compute_performance-ego_mean0.04229462790367302
sim_compute_performance-ego_median0.042215809027353925
sim_compute_performance-ego_min0.04200324518927213
sim_compute_robot_state-ego_max0.04482251907063422
sim_compute_robot_state-ego_mean0.04439766602022801
sim_compute_robot_state-ego_median0.04443740924199422
sim_compute_robot_state-ego_min0.04396756416143373
sim_compute_robot_state-npc0_max0.044089953104654946
sim_compute_robot_state-npc0_mean0.04355952854558449
sim_compute_robot_state-npc0_median0.04352433517061431
sim_compute_robot_state-npc0_min0.042800627022146064
sim_compute_robot_state-npc1_max0.04428115428662768
sim_compute_robot_state-npc1_mean0.042842983140148866
sim_compute_robot_state-npc1_median0.04275109132130941
sim_compute_robot_state-npc1_min0.04207759637099046
sim_compute_robot_state-npc2_max0.0427531886100769
sim_compute_robot_state-npc2_mean0.04232731864576038
sim_compute_robot_state-npc2_median0.04233479943386344
sim_compute_robot_state-npc2_min0.04205070550625141
sim_compute_robot_state-npc3_max0.04275186856587728
sim_compute_robot_state-npc3_mean0.042366511220082705
sim_compute_robot_state-npc3_median0.04233385801315308
sim_compute_robot_state-npc3_min0.042092501560104224
sim_compute_sim_state_max0.0244544514020284
sim_compute_sim_state_mean0.02421218458119378
sim_compute_sim_state_median0.02416032552719116
sim_compute_sim_state_min0.02376609024364278
sim_physics_max0.10972349405288696
sim_physics_mean0.10432756603416012
sim_physics_median0.10338190544483274
sim_physics_min0.09802233811580774
sim_render-ego_max0.04268543286757036
sim_render-ego_mean0.03873736017436875
sim_render-ego_median0.03842656658245967
sim_render-ego_min0.037771296930742694
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean10.45333333333336
survival_time_min3.2999999999999963
213862450enickyBaseline solution using reinforcement learningaido2-LFVI-sim-testingstep1-simulationhost-errorno8410:59:10
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2450/step1-simulation-ip-172-31-38-104-5317-job21386/logs/challenges-runner/stdout.log'
(hidden)
213632373Liam Paullminimal_agent (Python 3)aido2-LFVI-sim-testingstep1-simulationhost-errorno8410:58:40
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2373/step1-simulation-ip-172-31-38-104-5317-job21363/logs/challenges-runner/stdout.log'
(hidden)
213332410jp18813100494Baseline solution using reinforcement learningaido2-LFV-sim-testingstep1-simulationhost-errorno8410:59:38
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission2410/step1-simulation-ip-172-31-38-104-5317-job21333/logs/challenges-runner/stdout.log'
(hidden)
213262399Liam Paullrandom_agentaido2-LFVI-sim-testingstep1-simulationhost-errorno8410:20:46
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2399/step1-simulation-ip-172-31-38-104-5317-job21326/logs/challenges-runner/stdout.log'
(hidden)
213152424Andrea Censirandom_agentaido2-LFVI-sim-testingstep1-simulationsuccessno8410:35:11(hidden)
driven_lanedir_consec_median0.6537162971975778
survival_time_median2.5999999999999988
deviation-center-line_median0.14884708769804722
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.16073807716369629
agent_compute-ego_mean0.15342428436262612
agent_compute-ego_median0.1530721691938547
agent_compute-ego_min0.14580322163445608
deviation-center-line_max0.5141120095458062
deviation-center-line_mean0.1769000227005324
deviation-center-line_min0.06216029860587694
deviation-heading_max1.9929437250081896
deviation-heading_mean0.692977823395608
deviation-heading_median0.4928440244444201
deviation-heading_min0.24234723632033853
driven_any_max3.5343244022418343
driven_any_mean1.3470385992234175
driven_any_median1.001633421251194
driven_any_min0.3412551960104244
driven_lanedir_consec_max2.494308053441959
driven_lanedir_consec_mean0.881391085527715
driven_lanedir_consec_min0.3199180625488627
driven_lanedir_max2.494308053441959
driven_lanedir_mean0.8962528018383226
driven_lanedir_median0.7206363324756615
driven_lanedir_min0.3199180625488627
in-drivable-lane_max3.549999999999991
in-drivable-lane_mean0.839999999999998
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.24381204194660427, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.07386509979827495, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.155749936647053, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.1142291026779368, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.04364463045627256, "sim_compute_performance-ego": 0.08165977876397627, "sim_compute_robot_state-ego": 0.08761180201663246, "sim_compute_robot_state-npc0": 0.07492863981029656, "sim_compute_robot_state-npc1": 0.07839610305013536, "sim_compute_robot_state-npc2": 0.07657041127168679, "sim_compute_robot_state-npc3": 0.07614021663424335}, "udem1-1-0": {"driven_any": 0.47798123795353353, "sim_physics": 0.2233950921467372, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.06527015992573329, "in-drivable-lane": 0, "agent_compute-ego": 0.14580322163445608, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.1075853875705174, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.04223714556012835, "sim_compute_performance-ego": 0.0739310554095677, "sim_compute_robot_state-ego": 0.08321171147482735, "sim_compute_robot_state-npc0": 0.0739713226045881, "sim_compute_robot_state-npc1": 0.07711518662316459, "sim_compute_robot_state-npc2": 0.07862592594964164, "sim_compute_robot_state-npc3": 0.07819525684629168}, "udem1-2-0": {"driven_any": 2.1864003300000694, "sim_physics": 0.22622832048286512, "survival_time": 5.14999999999999, "driven_lanedir": 1.826231945363564, "sim_render-ego": 0.06914765395007087, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.14955765761218023, "deviation-heading": 0.8115272458314897, "set_robot_commands": 0.10598744929415509, "deviation-center-line": 0.1927316041609401, "driven_lanedir_consec": 1.826231945363564, "sim_compute_sim_state": 0.04268207364869349, "sim_compute_performance-ego": 0.07506786272363755, "sim_compute_robot_state-ego": 0.08155349620337625, "sim_compute_robot_state-npc0": 0.07569983630504423, "sim_compute_robot_state-npc1": 0.07405796328794609, "sim_compute_robot_state-npc2": 0.07564127560958121, "sim_compute_robot_state-npc3": 0.0770037243667158}, "udem1-3-0": {"driven_any": 0.9410513078477724, "sim_physics": 0.26276538769404095, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.07006259262561798, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.15684201320012411, "deviation-heading": 0.35154027506548424, "set_robot_commands": 0.10881949464480084, "deviation-center-line": 0.08975118546699407, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.04372881849606832, "sim_compute_performance-ego": 0.08199055989583333, "sim_compute_robot_state-ego": 0.089857613046964, "sim_compute_robot_state-npc0": 0.07565573354562123, "sim_compute_robot_state-npc1": 0.08010820051034291, "sim_compute_robot_state-npc2": 0.07819710671901703, "sim_compute_robot_state-npc3": 0.07697765529155731}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.24389471326555529, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.06924450965154738, "in-drivable-lane": 0, "agent_compute-ego": 0.15570188703991117, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.10712427184695288, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.04261324519202823, "sim_compute_performance-ego": 0.07458718617757161, "sim_compute_robot_state-ego": 0.07991508075169154, "sim_compute_robot_state-npc0": 0.0753218446459089, "sim_compute_robot_state-npc1": 0.07842764400300525, "sim_compute_robot_state-npc2": 0.08146441550481887, "sim_compute_robot_state-npc3": 0.074594361441476}, "udem1-5-0": {"driven_any": 3.5343244022418343, "sim_physics": 0.22234635153216517, "survival_time": 8.349999999999984, "driven_lanedir": 2.494308053441959, "sim_render-ego": 0.0702948113401493, "in-drivable-lane": 2, "agent_compute-ego": 0.15330852434306802, "deviation-heading": 1.9929437250081896, "set_robot_commands": 0.10726580362833904, "deviation-center-line": 0.5141120095458062, "driven_lanedir_consec": 2.494308053441959, "sim_compute_sim_state": 0.043143686420189405, "sim_compute_performance-ego": 0.07719784296915203, "sim_compute_robot_state-ego": 0.08674887982671133, "sim_compute_robot_state-npc0": 0.07730037580707116, "sim_compute_robot_state-npc1": 0.07760465644790741, "sim_compute_robot_state-npc2": 0.07690965629623321, "sim_compute_robot_state-npc3": 0.07608343455605879}, "udem1-6-0": {"driven_any": 2.176040550989443, "sim_physics": 0.24984066463211207, "survival_time": 5.14999999999999, "driven_lanedir": 1.9748003483573369, "sim_render-ego": 0.07094486708779937, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.15889501108706575, "deviation-heading": 0.7877172510198454, "set_robot_commands": 0.1110233080039904, "deviation-center-line": 0.30661477465861414, "driven_lanedir_consec": 1.9748003483573369, "sim_compute_sim_state": 0.0453743193913432, "sim_compute_performance-ego": 0.0782510933366794, "sim_compute_robot_state-ego": 0.08853120016820222, "sim_compute_robot_state-npc0": 0.07741397792853198, "sim_compute_robot_state-npc1": 0.07549771753329675, "sim_compute_robot_state-npc2": 0.07659294304338474, "sim_compute_robot_state-npc3": 0.07668140559520537}, "udem1-7-0": {"driven_any": 1.0050236108346422, "sim_physics": 0.250516676902771, "survival_time": 2.499999999999999, "driven_lanedir": 0.8701796426684156, "sim_render-ego": 0.07094158172607422, "in-drivable-lane": 0, "agent_compute-ego": 0.14982558250427247, "deviation-heading": 0.9227446658975216, "set_robot_commands": 0.1076087236404419, "deviation-center-line": 0.14884708769804722, "driven_lanedir_consec": 0.8546116490381102, "sim_compute_sim_state": 0.04319440364837646, "sim_compute_performance-ego": 0.0765477991104126, "sim_compute_robot_state-ego": 0.08141589641571045, "sim_compute_robot_state-npc0": 0.07625813484191894, "sim_compute_robot_state-npc1": 0.07671495437622071, "sim_compute_robot_state-npc2": 0.07521331310272217, "sim_compute_robot_state-npc3": 0.07484870910644531}, "udem1-8-0": {"driven_any": 0.9970416603019576, "sim_physics": 0.22972691517609817, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8041152574540316, "sim_render-ego": 0.0695979457635146, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.1530721691938547, "deviation-heading": 0.6817155913229235, "set_robot_commands": 0.11183290756665744, "deviation-center-line": 0.1898768080806288, "driven_lanedir_consec": 0.8041152574540316, "sim_compute_sim_state": 0.04357925745157095, "sim_compute_performance-ego": 0.07507197673504169, "sim_compute_robot_state-ego": 0.0831971076818613, "sim_compute_robot_state-npc0": 0.07774172379420353, "sim_compute_robot_state-npc1": 0.07726647303654598, "sim_compute_robot_state-npc2": 0.07703361144432655, "sim_compute_robot_state-npc3": 0.07907112286641048}, "udem1-9-0": {"driven_any": 0.4374998710472299, "sim_physics": 0.27533595378582293, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4177527119659043, "sim_render-ego": 0.07506942749023438, "in-drivable-lane": 0, "agent_compute-ego": 0.15997163149026725, "deviation-heading": 0.3796137446285097, "set_robot_commands": 0.10749408831963173, "deviation-center-line": 0.08207813392324871, "driven_lanedir_consec": 0.4177527119659043, "sim_compute_sim_state": 0.044530896040109486, "sim_compute_performance-ego": 0.07566545559809758, "sim_compute_robot_state-ego": 0.08785674205193153, "sim_compute_robot_state-npc0": 0.07827877081357516, "sim_compute_robot_state-npc1": 0.0770726387317364, "sim_compute_robot_state-npc2": 0.07649886608123779, "sim_compute_robot_state-npc3": 0.07974606293898362}, "udem1-10-0": {"driven_any": 0.3412551960104244, "sim_physics": 0.2429753758690574, "survival_time": 1.1000000000000003, "driven_lanedir": 0.3199180625488627, "sim_render-ego": 0.07092180035331032, "in-drivable-lane": 0, "agent_compute-ego": 0.15220315889878708, "deviation-heading": 0.3796986873122644, "set_robot_commands": 0.10222483764995229, "deviation-center-line": 0.06276575300040536, "driven_lanedir_consec": 0.3199180625488627, "sim_compute_sim_state": 0.04170470887964422, "sim_compute_performance-ego": 0.07909877733750777, "sim_compute_robot_state-ego": 0.07950985431671143, "sim_compute_robot_state-npc0": 0.07761441577564586, "sim_compute_robot_state-npc1": 0.07218058542771773, "sim_compute_robot_state-npc2": 0.07513643394817006, "sim_compute_robot_state-npc3": 0.07901796427640048}, "udem1-11-0": {"driven_any": 0.9951985237696338, "sim_physics": 0.23459696292877197, "survival_time": 2.499999999999999, "driven_lanedir": 0.7206363324756615, "sim_render-ego": 0.06849984169006347, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.16073807716369629, "deviation-heading": 0.30074636349085415, "set_robot_commands": 0.114860782623291, "deviation-center-line": 0.16567670185555422, "driven_lanedir_consec": 0.7185075497455284, "sim_compute_sim_state": 0.0422287654876709, "sim_compute_performance-ego": 0.07877715110778809, "sim_compute_robot_state-ego": 0.09415372848510742, "sim_compute_robot_state-npc0": 0.07819255828857422, "sim_compute_robot_state-npc1": 0.08025811672210693, "sim_compute_robot_state-npc2": 0.07727081775665283, "sim_compute_robot_state-npc3": 0.07529214382171631}, "udem1-12-0": {"driven_any": 1.964687870253437, "sim_physics": 0.23422106529803985, "survival_time": 4.699999999999991, "driven_lanedir": 0.35162821871931205, "sim_render-ego": 0.0682218683526871, "in-drivable-lane": 3.549999999999991, "agent_compute-ego": 0.14990516926379913, "deviation-heading": 0.24234723632033853, "set_robot_commands": 0.10414016754069227, "deviation-center-line": 0.13126670634798973, "driven_lanedir_consec": 0.35162821871931205, "sim_compute_sim_state": 0.04384088262598565, "sim_compute_performance-ego": 0.07758270172362632, "sim_compute_robot_state-ego": 0.08503055826146552, "sim_compute_robot_state-npc0": 0.0747975009552976, "sim_compute_robot_state-npc1": 0.07501454809878735, "sim_compute_robot_state-npc2": 0.07627192710308318, "sim_compute_robot_state-npc3": 0.07865206738735767}, "udem1-13-0": {"driven_any": 1.001633421251194, "sim_physics": 0.2413217333646921, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6537162971975778, "sim_render-ego": 0.071205001610976, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.14992090371938852, "deviation-heading": 0.4928440244444201, "set_robot_commands": 0.10321240241710956, "deviation-center-line": 0.13361893417935136, "driven_lanedir_consec": 0.6537162971975778, "sim_compute_sim_state": 0.04312876554635855, "sim_compute_performance-ego": 0.07784241896409255, "sim_compute_robot_state-ego": 0.08608456299855159, "sim_compute_robot_state-npc0": 0.0743665282542889, "sim_compute_robot_state-npc1": 0.07726682607944195, "sim_compute_robot_state-npc2": 0.07598881538097675, "sim_compute_robot_state-npc3": 0.0763402902162992}, "udem1-14-0": {"driven_any": 2.218769782727203, "sim_physics": 0.22500669388543992, "survival_time": 5.249999999999989, "driven_lanedir": 0.8125963661433656, "sim_render-ego": 0.0704779965536935, "in-drivable-lane": 2.499999999999993, "agent_compute-ego": 0.14986932164146788, "deviation-heading": 1.7163026916317128, "set_robot_commands": 0.1078948974609375, "deviation-center-line": 0.24126933160503777, "driven_lanedir_consec": 0.6077880672015845, "sim_compute_sim_state": 0.042055881591070265, "sim_compute_performance-ego": 0.07553659166608538, "sim_compute_robot_state-ego": 0.08327046803065709, "sim_compute_robot_state-npc0": 0.07954365866524833, "sim_compute_robot_state-npc1": 0.07815580822172619, "sim_compute_robot_state-npc2": 0.07803193501063756, "sim_compute_robot_state-npc3": 0.07859435989743187}}
set_robot_commands_max0.114860782623291
set_robot_commands_mean0.10808690832569376
set_robot_commands_median0.1075853875705174
set_robot_commands_min0.10222483764995229
sim_compute_performance-ego_max0.08199055989583333
sim_compute_performance-ego_mean0.07725388343460465
sim_compute_performance-ego_median0.07719784296915203
sim_compute_performance-ego_min0.0739310554095677
sim_compute_robot_state-ego_max0.09415372848510742
sim_compute_robot_state-ego_mean0.08519658011536009
sim_compute_robot_state-ego_median0.08503055826146552
sim_compute_robot_state-ego_min0.07950985431671143
sim_compute_robot_state-npc0_max0.07954365866524833
sim_compute_robot_state-npc0_mean0.07647233480238765
sim_compute_robot_state-npc0_median0.07625813484191894
sim_compute_robot_state-npc0_min0.0739713226045881
sim_compute_robot_state-npc1_max0.08025811672210693
sim_compute_robot_state-npc1_mean0.07700916147667211
sim_compute_robot_state-npc1_median0.07726647303654598
sim_compute_robot_state-npc1_min0.07218058542771773
sim_compute_robot_state-npc2_max0.08146441550481887
sim_compute_robot_state-npc2_mean0.07702983028147802
sim_compute_robot_state-npc2_median0.07659294304338474
sim_compute_robot_state-npc2_min0.07513643394817006
sim_compute_robot_state-npc3_max0.07974606293898362
sim_compute_robot_state-npc3_mean0.07714925168283956
sim_compute_robot_state-npc3_median0.07697765529155731
sim_compute_robot_state-npc3_min0.074594361441476
sim_compute_sim_state_max0.0453743193913432
sim_compute_sim_state_mean0.04317916536236733
sim_compute_sim_state_median0.043143686420189405
sim_compute_sim_state_min0.04170470887964422
sim_physics_max0.27533595378582293
sim_physics_mean0.2403989299273849
sim_physics_median0.2413217333646921
sim_physics_min0.22234635153216517
sim_render-ego_max0.07506942749023438
sim_render-ego_mean0.07025101052798312
sim_render-ego_median0.0702948113401493
sim_render-ego_min0.06527015992573329
simulation-passed1
survival_time_max8.349999999999984
survival_time_mean3.3333333333333286
survival_time_min1.0500000000000005
213082408jp18813100494Baseline solution using reinforcement learningaido2-LF-sim-testingstep1-simulationerrorno8410:03:24
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 87, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 87, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 > 
(hidden)
212822446enickyBaseline solution using reinforcement learningaido2-LF-sim-testingstep1-simulationhost-errorno8410:59:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-testing/submission2446/step1-simulation-ip-172-31-38-104-5317-job21282/logs/challenges-runner/stdout.log'
(hidden)
212652448enickyBaseline solution using reinforcement learningaido2-LFV-sim-testingstep1-simulationhost-errorno8410:28:08
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission2448/step1-simulation-ip-172-31-38-104-5317-job21265/logs/challenges-runner/stdout.log'
(hidden)
212592456Liam Paullchallenge-aido_LF-template-rosaido2-LF-sim-validationstep1-simulationerrorno8410:02:03
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
(hidden)
212512468Liam Paullchallenge-aido_LF-template-randomaido2-LF-sim-validationstep1-simulationsuccessno8410:05:51(hidden)
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.13005513204193506
in-drivable-lane_median0.29999999999999893


other stats
agent_compute-ego_max0.16732219413474755
agent_compute-ego_mean0.15764633144020873
agent_compute-ego_median0.15869702544866823
agent_compute-ego_min0.14481379412397555
deviation-center-line_max0.2072381278624381
deviation-center-line_mean0.13189395308989768
deviation-center-line_min0.0788839760023766
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3206717506075888
deviation-heading_median0.23606825392187256
deviation-heading_min0.19401296908406307
driven_any_max1.618214457700942
driven_any_mean0.8613785859332687
driven_any_median0.9933867744747804
driven_any_min0.23626560261560617
driven_lanedir_consec_max0.8010406009604816
driven_lanedir_consec_mean0.5597645764431625
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max0.8010406009604816
driven_lanedir_mean0.5597645764431625
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.619999999999998
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.12739171805205168, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.06849290706493237, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.16732219413474755, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.10541092466425012, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.04180708196428087, "sim_compute_performance-ego": 0.07786099116007487, "sim_compute_robot_state-ego": 0.0873853188973886}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.1205905699262432, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.06104382346658146, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.15869702544866823, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.10037673688402364, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.04041080848843444, "sim_compute_performance-ego": 0.07212064312953576, "sim_compute_robot_state-ego": 0.07830369238759957}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.13688300637637868, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.06412745924556956, "in-drivable-lane": 0, "agent_compute-ego": 0.1573047637939453, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.10260789534624884, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.038603993023143095, "sim_compute_performance-ego": 0.07892335162443273, "sim_compute_robot_state-ego": 0.08196673673741958}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.12029380798339843, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.06038158416748047, "in-drivable-lane": 0, "agent_compute-ego": 0.16009387969970704, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.09873509407043456, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.04060305595397949, "sim_compute_performance-ego": 0.06693792343139648, "sim_compute_robot_state-ego": 0.0737933349609375}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.618214457700942, "sim_physics": 0.10465933703169038, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.06166241440591933, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.14481379412397555, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.0910407651828814, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.03671266459211518, "sim_compute_performance-ego": 0.06329412098172345, "sim_compute_robot_state-ego": 0.06902350655084924}}
set_robot_commands_max0.10541092466425012
set_robot_commands_mean0.09963428322956772
set_robot_commands_median0.10037673688402364
set_robot_commands_min0.0910407651828814
sim_compute_performance-ego_max0.07892335162443273
sim_compute_performance-ego_mean0.07182740606543267
sim_compute_performance-ego_median0.07212064312953576
sim_compute_performance-ego_min0.06329412098172345
sim_compute_robot_state-ego_max0.0873853188973886
sim_compute_robot_state-ego_mean0.0780945179068389
sim_compute_robot_state-ego_median0.07830369238759957
sim_compute_robot_state-ego_min0.06902350655084924
sim_compute_sim_state_max0.04180708196428087
sim_compute_sim_state_mean0.039627520804390616
sim_compute_sim_state_median0.04041080848843444
sim_compute_sim_state_min0.03671266459211518
sim_physics_max0.13688300637637868
sim_physics_mean0.12196368787395248
sim_physics_median0.1205905699262432
sim_physics_min0.10465933703169038
sim_render-ego_max0.06849290706493237
sim_render-ego_mean0.06314163767009665
sim_render-ego_median0.06166241440591933
sim_render-ego_min0.06038158416748047
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.2599999999999985
survival_time_min0.8500000000000002
212172544Liam Paullchallenge-aido_LF-template-rosaido2-LF-sim-testingstep1-simulationsuccessno8410:17:41(hidden)
driven_lanedir_consec_median0.4126113388371666
survival_time_median2.6999999999999984
deviation-center-line_median0.11871775985231396
in-drivable-lane_median0.9499999999999976


other stats
agent_compute-ego_max0.07738031943639119
agent_compute-ego_mean0.07231898988919436
agent_compute-ego_median0.07297790902001518
agent_compute-ego_min0.06675512591997783
deviation-center-line_max0.2616199012603187
deviation-center-line_mean0.12663752615997417
deviation-center-line_min0.07143676193760398
deviation-heading_max1.4563581054905994
deviation-heading_mean0.5551840294909584
deviation-heading_median0.55940705176748
deviation-heading_min0.2596285487829774
driven_any_max1.6839250134630197
driven_any_mean0.7327294413370017
driven_any_median0.7309130538792373
driven_any_min0.2364518949491652
driven_lanedir_consec_max0.9416535020294268
driven_lanedir_consec_mean0.4469104002974222
driven_lanedir_consec_min0.2221238695940002
driven_lanedir_max0.9416535020294268
driven_lanedir_mean0.4469104002974222
driven_lanedir_median0.4126113388371666
driven_lanedir_min0.2221238695940002
in-drivable-lane_max2.1499999999999924
in-drivable-lane_mean0.8233333333333307
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.922682252912863, "sim_physics": 0.12814242739072987, "survival_time": 3.5499999999999954, "driven_lanedir": 0.36592094708219225, "sim_render-ego": 0.06411490641849142, "in-drivable-lane": 1.8999999999999944, "agent_compute-ego": 0.07324084765474562, "deviation-heading": 0.2596285487829774, "set_robot_commands": 0.10161597627989004, "deviation-center-line": 0.16987818035809085, "driven_lanedir_consec": 0.36592094708219225, "sim_compute_sim_state": 0.04141790430310746, "sim_compute_performance-ego": 0.07204385878334582, "sim_compute_robot_state-ego": 0.08214142960561834}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.7993934895811189, "sim_physics": 0.12073736588160196, "survival_time": 2.9999999999999973, "driven_lanedir": 0.6632135184572027, "sim_render-ego": 0.06475526094436646, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.07243167559305827, "deviation-heading": 0.5809378480871981, "set_robot_commands": 0.1005508820215861, "deviation-center-line": 0.1166759393545397, "driven_lanedir_consec": 0.6632135184572027, "sim_compute_sim_state": 0.041137484709421794, "sim_compute_performance-ego": 0.07361146608988443, "sim_compute_robot_state-ego": 0.07742774883906046}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.2364518949491652, "sim_physics": 0.11456064744429156, "survival_time": 1.1000000000000003, "driven_lanedir": 0.2221238695940002, "sim_render-ego": 0.06457213921980424, "in-drivable-lane": 0, "agent_compute-ego": 0.06781026450070468, "deviation-heading": 0.34882560565471876, "set_robot_commands": 0.09554320031946356, "deviation-center-line": 0.11316665887359229, "driven_lanedir_consec": 0.2221238695940002, "sim_compute_sim_state": 0.03954731334339489, "sim_compute_performance-ego": 0.07232692024924538, "sim_compute_robot_state-ego": 0.07865115729245273}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.5946812776947944, "sim_physics": 0.13727021217346191, "survival_time": 2.3999999999999995, "driven_lanedir": 0.5792955579972934, "sim_render-ego": 0.06728161374727885, "in-drivable-lane": 0, "agent_compute-ego": 0.07738031943639119, "deviation-heading": 0.5685240160177978, "set_robot_commands": 0.11204977333545683, "deviation-center-line": 0.15908349194402024, "driven_lanedir_consec": 0.5792955579972934, "sim_compute_sim_state": 0.04309699932734171, "sim_compute_performance-ego": 0.07665433486302693, "sim_compute_robot_state-ego": 0.08929054935773213}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.609146202404747, "sim_physics": 0.12735274563664975, "survival_time": 2.3, "driven_lanedir": 0.2919817227673325, "sim_render-ego": 0.06504191004711649, "in-drivable-lane": 0.9499999999999992, "agent_compute-ego": 0.07619613668192989, "deviation-heading": 0.3390861746033595, "set_robot_commands": 0.10620496065720268, "deviation-center-line": 0.1011892720306074, "driven_lanedir_consec": 0.2919817227673325, "sim_compute_sim_state": 0.04256815495698348, "sim_compute_performance-ego": 0.0738124536431354, "sim_compute_robot_state-ego": 0.08029303343399712}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.32177231722795413, "sim_physics": 0.12168269356091815, "survival_time": 1.2000000000000004, "driven_lanedir": 0.259671941523266, "sim_render-ego": 0.0633099873860677, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.06675512591997783, "deviation-heading": 0.55940705176748, "set_robot_commands": 0.0972947080930074, "deviation-center-line": 0.07983398841276981, "driven_lanedir_consec": 0.259671941523266, "sim_compute_sim_state": 0.04168285926183065, "sim_compute_performance-ego": 0.07071420550346375, "sim_compute_robot_state-ego": 0.07721539338429768}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.9218874635466072, "sim_physics": 0.11822570898594, "survival_time": 3.899999999999994, "driven_lanedir": 0.5450627289344152, "sim_render-ego": 0.06489226145622058, "in-drivable-lane": 1.2999999999999954, "agent_compute-ego": 0.07456938120035025, "deviation-heading": 0.6586708623908946, "set_robot_commands": 0.1027975449195275, "deviation-center-line": 0.13114121213460145, "driven_lanedir_consec": 0.5450627289344152, "sim_compute_sim_state": 0.04067861728179149, "sim_compute_performance-ego": 0.07342082720536453, "sim_compute_robot_state-ego": 0.07936207147744986}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.8035998912048764, "sim_physics": 0.12117667709078107, "survival_time": 2.799999999999998, "driven_lanedir": 0.4126113388371666, "sim_render-ego": 0.06479936838150024, "in-drivable-lane": 0.9999999999999972, "agent_compute-ego": 0.07297790902001518, "deviation-heading": 0.6465417997696941, "set_robot_commands": 0.10056817957333156, "deviation-center-line": 0.119844131874105, "driven_lanedir_consec": 0.4126113388371666, "sim_compute_sim_state": 0.040023556777409146, "sim_compute_performance-ego": 0.07299048985753741, "sim_compute_robot_state-ego": 0.08239869134766716}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.082025040886041, "sim_physics": 0.12023616385185856, "survival_time": 4.3499999999999925, "driven_lanedir": 0.4507633352947824, "sim_render-ego": 0.06726350455448546, "in-drivable-lane": 2.1499999999999924, "agent_compute-ego": 0.07300725202450807, "deviation-heading": 0.6137519362826567, "set_robot_commands": 0.10460495400702816, "deviation-center-line": 0.11871775985231396, "driven_lanedir_consec": 0.4507633352947824, "sim_compute_sim_state": 0.04303488786193146, "sim_compute_performance-ego": 0.07403587472849879, "sim_compute_robot_state-ego": 0.07980590305109134}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.6839250134630197, "sim_physics": 0.12791882211511785, "survival_time": 5.4999999999999885, "driven_lanedir": 0.9416535020294268, "sim_render-ego": 0.0646074121648615, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.07464621717279607, "deviation-heading": 1.4563581054905994, "set_robot_commands": 0.10251006429845635, "deviation-center-line": 0.2616199012603187, "driven_lanedir_consec": 0.9416535020294268, "sim_compute_sim_state": 0.0407594160600142, "sim_compute_performance-ego": 0.07365111871199174, "sim_compute_robot_state-ego": 0.07989509755914861}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.2667775284624888, "sim_physics": 0.14952253301938376, "survival_time": 1.2000000000000004, "driven_lanedir": 0.25235480657068055, "sim_render-ego": 0.0695716142654419, "in-drivable-lane": 0, "agent_compute-ego": 0.07116659482320149, "deviation-heading": 0.2735028974766809, "set_robot_commands": 0.09999000032742818, "deviation-center-line": 0.10630271043038582, "driven_lanedir_consec": 0.25235480657068055, "sim_compute_sim_state": 0.040599286556243896, "sim_compute_performance-ego": 0.07786126931508382, "sim_compute_robot_state-ego": 0.08712355295817058}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.0389325864387626, "sim_physics": 0.12102425872505486, "survival_time": 3.8499999999999943, "driven_lanedir": 0.4755729783153157, "sim_render-ego": 0.06543002809797015, "in-drivable-lane": 1.7499999999999938, "agent_compute-ego": 0.07338796033487692, "deviation-heading": 0.521699236138543, "set_robot_commands": 0.1008709529777626, "deviation-center-line": 0.10517150645857888, "driven_lanedir_consec": 0.4755729783153157, "sim_compute_sim_state": 0.03890473811657398, "sim_compute_performance-ego": 0.07098787790769107, "sim_compute_robot_state-ego": 0.07927745658081847}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.6835794874956865, "sim_physics": 0.1199024486541748, "survival_time": 2.499999999999999, "driven_lanedir": 0.6100305079584653, "sim_render-ego": 0.06486196041107178, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.07159759998321533, "deviation-heading": 0.4739455376152246, "set_robot_commands": 0.10289566040039062, "deviation-center-line": 0.12143860235134882, "driven_lanedir_consec": 0.6100305079584653, "sim_compute_sim_state": 0.04166799068450928, "sim_compute_performance-ego": 0.07203013420104981, "sim_compute_robot_state-ego": 0.07675758838653564}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.29517411990766207, "sim_physics": 0.13444508956028864, "survival_time": 1.3000000000000005, "driven_lanedir": 0.25251967754950355, "sim_render-ego": 0.0655203782595121, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.07068674380962665, "deviation-heading": 0.466101326638835, "set_robot_commands": 0.1038358761714055, "deviation-center-line": 0.07143676193760398, "driven_lanedir_consec": 0.25251967754950355, "sim_compute_sim_state": 0.04171520013075609, "sim_compute_performance-ego": 0.07482492923736572, "sim_compute_robot_state-ego": 0.08714752930861253}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.7309130538792373, "sim_physics": 0.1184472816961783, "survival_time": 2.6999999999999984, "driven_lanedir": 0.38087957155028906, "sim_render-ego": 0.06433938609229194, "in-drivable-lane": 0.9499999999999976, "agent_compute-ego": 0.06893082018251773, "deviation-heading": 0.5607794956477159, "set_robot_commands": 0.09222431977589925, "deviation-center-line": 0.12406277512673576, "driven_lanedir_consec": 0.38087957155028906, "sim_compute_sim_state": 0.04074898472538701, "sim_compute_performance-ego": 0.06837835135283293, "sim_compute_robot_state-ego": 0.06904760113468876}}
set_robot_commands_max0.11204977333545683
set_robot_commands_mean0.1015704702105224
set_robot_commands_median0.10161597627989004
set_robot_commands_min0.09222431977589925
sim_compute_performance-ego_max0.07786126931508382
sim_compute_performance-ego_mean0.07315627410996783
sim_compute_performance-ego_median0.07342082720536453
sim_compute_performance-ego_min0.06837835135283293
sim_compute_robot_state-ego_max0.08929054935773213
sim_compute_robot_state-ego_mean0.08038898691448942
sim_compute_robot_state-ego_median0.07980590305109134
sim_compute_robot_state-ego_min0.06904760113468876
sim_compute_sim_state_max0.04309699932734171
sim_compute_sim_state_mean0.04117222627311311
sim_compute_sim_state_median0.041137484709421794
sim_compute_sim_state_min0.03890473811657398
sim_physics_max0.14952253301938376
sim_physics_mean0.1253763383857621
sim_physics_median0.12117667709078107
sim_physics_min0.11456064744429156
sim_render-ego_max0.0695716142654419
sim_render-ego_mean0.06535744876309872
sim_render-ego_median0.06486196041107178
sim_render-ego_min0.0633099873860677
simulation-passed1
survival_time_max5.4999999999999885
survival_time_mean2.7766666666666637
survival_time_min1.1000000000000003
212082529Liam Paullbaseline-IL-sim-tensorflowaido2-LF-sim-validationstep1-simulationerrorno8410:02:29
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
(hidden)
211742884Andrea Danieleminimal_agent (Python 3)aido2-LFVI-sim-testingstep1-simulationhost-errorno8410:57:12
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2884/step1-simulation-ip-172-31-38-104-5317-job21174/logs/challenges-runner/stdout.log'
(hidden)
211602591Julian Zillybaseline-IL-logs-tensorflowaido2-LFV-sim-validationstep1-simulationerrorno8410:01:52
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 87, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 87, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 > 
(hidden)
211252698Konstantin Chaykachallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-testingstep1-simulationhost-errorno8410:47:02
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2698/step1-simulation-ip-172-31-38-104-5317-job21125/logs/challenges-runner/stdout.log'
(hidden)
211172633Andrea Censichallenge-aido_LF-template-randomaido2-LFV-sim-validationstep1-simulationsuccessno8410:10:14(hidden)
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.6999999999999984
deviation-center-line_median0.11519544839797546
in-drivable-lane_median0.5999999999999979


other stats
agent_compute-ego_max0.18026463291313077
agent_compute-ego_mean0.16680291433092545
agent_compute-ego_median0.1750034800282231
agent_compute-ego_min0.14417771602931775
deviation-center-line_max0.1743913456502906
deviation-center-line_mean0.12651756552531995
deviation-center-line_min0.09047885846140293
deviation-heading_max0.5020407942599273
deviation-heading_mean0.3641979127291371
deviation-heading_median0.34764983431811214
deviation-heading_min0.19401296908406307
driven_any_max1.6169932854589035
driven_any_mean1.0272530755098266
driven_any_median1.0243289631429708
driven_any_min0.4119037580118998
driven_lanedir_consec_max1.2433038249708377
driven_lanedir_consec_mean0.7800316850006626
driven_lanedir_consec_min0.3792793194470208
driven_lanedir_max1.2433038249708377
driven_lanedir_mean0.7800316850006626
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.3792793194470208
in-drivable-lane_max0.9999999999999976
in-drivable-lane_mean0.4899999999999984
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.2194281463269834, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.06444881138978181, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.1750034800282231, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.1085559571230853, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.04073879895386873, "sim_compute_performance-ego": 0.07654258056923195, "sim_compute_robot_state-ego": 0.08613808508272525, "sim_compute_robot_state-npc0": 0.07554284731547038, "sim_compute_robot_state-npc1": 0.07860416836208767, "sim_compute_robot_state-npc2": 0.07472786196955929, "sim_compute_robot_state-npc3": 0.07446683336187292}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4119037580118998, "sim_physics": 0.20434903144836425, "survival_time": 1.2500000000000004, "driven_lanedir": 0.3792793194470208, "sim_render-ego": 0.06643013954162598, "in-drivable-lane": 0, "agent_compute-ego": 0.1565767478942871, "deviation-heading": 0.47860435001913393, "set_robot_commands": 0.10517814636230469, "deviation-center-line": 0.09344288799285194, "driven_lanedir_consec": 0.3792793194470208, "sim_compute_sim_state": 0.04146278381347656, "sim_compute_performance-ego": 0.07114498138427734, "sim_compute_robot_state-ego": 0.07924938201904297, "sim_compute_robot_state-npc0": 0.0754024314880371, "sim_compute_robot_state-npc1": 0.07255032539367676, "sim_compute_robot_state-npc2": 0.07402278900146485, "sim_compute_robot_state-npc3": 0.07287243843078613}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.579862082650716, "sim_physics": 0.1709588985694082, "survival_time": 3.7999999999999945, "driven_lanedir": 1.2433038249708377, "sim_render-ego": 0.05637203392229582, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.14417771602931775, "deviation-heading": 0.5020407942599273, "set_robot_commands": 0.09242032389891774, "deviation-center-line": 0.1743913456502906, "driven_lanedir_consec": 1.2433038249708377, "sim_compute_sim_state": 0.03708664367073461, "sim_compute_performance-ego": 0.06529947017368518, "sim_compute_robot_state-ego": 0.06902218806116205, "sim_compute_robot_state-npc0": 0.06623543877350657, "sim_compute_robot_state-npc1": 0.06653680926875065, "sim_compute_robot_state-npc2": 0.06696141393561113, "sim_compute_robot_state-npc3": 0.06405956180472124}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6169932854589035, "sim_physics": 0.23113025592852243, "survival_time": 3.949999999999994, "driven_lanedir": 1.2126117310470583, "sim_render-ego": 0.07345211053196388, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.18026463291313077, "deviation-heading": 0.34764983431811214, "set_robot_commands": 0.1169282213042054, "deviation-center-line": 0.11519544839797546, "driven_lanedir_consec": 1.2126117310470583, "sim_compute_sim_state": 0.04506051389476921, "sim_compute_performance-ego": 0.08211629300177851, "sim_compute_robot_state-ego": 0.09102447727058507, "sim_compute_robot_state-npc0": 0.08114040048816536, "sim_compute_robot_state-npc1": 0.08045436460760576, "sim_compute_robot_state-npc2": 0.0821246074724801, "sim_compute_robot_state-npc3": 0.08131971842126001}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5031772882846431, "sim_physics": 0.22678689445768083, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4926325270527603, "sim_render-ego": 0.07303336688450404, "in-drivable-lane": 0, "agent_compute-ego": 0.17799199478966848, "deviation-heading": 0.29868161596444953, "set_robot_commands": 0.1207939386367798, "deviation-center-line": 0.09047885846140293, "driven_lanedir_consec": 0.4926325270527603, "sim_compute_sim_state": 0.04551115206309727, "sim_compute_performance-ego": 0.08283446516309466, "sim_compute_robot_state-ego": 0.09840757506234306, "sim_compute_robot_state-npc0": 0.08046126365661621, "sim_compute_robot_state-npc1": 0.08256052221570696, "sim_compute_robot_state-npc2": 0.08176023619515556, "sim_compute_robot_state-npc3": 0.08036643266677856}}
set_robot_commands_max0.1207939386367798
set_robot_commands_mean0.10877531746505856
set_robot_commands_median0.1085559571230853
set_robot_commands_min0.09242032389891774
sim_compute_performance-ego_max0.08283446516309466
sim_compute_performance-ego_mean0.07558755805841352
sim_compute_performance-ego_median0.07654258056923195
sim_compute_performance-ego_min0.06529947017368518
sim_compute_robot_state-ego_max0.09840757506234306
sim_compute_robot_state-ego_mean0.08476834149917169
sim_compute_robot_state-ego_median0.08613808508272525
sim_compute_robot_state-ego_min0.06902218806116205
sim_compute_robot_state-npc0_max0.08114040048816536
sim_compute_robot_state-npc0_mean0.07575647634435913
sim_compute_robot_state-npc0_median0.07554284731547038
sim_compute_robot_state-npc0_min0.06623543877350657
sim_compute_robot_state-npc1_max0.08256052221570696
sim_compute_robot_state-npc1_mean0.07614123796956555
sim_compute_robot_state-npc1_median0.07860416836208767
sim_compute_robot_state-npc1_min0.06653680926875065
sim_compute_robot_state-npc2_max0.0821246074724801
sim_compute_robot_state-npc2_mean0.07591938171485418
sim_compute_robot_state-npc2_median0.07472786196955929
sim_compute_robot_state-npc2_min0.06696141393561113
sim_compute_robot_state-npc3_max0.08131971842126001
sim_compute_robot_state-npc3_mean0.07461699693708376
sim_compute_robot_state-npc3_median0.07446683336187292
sim_compute_robot_state-npc3_min0.06405956180472124
sim_compute_sim_state_max0.04551115206309727
sim_compute_sim_state_mean0.04197197847918928
sim_compute_sim_state_median0.04146278381347656
sim_compute_sim_state_min0.03708664367073461
sim_physics_max0.23113025592852243
sim_physics_mean0.2105306453461918
sim_physics_median0.2194281463269834
sim_physics_min0.1709588985694082
sim_render-ego_max0.07345211053196388
sim_render-ego_mean0.0667472924540343
sim_render-ego_median0.06643013954162598
sim_render-ego_min0.05637203392229582
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.6199999999999974
survival_time_min1.2500000000000004
210882864AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-testingstep1-simulationhost-errorno8410:56:45
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2864/step1-simulation-ip-172-31-38-104-5317-job21088/logs/challenges-runner/stdout.log'
(hidden)
210712707RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-testingstep1-simulationhost-errorno8410:11:06
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission2707/step1-simulation-ip-172-31-38-104-5317-job21071/logs/challenges-runner/stdout.log'
(hidden)
210633110ssviridovBaselineaido2-AMOD-fleet_sizestep2-scoringsuccessyes8410:01:23(hidden)
fleet_size-1000000000


other stats
efficiency-62.9040012273548
service_quality-32.277470306839234
210602725jp18813100494test for ppoaido2-LF-sim-testingstep1-simulationerrorno8410:02:02
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 305, in run_episode
    sim_ci.write_topic_and_expect_zero('set_robot_commands', commands)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "set_robot_commands".
                   |
                   || Traceback (most recent call last):
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "gym_bridge.py", line 301, in on_received_set_robot_commands
                   ||     raise Exception(msg)
                   || Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}.
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 135, in main
 >     physics_dt=config.physics_dt)
 >   File "experiment_manager.py", line 305, in run_episode
 >     sim_ci.write_topic_and_expect_zero('set_robot_commands', commands)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
 >     others = read_until_over(fpout, timeout=timeout, nickname=nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
 >     raise RemoteNodeAborted(m)
 > zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
 > 
 > error in simulator |Exception while handling a message on topic "set_robot_commands".
 >                    |
 >                    || Traceback (most recent call last):
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
 >                    ||     handle_message_node(parsed, receiver0, context0)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
 >                    ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
 >                    ||     f(**kwargs)
 >                    ||   File "gym_bridge.py", line 301, in on_received_set_robot_commands
 >                    ||     raise Exception(msg)
 >                    || Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}.
 >                    ||
 > 
(hidden)
210552731Bhairav Mehtachallenge-aido_LF-template-ros - Template solution using ROSaido2-LF-sim-validationstep1-simulationfailedno8410:01:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 293, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 297, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 236, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 1 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
(hidden)
210512734Bhairav Mehtachallenge-aido_LF-template-ros - Template solution using ROSaido2-LFV-sim-testingstep1-simulationfailedno8410:01:49
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 293, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 297, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 236, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 1 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
(hidden)
210492735Bhairav Mehtachallenge-aido_LF-template-ros - Template solution using ROSaido2-LFV-sim-validationstep1-simulationfailedno8410:01:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 293, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 297, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 236, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 1 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
(hidden)
210462743RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LF-sim-validationstep1-simulationfailedno8410:01:46
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 293, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 297, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 236, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 1 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
(hidden)
210422747RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationfailedno8410:01:50
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 293, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 297, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 236, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 1 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
(hidden)
210082812Andrea Censichallenge-aido_LF-template-random - random_agentaido2-LFVI-sim-testingstep1-simulationsuccessno8410:33:49(hidden)
driven_lanedir_consec_median0.6537162971975778
survival_time_median2.5999999999999988
deviation-center-line_median0.14884708769804722
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.21659511252294616
agent_compute-ego_mean0.16550291866174568
agent_compute-ego_median0.16081095695495606
agent_compute-ego_min0.15545074462890626
deviation-center-line_max0.5141120095458062
deviation-center-line_mean0.1769000227005324
deviation-center-line_min0.06216029860587694
deviation-heading_max1.9929437250081896
deviation-heading_mean0.692977823395608
deviation-heading_median0.4928440244444201
deviation-heading_min0.24234723632033853
driven_any_max3.5343244022418343
driven_any_mean1.3470385992234175
driven_any_median1.001633421251194
driven_any_min0.3412551960104244
driven_lanedir_consec_max2.494308053441959
driven_lanedir_consec_mean0.881391085527715
driven_lanedir_consec_min0.3199180625488627
driven_lanedir_max2.494308053441959
driven_lanedir_mean0.8962528018383226
driven_lanedir_median0.7206363324756615
driven_lanedir_min0.3199180625488627
in-drivable-lane_max3.549999999999991
in-drivable-lane_mean0.839999999999998
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.3110995715177512, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.0778094907350178, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.21659511252294616, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.13957451566865173, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.04786097550693946, "sim_compute_performance-ego": 0.08820333963708032, "sim_compute_robot_state-ego": 0.09977272190625154, "sim_compute_robot_state-npc0": 0.0900632580624351, "sim_compute_robot_state-npc1": 0.08753388742857342, "sim_compute_robot_state-npc2": 0.08788973470277424, "sim_compute_robot_state-npc3": 0.0868989274471621}, "udem1-1-0": {"driven_any": 0.47798123795353353, "sim_physics": 0.21925548996244157, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.06615428413663592, "in-drivable-lane": 0, "agent_compute-ego": 0.15731869425092423, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.09949898719787598, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.03716215917042324, "sim_compute_performance-ego": 0.07571171862738472, "sim_compute_robot_state-ego": 0.07501386744635445, "sim_compute_robot_state-npc0": 0.07112950938088554, "sim_compute_robot_state-npc1": 0.06790666920798165, "sim_compute_robot_state-npc2": 0.07147009883608137, "sim_compute_robot_state-npc3": 0.06811400822230748}, "udem1-2-0": {"driven_any": 2.1864003300000694, "sim_physics": 0.20648405852827056, "survival_time": 5.14999999999999, "driven_lanedir": 1.826231945363564, "sim_render-ego": 0.06948332184726752, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.1654636790451494, "deviation-heading": 0.8115272458314897, "set_robot_commands": 0.10273722537512918, "deviation-center-line": 0.1927316041609401, "driven_lanedir_consec": 1.826231945363564, "sim_compute_sim_state": 0.040638053301468635, "sim_compute_performance-ego": 0.07501652171310869, "sim_compute_robot_state-ego": 0.08147875082145617, "sim_compute_robot_state-npc0": 0.07419858858423325, "sim_compute_robot_state-npc1": 0.0747249844004807, "sim_compute_robot_state-npc2": 0.07219008566106408, "sim_compute_robot_state-npc3": 0.07346612967333747}, "udem1-3-0": {"driven_any": 0.9410513078477724, "sim_physics": 0.2324659377336502, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.06782769163449605, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.15820014476776123, "deviation-heading": 0.35154027506548424, "set_robot_commands": 0.10356186827023824, "deviation-center-line": 0.08975118546699407, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.04373978078365326, "sim_compute_performance-ego": 0.07520692547162373, "sim_compute_robot_state-ego": 0.07465944190820058, "sim_compute_robot_state-npc0": 0.07244965930779775, "sim_compute_robot_state-npc1": 0.07261696457862854, "sim_compute_robot_state-npc2": 0.07191290458043416, "sim_compute_robot_state-npc3": 0.07182263831297557}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.21597512563069665, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.06165244465782529, "in-drivable-lane": 0, "agent_compute-ego": 0.16036029089064824, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.10386804171970912, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.040945484524681455, "sim_compute_performance-ego": 0.07237468447004046, "sim_compute_robot_state-ego": 0.075674987974621, "sim_compute_robot_state-npc0": 0.06950874555678595, "sim_compute_robot_state-npc1": 0.07365298271179199, "sim_compute_robot_state-npc2": 0.07670810109093076, "sim_compute_robot_state-npc3": 0.07403774488539923}, "udem1-5-0": {"driven_any": 3.5343244022418343, "sim_physics": 0.21730153860446225, "survival_time": 8.349999999999984, "driven_lanedir": 2.494308053441959, "sim_render-ego": 0.06528256753247655, "in-drivable-lane": 2, "agent_compute-ego": 0.1653471035871677, "deviation-heading": 1.9929437250081896, "set_robot_commands": 0.10168692023454312, "deviation-center-line": 0.5141120095458062, "driven_lanedir_consec": 2.494308053441959, "sim_compute_sim_state": 0.04010626513087107, "sim_compute_performance-ego": 0.07263006136089029, "sim_compute_robot_state-ego": 0.07874892000666635, "sim_compute_robot_state-npc0": 0.07236734264625047, "sim_compute_robot_state-npc1": 0.07249128889894771, "sim_compute_robot_state-npc2": 0.07146469276108428, "sim_compute_robot_state-npc3": 0.07225341425684398}, "udem1-6-0": {"driven_any": 2.176040550989443, "sim_physics": 0.24366234344186136, "survival_time": 5.14999999999999, "driven_lanedir": 1.9748003483573369, "sim_render-ego": 0.0685036251845869, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.16648222867724966, "deviation-heading": 0.7877172510198454, "set_robot_commands": 0.10086916951299872, "deviation-center-line": 0.30661477465861414, "driven_lanedir_consec": 1.9748003483573369, "sim_compute_sim_state": 0.04140274270066937, "sim_compute_performance-ego": 0.07722795819773258, "sim_compute_robot_state-ego": 0.08330831944363788, "sim_compute_robot_state-npc0": 0.07233350253799586, "sim_compute_robot_state-npc1": 0.07359294752472813, "sim_compute_robot_state-npc2": 0.07282766786593835, "sim_compute_robot_state-npc3": 0.07516505648788896}, "udem1-7-0": {"driven_any": 1.0050236108346422, "sim_physics": 0.2406316614151001, "survival_time": 2.499999999999999, "driven_lanedir": 0.8701796426684156, "sim_render-ego": 0.06911999225616455, "in-drivable-lane": 0, "agent_compute-ego": 0.15545074462890626, "deviation-heading": 0.9227446658975216, "set_robot_commands": 0.09788959503173827, "deviation-center-line": 0.14884708769804722, "driven_lanedir_consec": 0.8546116490381102, "sim_compute_sim_state": 0.04070052146911621, "sim_compute_performance-ego": 0.0709388256072998, "sim_compute_robot_state-ego": 0.07993154048919678, "sim_compute_robot_state-npc0": 0.07129601001739502, "sim_compute_robot_state-npc1": 0.0704301118850708, "sim_compute_robot_state-npc2": 0.07165008068084716, "sim_compute_robot_state-npc3": 0.07050049304962158}, "udem1-8-0": {"driven_any": 0.9970416603019576, "sim_physics": 0.20021087389725903, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8041152574540316, "sim_render-ego": 0.06319884153512809, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.15915326888744646, "deviation-heading": 0.6817155913229235, "set_robot_commands": 0.10123143287805413, "deviation-center-line": 0.1898768080806288, "driven_lanedir_consec": 0.8041152574540316, "sim_compute_sim_state": 0.0397259730559129, "sim_compute_performance-ego": 0.0687654110101553, "sim_compute_robot_state-ego": 0.07305092077988845, "sim_compute_robot_state-npc0": 0.07242154616575974, "sim_compute_robot_state-npc1": 0.06925833225250244, "sim_compute_robot_state-npc2": 0.07128451879207905, "sim_compute_robot_state-npc3": 0.0707200628060561}, "udem1-9-0": {"driven_any": 0.4374998710472299, "sim_physics": 0.2424120352818416, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4177527119659043, "sim_render-ego": 0.0653612521978525, "in-drivable-lane": 0, "agent_compute-ego": 0.15975031485924354, "deviation-heading": 0.3796137446285097, "set_robot_commands": 0.09652709043942966, "deviation-center-line": 0.08207813392324871, "driven_lanedir_consec": 0.4177527119659043, "sim_compute_sim_state": 0.03764046155489408, "sim_compute_performance-ego": 0.07575198320242074, "sim_compute_robot_state-ego": 0.0767978888291579, "sim_compute_robot_state-npc0": 0.06677180987138015, "sim_compute_robot_state-npc1": 0.06706510140345646, "sim_compute_robot_state-npc2": 0.06844583841470572, "sim_compute_robot_state-npc3": 0.06951099175673264}, "udem1-10-0": {"driven_any": 0.3412551960104244, "sim_physics": 0.24976598132740369, "survival_time": 1.1000000000000003, "driven_lanedir": 0.3199180625488627, "sim_render-ego": 0.06875750151547519, "in-drivable-lane": 0, "agent_compute-ego": 0.17230741544203323, "deviation-heading": 0.3796986873122644, "set_robot_commands": 0.10309022123163396, "deviation-center-line": 0.06276575300040536, "driven_lanedir_consec": 0.3199180625488627, "sim_compute_sim_state": 0.042482939633456146, "sim_compute_performance-ego": 0.07798874378204346, "sim_compute_robot_state-ego": 0.081368934024464, "sim_compute_robot_state-npc0": 0.07710846987637607, "sim_compute_robot_state-npc1": 0.08038404854861173, "sim_compute_robot_state-npc2": 0.07386239008470015, "sim_compute_robot_state-npc3": 0.07326666875319048}, "udem1-11-0": {"driven_any": 0.9951985237696338, "sim_physics": 0.1988505506515503, "survival_time": 2.499999999999999, "driven_lanedir": 0.7206363324756615, "sim_render-ego": 0.06868031978607178, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.16081095695495606, "deviation-heading": 0.30074636349085415, "set_robot_commands": 0.09527178287506104, "deviation-center-line": 0.16567670185555422, "driven_lanedir_consec": 0.7185075497455284, "sim_compute_sim_state": 0.0388484001159668, "sim_compute_performance-ego": 0.0754028606414795, "sim_compute_robot_state-ego": 0.08030835151672364, "sim_compute_robot_state-npc0": 0.06951179027557373, "sim_compute_robot_state-npc1": 0.06930240154266358, "sim_compute_robot_state-npc2": 0.06819756984710694, "sim_compute_robot_state-npc3": 0.0678904676437378}, "udem1-12-0": {"driven_any": 1.964687870253437, "sim_physics": 0.21865757221871235, "survival_time": 4.699999999999991, "driven_lanedir": 0.35162821871931205, "sim_render-ego": 0.06570876151957411, "in-drivable-lane": 3.549999999999991, "agent_compute-ego": 0.16465588579786586, "deviation-heading": 0.24234723632033853, "set_robot_commands": 0.10201190380339926, "deviation-center-line": 0.13126670634798973, "driven_lanedir_consec": 0.35162821871931205, "sim_compute_sim_state": 0.04147121246824873, "sim_compute_performance-ego": 0.07416995535505579, "sim_compute_robot_state-ego": 0.07996138106001184, "sim_compute_robot_state-npc0": 0.07466123966460532, "sim_compute_robot_state-npc1": 0.0715049479870086, "sim_compute_robot_state-npc2": 0.0721270556145526, "sim_compute_robot_state-npc3": 0.07227529870702866}, "udem1-13-0": {"driven_any": 1.001633421251194, "sim_physics": 0.2289850620123056, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6537162971975778, "sim_render-ego": 0.06565452080506545, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.16369539499282837, "deviation-heading": 0.4928440244444201, "set_robot_commands": 0.09990628407551692, "deviation-center-line": 0.13361893417935136, "driven_lanedir_consec": 0.6537162971975778, "sim_compute_sim_state": 0.041524254358731784, "sim_compute_performance-ego": 0.07198644142884475, "sim_compute_robot_state-ego": 0.08108634215134841, "sim_compute_robot_state-npc0": 0.07423266539206871, "sim_compute_robot_state-npc1": 0.07467375810329731, "sim_compute_robot_state-npc2": 0.07531945522014911, "sim_compute_robot_state-npc3": 0.0737531093450693}, "udem1-14-0": {"driven_any": 2.218769782727203, "sim_physics": 0.212126924878075, "survival_time": 5.249999999999989, "driven_lanedir": 0.8125963661433656, "sim_render-ego": 0.06410801978338332, "in-drivable-lane": 2.499999999999993, "agent_compute-ego": 0.15695254462105887, "deviation-heading": 1.7163026916317128, "set_robot_commands": 0.1002380234854562, "deviation-center-line": 0.24126933160503777, "driven_lanedir_consec": 0.6077880672015845, "sim_compute_sim_state": 0.041779795147123794, "sim_compute_performance-ego": 0.07095565341767811, "sim_compute_robot_state-ego": 0.07884639558337983, "sim_compute_robot_state-npc0": 0.07332822708856492, "sim_compute_robot_state-npc1": 0.07561419350760323, "sim_compute_robot_state-npc2": 0.07550410770234607, "sim_compute_robot_state-npc3": 0.07315109797886439}}
set_robot_commands_max0.13957451566865173
set_robot_commands_mean0.10319753745329573
set_robot_commands_median0.10123143287805413
set_robot_commands_min0.09527178287506104
sim_compute_performance-ego_max0.08820333963708032
sim_compute_performance-ego_mean0.07482207226152254
sim_compute_performance-ego_median0.07501652171310869
sim_compute_performance-ego_min0.0687654110101553
sim_compute_robot_state-ego_max0.09977272190625154
sim_compute_robot_state-ego_mean0.08000058426275726
sim_compute_robot_state-ego_median0.07993154048919678
sim_compute_robot_state-ego_min0.07305092077988845
sim_compute_robot_state-npc0_max0.0900632580624351
sim_compute_robot_state-npc0_mean0.07342549096187385
sim_compute_robot_state-npc0_median0.07242154616575974
sim_compute_robot_state-npc0_min0.06677180987138015
sim_compute_robot_state-npc1_max0.08753388742857342
sim_compute_robot_state-npc1_mean0.07338350799875641
sim_compute_robot_state-npc1_median0.07261696457862854
sim_compute_robot_state-npc1_min0.06706510140345646
sim_compute_robot_state-npc2_max0.08788973470277424
sim_compute_robot_state-npc2_mean0.07339028679031961
sim_compute_robot_state-npc2_median0.0721270556145526
sim_compute_robot_state-npc2_min0.06819756984710694
sim_compute_robot_state-npc3_max0.0868989274471621
sim_compute_robot_state-npc3_mean0.07285507395508103
sim_compute_robot_state-npc3_median0.07227529870702866
sim_compute_robot_state-npc3_min0.0678904676437378
sim_compute_sim_state_max0.04786097550693946
sim_compute_sim_state_mean0.04106860126147713
sim_compute_sim_state_median0.040945484524681455
sim_compute_sim_state_min0.03716215917042324
sim_physics_max0.3110995715177512
sim_physics_mean0.22919231514009217
sim_physics_median0.21925548996244157
sim_physics_min0.1988505506515503
sim_render-ego_max0.0778094907350178
sim_render-ego_mean0.06715350900846806
sim_render-ego_median0.06615428413663592
sim_render-ego_min0.06165244465782529
simulation-passed1
survival_time_max8.349999999999984
survival_time_mean3.3333333333333286
survival_time_min1.0500000000000005
210022791RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LF-sim-testingstep1-simulationhost-errorno8410:00:07
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-testing/submission2791/step1-simulation-ip-172-31-38-104-5317-job21002/logs/challenges-runner/stdout.log'
(hidden)
209902822Liam Paullchallenge-aido_LF-template-randomaido2-LFV-sim-testingstep1-simulationsuccessno8410:23:10(hidden)
driven_lanedir_consec_median0.47061816984274774
survival_time_median2.4499999999999993
deviation-center-line_median0.09047885846140293
in-drivable-lane_median0.7999999999999974


other stats
agent_compute-ego_max0.172661166442068
agent_compute-ego_mean0.16490555689795522
agent_compute-ego_median0.16571319544756857
agent_compute-ego_min0.1485067714344371
deviation-center-line_max0.1743913456502906
deviation-center-line_mean0.11122054568266285
deviation-center-line_min0.05649185041622204
deviation-heading_max0.8583234710357402
deviation-heading_mean0.371509805944231
deviation-heading_median0.3477435217829391
deviation-heading_min0.1892537419842091
driven_any_max1.6169932854589035
driven_any_mean0.8549136794030617
driven_any_median0.9441796518284096
driven_any_min0.28668385087664994
driven_lanedir_consec_max1.2433038249708377
driven_lanedir_consec_mean0.5622432991481365
driven_lanedir_consec_min0.24216121529728207
driven_lanedir_max1.2433038249708377
driven_lanedir_mean0.5622432991481365
driven_lanedir_median0.47061816984274774
driven_lanedir_min0.24216121529728207
in-drivable-lane_max1.3499999999999996
in-drivable-lane_mean0.5899999999999992
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.1817838262628626, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.06418722205691868, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.16515219653094257, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.1091664985374168, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.04303456677330865, "sim_compute_performance-ego": 0.07700781027475993, "sim_compute_robot_state-ego": 0.08318311638302273, "sim_compute_robot_state-npc0": 0.07562520768907335, "sim_compute_robot_state-npc1": 0.07586420465398717, "sim_compute_robot_state-npc2": 0.07991086553644251, "sim_compute_robot_state-npc3": 0.07472395455395733}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4119037580118998, "sim_physics": 0.19939363479614255, "survival_time": 1.2500000000000004, "driven_lanedir": 0.3792793194470208, "sim_render-ego": 0.06487111091613769, "in-drivable-lane": 0, "agent_compute-ego": 0.1642424011230469, "deviation-heading": 0.47860435001913393, "set_robot_commands": 0.10877954483032226, "deviation-center-line": 0.09344288799285194, "driven_lanedir_consec": 0.3792793194470208, "sim_compute_sim_state": 0.042962236404418944, "sim_compute_performance-ego": 0.0734758186340332, "sim_compute_robot_state-ego": 0.07180252075195312, "sim_compute_robot_state-npc0": 0.07130143165588379, "sim_compute_robot_state-npc1": 0.07627883911132813, "sim_compute_robot_state-npc2": 0.0763346290588379, "sim_compute_robot_state-npc3": 0.07206114768981933}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.579862082650716, "sim_physics": 0.22168454998417905, "survival_time": 3.7999999999999945, "driven_lanedir": 1.2433038249708377, "sim_render-ego": 0.06863986504705329, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.16732404106541685, "deviation-heading": 0.5020407942599273, "set_robot_commands": 0.10857357163178294, "deviation-center-line": 0.1743913456502906, "driven_lanedir_consec": 1.2433038249708377, "sim_compute_sim_state": 0.04228681325912475, "sim_compute_performance-ego": 0.07649665443520796, "sim_compute_robot_state-ego": 0.08479608046381097, "sim_compute_robot_state-npc0": 0.07165085328252692, "sim_compute_robot_state-npc1": 0.0724489500648097, "sim_compute_robot_state-npc2": 0.07266783714294434, "sim_compute_robot_state-npc3": 0.07445811284215827}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6169932854589035, "sim_physics": 0.19635938994492155, "survival_time": 3.949999999999994, "driven_lanedir": 1.2126117310470583, "sim_render-ego": 0.06669531291044212, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.1650288255908821, "deviation-heading": 0.34764983431811214, "set_robot_commands": 0.1031789719303952, "deviation-center-line": 0.11519544839797546, "driven_lanedir_consec": 1.2126117310470583, "sim_compute_sim_state": 0.039977429788323894, "sim_compute_performance-ego": 0.07110165342499938, "sim_compute_robot_state-ego": 0.07638011401212669, "sim_compute_robot_state-npc0": 0.07214870935753931, "sim_compute_robot_state-npc1": 0.07020232043688811, "sim_compute_robot_state-npc2": 0.07229848753047895, "sim_compute_robot_state-npc3": 0.07297960715957835}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5031772882846431, "sim_physics": 0.18905065740857804, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4926325270527603, "sim_render-ego": 0.06988472597939628, "in-drivable-lane": 0, "agent_compute-ego": 0.1659436821937561, "deviation-heading": 0.29868161596444953, "set_robot_commands": 0.10470222575323922, "deviation-center-line": 0.09047885846140293, "driven_lanedir_consec": 0.4926325270527603, "sim_compute_sim_state": 0.043318595205034525, "sim_compute_performance-ego": 0.07767938716070992, "sim_compute_robot_state-ego": 0.08461343390601021, "sim_compute_robot_state-npc0": 0.07573804685047694, "sim_compute_robot_state-npc1": 0.07135183470589775, "sim_compute_robot_state-npc2": 0.07547509670257568, "sim_compute_robot_state-npc3": 0.07517327581133161}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.0440585337030532, "sim_physics": 0.17979915936787924, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5009536316076031, "sim_render-ego": 0.06396753699691207, "in-drivable-lane": 1.0000000000000009, "agent_compute-ego": 0.1614889480449535, "deviation-heading": 0.8583234710357402, "set_robot_commands": 0.10820781743084942, "deviation-center-line": 0.1636702708892767, "driven_lanedir_consec": 0.5009536316076031, "sim_compute_sim_state": 0.04185135276229293, "sim_compute_performance-ego": 0.07181874910990398, "sim_compute_robot_state-ego": 0.07854513768796567, "sim_compute_robot_state-npc0": 0.0722846543347394, "sim_compute_robot_state-npc1": 0.07352705796559651, "sim_compute_robot_state-npc2": 0.07134625205287227, "sim_compute_robot_state-npc3": 0.07301865683661567}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.045672984989961, "sim_physics": 0.1760011778937446, "survival_time": 2.6999999999999984, "driven_lanedir": 0.636422849544542, "sim_render-ego": 0.06793227902165165, "in-drivable-lane": 0.7999999999999974, "agent_compute-ego": 0.16571319544756857, "deviation-heading": 0.39273667682101354, "set_robot_commands": 0.10599517380749736, "deviation-center-line": 0.08970073753261001, "driven_lanedir_consec": 0.636422849544542, "sim_compute_sim_state": 0.04048483901553684, "sim_compute_performance-ego": 0.07634294916082311, "sim_compute_robot_state-ego": 0.07994974542547155, "sim_compute_robot_state-npc0": 0.07325735798588505, "sim_compute_robot_state-npc1": 0.07527717396065041, "sim_compute_robot_state-npc2": 0.07193678838235361, "sim_compute_robot_state-npc3": 0.07304241039134839}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.877256228857091, "sim_physics": 0.2134642283121745, "survival_time": 2.25, "driven_lanedir": 0.38713958773684665, "sim_render-ego": 0.06784293386671278, "in-drivable-lane": 0.9999999999999996, "agent_compute-ego": 0.16714492373996312, "deviation-heading": 0.384474040685185, "set_robot_commands": 0.1070089022318522, "deviation-center-line": 0.07500609722918031, "driven_lanedir_consec": 0.38713958773684665, "sim_compute_sim_state": 0.043047995037502715, "sim_compute_performance-ego": 0.07346058421664768, "sim_compute_robot_state-ego": 0.07972117000155979, "sim_compute_robot_state-npc0": 0.07584906684027778, "sim_compute_robot_state-npc1": 0.07311399777730306, "sim_compute_robot_state-npc2": 0.07340392536587186, "sim_compute_robot_state-npc3": 0.07215595775180393}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.8433010974171573, "sim_physics": 0.17214222058005954, "survival_time": 2.3, "driven_lanedir": 0.24216121529728207, "sim_render-ego": 0.0655155492865521, "in-drivable-lane": 1.3499999999999996, "agent_compute-ego": 0.1637867326321809, "deviation-heading": 0.34566993689777425, "set_robot_commands": 0.10420170555944028, "deviation-center-line": 0.07136114712800473, "driven_lanedir_consec": 0.24216121529728207, "sim_compute_sim_state": 0.03942896490511687, "sim_compute_performance-ego": 0.07251722398011581, "sim_compute_robot_state-ego": 0.0788884629373965, "sim_compute_robot_state-npc0": 0.0725989704546721, "sim_compute_robot_state-npc1": 0.06996172407399052, "sim_compute_robot_state-npc2": 0.07127598575923753, "sim_compute_robot_state-npc3": 0.07335989371589992}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.9785013113851152, "sim_physics": 0.23030948162078857, "survival_time": 2.499999999999999, "driven_lanedir": 0.9700080716683566, "sim_render-ego": 0.06677783966064453, "in-drivable-lane": 0, "agent_compute-ego": 0.1681440544128418, "deviation-heading": 0.3477435217829391, "set_robot_commands": 0.11104859352111816, "deviation-center-line": 0.15723242276422936, "driven_lanedir_consec": 0.9700080716683566, "sim_compute_sim_state": 0.045733098983764646, "sim_compute_performance-ego": 0.07617543697357178, "sim_compute_robot_state-ego": 0.08661375522613525, "sim_compute_robot_state-npc0": 0.0770398473739624, "sim_compute_robot_state-npc1": 0.07393918514251709, "sim_compute_robot_state-npc2": 0.07449909210205079, "sim_compute_robot_state-npc3": 0.07894308090209962}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.3129292149411446, "sim_physics": 0.20924774408340455, "survival_time": 1.0000000000000002, "driven_lanedir": 0.2864579763812336, "sim_render-ego": 0.0643017053604126, "in-drivable-lane": 0, "agent_compute-ego": 0.16819534301757813, "deviation-heading": 0.38016104491567027, "set_robot_commands": 0.0997551441192627, "deviation-center-line": 0.05649185041622204, "driven_lanedir_consec": 0.2864579763812336, "sim_compute_sim_state": 0.04020578861236572, "sim_compute_performance-ego": 0.07426296472549439, "sim_compute_robot_state-ego": 0.08285254240036011, "sim_compute_robot_state-npc0": 0.07233911752700806, "sim_compute_robot_state-npc1": 0.0719254493713379, "sim_compute_robot_state-npc2": 0.06743454933166504, "sim_compute_robot_state-npc3": 0.07315328121185302}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.28668385087664994, "sim_physics": 0.18737440360219856, "survival_time": 0.9500000000000004, "driven_lanedir": 0.278237107094367, "sim_render-ego": 0.06485458424216822, "in-drivable-lane": 0, "agent_compute-ego": 0.172661166442068, "deviation-heading": 0.19732996798555608, "set_robot_commands": 0.10478819044012772, "deviation-center-line": 0.08671144489080887, "driven_lanedir_consec": 0.278237107094367, "sim_compute_sim_state": 0.0405788170663934, "sim_compute_performance-ego": 0.07003588425485711, "sim_compute_robot_state-ego": 0.10179424285888672, "sim_compute_robot_state-npc0": 0.0705812604803788, "sim_compute_robot_state-npc1": 0.07571547909786827, "sim_compute_robot_state-npc2": 0.07049235544706646, "sim_compute_robot_state-npc3": 0.06949569049634431}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0034139953700494, "sim_physics": 0.16565815925598146, "survival_time": 2.499999999999999, "driven_lanedir": 0.47061816984274774, "sim_render-ego": 0.06420241832733155, "in-drivable-lane": 1.1499999999999986, "agent_compute-ego": 0.16140289783477782, "deviation-heading": 0.2263485117662671, "set_robot_commands": 0.10838538646697998, "deviation-center-line": 0.1594122934983136, "driven_lanedir_consec": 0.47061816984274774, "sim_compute_sim_state": 0.03723715782165527, "sim_compute_performance-ego": 0.07060580730438232, "sim_compute_robot_state-ego": 0.07263741970062255, "sim_compute_robot_state-npc0": 0.07178138732910157, "sim_compute_robot_state-npc1": 0.07202898979187011, "sim_compute_robot_state-npc2": 0.07202531337738037, "sim_compute_robot_state-npc3": 0.07143052101135254}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.351442944128162, "sim_physics": 0.22006806460293857, "survival_time": 1.1000000000000003, "driven_lanedir": 0.34114611245347204, "sim_render-ego": 0.06167436729777943, "in-drivable-lane": 0, "agent_compute-ego": 0.1485067714344371, "deviation-heading": 0.1892537419842091, "set_robot_commands": 0.09825794263319536, "deviation-center-line": 0.08918370733714856, "driven_lanedir_consec": 0.34114611245347204, "sim_compute_sim_state": 0.03824018348347057, "sim_compute_performance-ego": 0.07027196884155273, "sim_compute_robot_state-ego": 0.07259435003454034, "sim_compute_robot_state-npc0": 0.06974547559564764, "sim_compute_robot_state-npc1": 0.06894649158824574, "sim_compute_robot_state-npc2": 0.06685627590526234, "sim_compute_robot_state-npc3": 0.07394218444824219}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.9441796518284096, "sim_physics": 0.20800308791958555, "survival_time": 2.4499999999999993, "driven_lanedir": 0.4203463405922858, "sim_render-ego": 0.06571698188781738, "in-drivable-lane": 1.0999999999999988, "agent_compute-ego": 0.16884817395891463, "deviation-heading": 0.4296166116434236, "set_robot_commands": 0.1098639624459403, "deviation-center-line": 0.08695038592754878, "driven_lanedir_consec": 0.4203463405922858, "sim_compute_sim_state": 0.04350402890419473, "sim_compute_performance-ego": 0.07239760184774593, "sim_compute_robot_state-ego": 0.07784267347686145, "sim_compute_robot_state-npc0": 0.0733185592962771, "sim_compute_robot_state-npc1": 0.06970712603354941, "sim_compute_robot_state-npc2": 0.07140465171969666, "sim_compute_robot_state-npc3": 0.07368280449692084}}
set_robot_commands_max0.11104859352111816
set_robot_commands_mean0.106127575422628
set_robot_commands_median0.1070089022318522
set_robot_commands_min0.09825794263319536
sim_compute_performance-ego_max0.07767938716070992
sim_compute_performance-ego_mean0.07357669962298702
sim_compute_performance-ego_median0.07346058421664768
sim_compute_performance-ego_min0.07003588425485711
sim_compute_robot_state-ego_max0.10179424285888672
sim_compute_robot_state-ego_mean0.08081431768444823
sim_compute_robot_state-ego_median0.07972117000155979
sim_compute_robot_state-ego_min0.07180252075195312
sim_compute_robot_state-npc0_max0.0770398473739624
sim_compute_robot_state-npc0_mean0.07301732973689669
sim_compute_robot_state-npc0_median0.07233911752700806
sim_compute_robot_state-npc0_min0.06974547559564764
sim_compute_robot_state-npc1_max0.07627883911132813
sim_compute_robot_state-npc1_mean0.07268592158505598
sim_compute_robot_state-npc1_median0.0724489500648097
sim_compute_robot_state-npc1_min0.06894649158824574
sim_compute_robot_state-npc2_max0.07991086553644251
sim_compute_robot_state-npc2_mean0.07249080702764908
sim_compute_robot_state-npc2_median0.07202531337738037
sim_compute_robot_state-npc2_min0.06685627590526234
sim_compute_robot_state-npc3_max0.07894308090209962
sim_compute_robot_state-npc3_mean0.07344137195462169
sim_compute_robot_state-npc3_median0.07315328121185302
sim_compute_robot_state-npc3_min0.06949569049634431
sim_compute_sim_state_max0.045733098983764646
sim_compute_sim_state_mean0.04145945786816697
sim_compute_sim_state_median0.04185135276229293
sim_compute_sim_state_min0.03723715782165527
sim_physics_max0.23030948162078857
sim_physics_mean0.1966893190423626
sim_physics_median0.19635938994492155
sim_physics_min0.16565815925598146
sim_render-ego_max0.06988472597939628
sim_render-ego_mean0.06580429552386204
sim_render-ego_median0.0655155492865521
sim_render-ego_min0.06167436729777943
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.2366666666666655
survival_time_min0.9500000000000004
209632899Julian ZillyBaseline-IL-logs-tensorflowaido2-LF-sim-testingstep1-simulationsuccessno8410:34:31(hidden)
driven_lanedir_consec_median0.6549412874713222
survival_time_median5.999999999999987
deviation-center-line_median0.25006157232718784
in-drivable-lane_median0.8500000000000121


other stats
agent_compute-ego_max0.2647633904316386
agent_compute-ego_mean0.24105427312095337
agent_compute-ego_median0.2455602258932395
agent_compute-ego_min0.19699199035249904
deviation-center-line_max0.6241759141191141
deviation-center-line_mean0.3068684605089798
deviation-center-line_min0.1894224465968746
deviation-heading_max1.8316596564812408
deviation-heading_mean0.9459892114251056
deviation-heading_median0.894571035890676
deviation-heading_min0.4466803434437456
driven_any_max2.1603932640628676
driven_any_mean0.9275729246507748
driven_any_median0.989883133670337
driven_any_min0.26487104677468254
driven_lanedir_consec_max1.971700196032653
driven_lanedir_consec_mean0.7345683073809941
driven_lanedir_consec_min0.25716501458078334
driven_lanedir_max1.971700196032653
driven_lanedir_mean0.7345683073809941
driven_lanedir_median0.6549412874713222
driven_lanedir_min0.25716501458078334
in-drivable-lane_max2.549999999999991
in-drivable-lane_mean0.9133333333333342
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9986182841320406, "sim_physics": 0.08366315620989839, "survival_time": 6.0499999999999865, "driven_lanedir": 0.6700722150858742, "sim_render-ego": 0.05318575851188218, "in-drivable-lane": 1.649999999999994, "agent_compute-ego": 0.21492595120895008, "deviation-heading": 0.9583668413767816, "set_robot_commands": 0.09076221127155398, "deviation-center-line": 0.2291813678313231, "driven_lanedir_consec": 0.6700722150858742, "sim_compute_sim_state": 0.03280589009119459, "sim_compute_performance-ego": 0.05970906029062823, "sim_compute_robot_state-ego": 0.06315867565880137}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.0073534333983163, "sim_physics": 0.11908511060183166, "survival_time": 6.099999999999986, "driven_lanedir": 0.866347420056429, "sim_render-ego": 0.06254772670933458, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.2455602258932395, "deviation-heading": 1.7107711774750836, "set_robot_commands": 0.09721514240640108, "deviation-center-line": 0.486906642292682, "driven_lanedir_consec": 0.866347420056429, "sim_compute_sim_state": 0.04011696283934546, "sim_compute_performance-ego": 0.0705736406513902, "sim_compute_robot_state-ego": 0.07536427896530902}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.26487104677468254, "sim_physics": 0.12330744717572188, "survival_time": 1.850000000000001, "driven_lanedir": 0.25716501458078334, "sim_render-ego": 0.06316297763102763, "in-drivable-lane": 0, "agent_compute-ego": 0.25503398921038656, "deviation-heading": 0.4466803434437456, "set_robot_commands": 0.10775615717913652, "deviation-center-line": 0.1894224465968746, "driven_lanedir_consec": 0.25716501458078334, "sim_compute_sim_state": 0.03912337406261547, "sim_compute_performance-ego": 0.07288327088227144, "sim_compute_robot_state-ego": 0.07897280357979439}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4483038440777001, "sim_physics": 0.09980157326007712, "survival_time": 2.8999999999999977, "driven_lanedir": 0.4266216502858877, "sim_render-ego": 0.05407354749482254, "in-drivable-lane": 0, "agent_compute-ego": 0.19699199035249904, "deviation-heading": 0.9042995745888692, "set_robot_commands": 0.08330475050827553, "deviation-center-line": 0.20006547654235235, "driven_lanedir_consec": 0.4266216502858877, "sim_compute_sim_state": 0.03391145426651527, "sim_compute_performance-ego": 0.0592402301985642, "sim_compute_robot_state-ego": 0.06398651928737245}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.5751381949161964, "sim_physics": 0.11595260395723232, "survival_time": 9.349999999999998, "driven_lanedir": 1.1899106749840844, "sim_render-ego": 0.061281159599834586, "in-drivable-lane": 2.1500000000000155, "agent_compute-ego": 0.23066232421181415, "deviation-heading": 0.5495145238224983, "set_robot_commands": 0.09268837306588729, "deviation-center-line": 0.6241759141191141, "driven_lanedir_consec": 1.1899106749840844, "sim_compute_sim_state": 0.037654237951186886, "sim_compute_performance-ego": 0.06793063082159521, "sim_compute_robot_state-ego": 0.07587160401165804}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.44830384404055934, "sim_physics": 0.1229705522800314, "survival_time": 2.8999999999999977, "driven_lanedir": 0.4227991506608033, "sim_render-ego": 0.06271170336624672, "in-drivable-lane": 0, "agent_compute-ego": 0.23265204347413165, "deviation-heading": 0.9814033650024896, "set_robot_commands": 0.09230923652648926, "deviation-center-line": 0.20796969664933015, "driven_lanedir_consec": 0.4227991506608033, "sim_compute_sim_state": 0.03863971397794526, "sim_compute_performance-ego": 0.07264528603389345, "sim_compute_robot_state-ego": 0.07489779488793735}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0160885841317506, "sim_physics": 0.11609025311663868, "survival_time": 6.149999999999986, "driven_lanedir": 0.5516827787704441, "sim_render-ego": 0.06233537100194916, "in-drivable-lane": 2.549999999999991, "agent_compute-ego": 0.2624956030186599, "deviation-heading": 0.894571035890676, "set_robot_commands": 0.10043726703984952, "deviation-center-line": 0.25006157232718784, "driven_lanedir_consec": 0.5516827787704441, "sim_compute_sim_state": 0.0397141367439332, "sim_compute_performance-ego": 0.07119378035630637, "sim_compute_robot_state-ego": 0.08014695431158794}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.29107361379694296, "sim_physics": 0.12205849885940552, "survival_time": 2.000000000000001, "driven_lanedir": 0.2837569298931477, "sim_render-ego": 0.0603921115398407, "in-drivable-lane": 0, "agent_compute-ego": 0.2428607642650604, "deviation-heading": 0.4484407094191357, "set_robot_commands": 0.09839978218078611, "deviation-center-line": 0.1986478962896645, "driven_lanedir_consec": 0.2837569298931477, "sim_compute_sim_state": 0.03794429302215576, "sim_compute_performance-ego": 0.06778744459152222, "sim_compute_robot_state-ego": 0.07347679734230042}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.6754177240838395, "sim_physics": 0.11820182346162342, "survival_time": 4.199999999999993, "driven_lanedir": 0.6683847145695121, "sim_render-ego": 0.06348723456973121, "in-drivable-lane": 0, "agent_compute-ego": 0.2322179334504264, "deviation-heading": 0.6032142883336166, "set_robot_commands": 0.1007036424818493, "deviation-center-line": 0.29562265332753457, "driven_lanedir_consec": 0.6683847145695121, "sim_compute_sim_state": 0.03825338397707258, "sim_compute_performance-ego": 0.07086105857576643, "sim_compute_robot_state-ego": 0.07797286907831828}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.98988313243351, "sim_physics": 0.1283694307009379, "survival_time": 5.999999999999987, "driven_lanedir": 0.6549412874713222, "sim_render-ego": 0.06354642113049826, "in-drivable-lane": 1.499999999999995, "agent_compute-ego": 0.24977941115697225, "deviation-heading": 1.1877020241228389, "set_robot_commands": 0.10039054950078329, "deviation-center-line": 0.2693854763256457, "driven_lanedir_consec": 0.6549412874713222, "sim_compute_sim_state": 0.03947548866271973, "sim_compute_performance-ego": 0.0724226971467336, "sim_compute_robot_state-ego": 0.07861407200495402}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5838733491363972, "sim_physics": 0.12601592058831074, "survival_time": 9.4, "driven_lanedir": 1.212417394198632, "sim_render-ego": 0.06371810588430851, "in-drivable-lane": 2.000000000000017, "agent_compute-ego": 0.2501228020546284, "deviation-heading": 0.8899374696565366, "set_robot_commands": 0.0972701907157898, "deviation-center-line": 0.3290677259965499, "driven_lanedir_consec": 1.212417394198632, "sim_compute_sim_state": 0.03881003374749042, "sim_compute_performance-ego": 0.06993260535788029, "sim_compute_robot_state-ego": 0.07993323118128676}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.00735343359672, "sim_physics": 0.12229656586881545, "survival_time": 6.099999999999986, "driven_lanedir": 0.6366100464147634, "sim_render-ego": 0.06669930356447815, "in-drivable-lane": 1.4499999999999948, "agent_compute-ego": 0.2647633904316386, "deviation-heading": 1.8316596564812408, "set_robot_commands": 0.1046805811710045, "deviation-center-line": 0.3820159901322753, "driven_lanedir_consec": 0.6366100464147634, "sim_compute_sim_state": 0.040818132337976674, "sim_compute_performance-ego": 0.07525582587132688, "sim_compute_robot_state-ego": 0.08128089592105052}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.989883133670337, "sim_physics": 0.11973767479260762, "survival_time": 5.999999999999987, "driven_lanedir": 0.7597707676718418, "sim_render-ego": 0.06501034299532572, "in-drivable-lane": 1.149999999999996, "agent_compute-ego": 0.25143518249193825, "deviation-heading": 0.8028925560267159, "set_robot_commands": 0.10381012161572774, "deviation-center-line": 0.22889793589720195, "driven_lanedir_consec": 0.7597707676718418, "sim_compute_sim_state": 0.04015403985977173, "sim_compute_performance-ego": 0.0718090295791626, "sim_compute_robot_state-ego": 0.08312347531318665}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.45703898750976224, "sim_physics": 0.11044188273155084, "survival_time": 2.9499999999999975, "driven_lanedir": 0.4463443700387306, "sim_render-ego": 0.06178282075009103, "in-drivable-lane": 0, "agent_compute-ego": 0.23128144619828564, "deviation-heading": 0.6366855905952253, "set_robot_commands": 0.0982440891912428, "deviation-center-line": 0.2083565966184449, "driven_lanedir_consec": 0.4463443700387306, "sim_compute_sim_state": 0.03687294863038144, "sim_compute_performance-ego": 0.07073812565561068, "sim_compute_robot_state-ego": 0.07382876994246143}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1603932640628676, "sim_physics": 0.10883107429414283, "survival_time": 12.700000000000044, "driven_lanedir": 1.971700196032653, "sim_render-ego": 0.06395046917472298, "in-drivable-lane": 0.8500000000000121, "agent_compute-ego": 0.25503103939567023, "deviation-heading": 1.343699015141128, "set_robot_commands": 0.10008831005396804, "deviation-center-line": 0.5032495166885163, "driven_lanedir_consec": 1.971700196032653, "sim_compute_sim_state": 0.03936129104434036, "sim_compute_performance-ego": 0.07294637293327512, "sim_compute_robot_state-ego": 0.0787047692171232}}
set_robot_commands_max0.10775615717913652
set_robot_commands_mean0.09787069366058296
set_robot_commands_median0.09839978218078611
set_robot_commands_min0.08330475050827553
sim_compute_performance-ego_max0.07525582587132688
sim_compute_performance-ego_mean0.06972860392972845
sim_compute_performance-ego_median0.07086105857576643
sim_compute_performance-ego_min0.0592402301985642
sim_compute_robot_state-ego_max0.08312347531318665
sim_compute_robot_state-ego_mean0.07595556738020945
sim_compute_robot_state-ego_median0.07797286907831828
sim_compute_robot_state-ego_min0.06315867565880137
sim_compute_sim_state_max0.040818132337976674
sim_compute_sim_state_mean0.038243692080976326
sim_compute_sim_state_median0.03881003374749042
sim_compute_sim_state_min0.03280589009119459
sim_physics_max0.1283694307009379
sim_physics_mean0.11578823785992168
sim_physics_median0.11908511060183166
sim_physics_min0.08366315620989839
sim_render-ego_max0.06669930356447815
sim_render-ego_mean0.0618590035949396
sim_render-ego_median0.06271170336624672
sim_render-ego_min0.05318575851188218
simulation-passed1
survival_time_max12.700000000000044
survival_time_mean5.643333333333331
survival_time_min1.850000000000001
209263058Andrea Censirotationaido2-LFV-sim-testingstep1-simulationhost-errorno8410:56:29
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission3058/step1-simulation-ip-172-31-38-104-5317-job20926/logs/challenges-runner/stdout.log'
(hidden)
209192941Bhairav MehtaBaseline solution using reinforcement learningaido2-LFVI-sim-validationstep1-simulationerrorno8410:00:32
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
(hidden)
208583074Julian ZillyBaseline-IL-logs-tensorflowaido2-LFV-sim-testingstep1-simulationhost-errorno8410:59:44
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission3074/step1-simulation-ip-172-31-38-104-5317-job20858/logs/challenges-runner/stdout.log'
(hidden)
208343070Julian ZillyBaseline-IL-logs-tensorflowaido2-LF-sim-testingstep1-simulationhost-errorno8410:23:25
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-testing/submission3070/step1-simulation-ip-172-31-38-104-5317-job20834/logs/challenges-runner/stdout.log'
(hidden)
208093101awoo424HKU Duckietown Projectaido2-LFV-sim-validationstep1-simulationsuccessno8410:36:01(hidden)
driven_lanedir_consec_median0.439925096833917
survival_time_median13.250000000000052
deviation-center-line_median0.6239853399057752
in-drivable-lane_median4.9000000000000465


other stats
agent_compute-ego_max0.2342921382976028
agent_compute-ego_mean0.22661939502614417
agent_compute-ego_median0.2277700996398926
agent_compute-ego_min0.21907610647668144
deviation-center-line_max0.7435127051255117
deviation-center-line_mean0.595602707926662
deviation-center-line_min0.4065453237131485
deviation-heading_max3.231178580537095
deviation-heading_mean2.453660223543085
deviation-heading_median2.405225835484903
deviation-heading_min1.165103464359677
driven_any_max0.9487107508352632
driven_any_mean0.855995194707601
driven_any_median0.8333307977381974
driven_any_min0.7372626499997462
driven_lanedir_consec_max0.8275161517633438
driven_lanedir_consec_mean0.5709451062193592
driven_lanedir_consec_min0.38940065476844854
driven_lanedir_max0.8275161517633438
driven_lanedir_mean0.5709451062193592
driven_lanedir_median0.439925096833917
driven_lanedir_min0.38940065476844854
in-drivable-lane_max6.600000000000069
in-drivable-lane_mean3.990000000000046
in-drivable-lane_min1.2500000000000178
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.8333307977381974, "sim_physics": 0.189065043431408, "survival_time": 13.250000000000052, "driven_lanedir": 0.3992873378221544, "sim_render-ego": 0.06551824245812758, "in-drivable-lane": 6.600000000000069, "agent_compute-ego": 0.2342921382976028, "deviation-heading": 1.165103464359677, "set_robot_commands": 0.1042845959933299, "deviation-center-line": 0.7076613834832143, "driven_lanedir_consec": 0.3992873378221544, "sim_compute_sim_state": 0.04017598763951716, "sim_compute_performance-ego": 0.07253786752808769, "sim_compute_robot_state-ego": 0.07914047151241663, "sim_compute_robot_state-npc0": 0.07179236322079065, "sim_compute_robot_state-npc1": 0.072026911321676, "sim_compute_robot_state-npc2": 0.07137072401226692, "sim_compute_robot_state-npc3": 0.07298133058368035}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9464030616250384, "sim_physics": 0.1873590024312337, "survival_time": 14.950000000000076, "driven_lanedir": 0.8275161517633438, "sim_render-ego": 0.06409026861190796, "in-drivable-lane": 1.2500000000000178, "agent_compute-ego": 0.2277700996398926, "deviation-heading": 3.231178580537095, "set_robot_commands": 0.10107333660125732, "deviation-center-line": 0.6239853399057752, "driven_lanedir_consec": 0.8275161517633438, "sim_compute_sim_state": 0.04072420597076416, "sim_compute_performance-ego": 0.06961399714152018, "sim_compute_robot_state-ego": 0.07678272565205892, "sim_compute_robot_state-npc0": 0.07237754821777344, "sim_compute_robot_state-npc1": 0.07232784986495971, "sim_compute_robot_state-npc2": 0.07189279397328695, "sim_compute_robot_state-npc3": 0.07230599244435629}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.8142687133397596, "sim_physics": 0.182348615769651, "survival_time": 13.150000000000052, "driven_lanedir": 0.439925096833917, "sim_render-ego": 0.06401432693684508, "in-drivable-lane": 5.500000000000071, "agent_compute-ego": 0.22327109797372563, "deviation-heading": 2.405225835484903, "set_robot_commands": 0.10001238036065048, "deviation-center-line": 0.496308787405661, "driven_lanedir_consec": 0.439925096833917, "sim_compute_sim_state": 0.04009717226934977, "sim_compute_performance-ego": 0.07003783091846075, "sim_compute_robot_state-ego": 0.07639892744927351, "sim_compute_robot_state-npc0": 0.07287078665236557, "sim_compute_robot_state-npc1": 0.0715619584000156, "sim_compute_robot_state-npc2": 0.07122658504732661, "sim_compute_robot_state-npc3": 0.07193901239692485}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.7372626499997462, "sim_physics": 0.16914276094395717, "survival_time": 11.65000000000003, "driven_lanedir": 0.38940065476844854, "sim_render-ego": 0.05982533340290381, "in-drivable-lane": 4.9000000000000465, "agent_compute-ego": 0.21907610647668144, "deviation-heading": 2.3072659701668887, "set_robot_commands": 0.09573908220544904, "deviation-center-line": 0.4065453237131485, "driven_lanedir_consec": 0.38940065476844854, "sim_compute_sim_state": 0.036069920134646695, "sim_compute_performance-ego": 0.06608295747650539, "sim_compute_robot_state-ego": 0.07274575499505956, "sim_compute_robot_state-npc0": 0.06745799621287334, "sim_compute_robot_state-npc1": 0.0663107689869762, "sim_compute_robot_state-npc2": 0.0662936304771849, "sim_compute_robot_state-npc3": 0.06562667650214592}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.9487107508352632, "sim_physics": 0.17814332882563272, "survival_time": 14.950000000000076, "driven_lanedir": 0.7985962899089323, "sim_render-ego": 0.06444637854894002, "in-drivable-lane": 1.7000000000000242, "agent_compute-ego": 0.2286875327428182, "deviation-heading": 3.1595272671668613, "set_robot_commands": 0.0971648621559143, "deviation-center-line": 0.7435127051255117, "driven_lanedir_consec": 0.7985962899089323, "sim_compute_sim_state": 0.0403385329246521, "sim_compute_performance-ego": 0.07097038984298706, "sim_compute_robot_state-ego": 0.07930594364802043, "sim_compute_robot_state-npc0": 0.06973622878392538, "sim_compute_robot_state-npc1": 0.07054235935211181, "sim_compute_robot_state-npc2": 0.06982650836308797, "sim_compute_robot_state-npc3": 0.07079195578893026}}
set_robot_commands_max0.1042845959933299
set_robot_commands_mean0.09965485146332022
set_robot_commands_median0.10001238036065048
set_robot_commands_min0.09573908220544904
sim_compute_performance-ego_max0.07253786752808769
sim_compute_performance-ego_mean0.06984860858151223
sim_compute_performance-ego_median0.07003783091846075
sim_compute_performance-ego_min0.06608295747650539
sim_compute_robot_state-ego_max0.07930594364802043
sim_compute_robot_state-ego_mean0.07687476465136582
sim_compute_robot_state-ego_median0.07678272565205892
sim_compute_robot_state-ego_min0.07274575499505956
sim_compute_robot_state-npc0_max0.07287078665236557
sim_compute_robot_state-npc0_mean0.07084698461754567
sim_compute_robot_state-npc0_median0.07179236322079065
sim_compute_robot_state-npc0_min0.06745799621287334
sim_compute_robot_state-npc1_max0.07232784986495971
sim_compute_robot_state-npc1_mean0.07055396958514785
sim_compute_robot_state-npc1_median0.0715619584000156
sim_compute_robot_state-npc1_min0.0663107689869762
sim_compute_robot_state-npc2_max0.07189279397328695
sim_compute_robot_state-npc2_mean0.07012204837463067
sim_compute_robot_state-npc2_median0.07122658504732661
sim_compute_robot_state-npc2_min0.0662936304771849
sim_compute_robot_state-npc3_max0.07298133058368035
sim_compute_robot_state-npc3_mean0.07072899354320754
sim_compute_robot_state-npc3_median0.07193901239692485
sim_compute_robot_state-npc3_min0.06562667650214592
sim_compute_sim_state_max0.04072420597076416
sim_compute_sim_state_mean0.039481163787785975
sim_compute_sim_state_median0.04017598763951716
sim_compute_sim_state_min0.036069920134646695
sim_physics_max0.189065043431408
sim_physics_mean0.18121175028037656
sim_physics_median0.182348615769651
sim_physics_min0.16914276094395717
sim_render-ego_max0.06551824245812758
sim_render-ego_mean0.0635789099917449
sim_render-ego_median0.06409026861190796
sim_render-ego_min0.05982533340290381
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean13.590000000000057
survival_time_min11.65000000000003
207772964Julian ZillyBaseline-IL-logs-tensorflowaido2-LFVI-sim-testingstep1-simulationhost-errorno8410:59:34
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2964/step1-simulation-ip-172-31-38-104-5317-job20777/logs/challenges-runner/stdout.log'
(hidden)