Job ID submission user user label challenge step status up to date evaluator date started date completed duration message 31151
6355
Amaury Camus 🇨🇭challenge-aido_LF-template-random aido3-off-LF-real-validation
eval0-visualize error no MyEval
2020-02-13 17:06:49+00:00 2020-02-13 17:08:24+00:00 0:01:35 InvalidEvaluator:
Tr [...] InvalidEvaluator:
Traceback (most recent call last):
File "main.py", line 19, in wrap
stats = evaluate_robotarium_log(logdir, outd=outd)
File "/project/evaluate_robotarium_log.py", line 41, in evaluate_robotarium_log
d = yaml.load(open(mapfile).read())
FileNotFoundError: [Errno 2] No such file or directory: '/challenges/previous-steps/eval0/trajectories/map.yaml'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 676, in scoring_context
yield cie
File "main.py", line 34, in <module>
wrap(cie)
File "main.py", line 24, in wrap
raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 31112
6294
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation success yes MyEval
2020-02-13 11:41:34+00:00 2020-02-13 11:48:57+00:00 0:07:23 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 2.989390939479974 survival_time_median 14.950000000000076 deviation-center-line_median 0.6208176912586028 in-drivable-lane_median 0
other stats agent_compute-ego_max 0.01878458579381307 agent_compute-ego_mean 0.017017368793487548 agent_compute-ego_median 0.016646543343861897 agent_compute-ego_min 0.01546119451522827 deviation-center-line_max 0.870542544660809 deviation-center-line_mean 0.676638461937978 deviation-center-line_min 0.572040341697189 deviation-heading_max 2.3701441574852384 deviation-heading_mean 1.902083318016055 deviation-heading_median 2.020185012301047 deviation-heading_min 1.4013815211853973 driven_any_max 3.587766476950398 driven_any_mean 3.044560760433511 driven_any_median 3.0218742630559117 driven_any_min 2.271725515587425 driven_lanedir_consec_max 3.5653494016831635 driven_lanedir_consec_mean 3.0144114404695346 driven_lanedir_consec_min 2.2477225830986063 driven_lanedir_max 3.5653494016831635 driven_lanedir_mean 3.0144114404695346 driven_lanedir_median 2.989390939479974 driven_lanedir_min 2.2477225830986063 in-drivable-lane_max 0 in-drivable-lane_mean 0 in-drivable-lane_min 0 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.271725515587425, "sim_physics": 0.0521248467763265, "survival_time": 14.950000000000076, "driven_lanedir": 2.2477225830986063, "sim_render-ego": 0.00714099645614624, "in-drivable-lane": 0, "agent_compute-ego": 0.01878458579381307, "deviation-heading": 2.020185012301047, "set_robot_commands": 0.009670936266581215, "deviation-center-line": 0.572040341697189, "driven_lanedir_consec": 2.2477225830986063, "sim_compute_sim_state": 0.004193565050760905, "sim_compute_performance-ego": 0.004700158437093099, "sim_compute_robot_state-ego": 0.005772005716959635}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.757213485152363, "sim_physics": 0.04723353703816731, "survival_time": 14.950000000000076, "driven_lanedir": 2.7376124646514053, "sim_render-ego": 0.006975277264912923, "in-drivable-lane": 0, "agent_compute-ego": 0.016646543343861897, "deviation-heading": 1.6403481780947269, "set_robot_commands": 0.008690992991129557, "deviation-center-line": 0.617641578145205, "driven_lanedir_consec": 2.7376124646514053, "sim_compute_sim_state": 0.0037564659118652344, "sim_compute_performance-ego": 0.004401423136393229, "sim_compute_robot_state-ego": 0.0054229315121968585}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 3.587766476950398, "sim_physics": 0.04758253812789917, "survival_time": 14.950000000000076, "driven_lanedir": 3.5319818134345207, "sim_render-ego": 0.006927462418874105, "in-drivable-lane": 0, "agent_compute-ego": 0.01546119451522827, "deviation-heading": 2.3701441574852384, "set_robot_commands": 0.009029197692871091, "deviation-center-line": 0.870542544660809, "driven_lanedir_consec": 3.5319818134345207, "sim_compute_sim_state": 0.0038524762789408367, "sim_compute_performance-ego": 0.004382890860239665, "sim_compute_robot_state-ego": 0.005415594577789307}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 3.584224061421459, "sim_physics": 0.04888757546742757, "survival_time": 14.950000000000076, "driven_lanedir": 3.5653494016831635, "sim_render-ego": 0.006977479457855224, "in-drivable-lane": 0, "agent_compute-ego": 0.016435712178548178, "deviation-heading": 1.4013815211853973, "set_robot_commands": 0.009162846406300863, "deviation-center-line": 0.6208176912586028, "driven_lanedir_consec": 3.5653494016831635, "sim_compute_sim_state": 0.003877971172332763, "sim_compute_performance-ego": 0.004427278836568196, "sim_compute_robot_state-ego": 0.005543184280395508}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 3.0218742630559117, "sim_physics": 0.05070802529652913, "survival_time": 14.950000000000076, "driven_lanedir": 2.989390939479974, "sim_render-ego": 0.00706391970316569, "in-drivable-lane": 0, "agent_compute-ego": 0.017758808135986327, "deviation-heading": 2.078357721013867, "set_robot_commands": 0.009229853947957356, "deviation-center-line": 0.7021501539280846, "driven_lanedir_consec": 2.989390939479974, "sim_compute_sim_state": 0.00401114543279012, "sim_compute_performance-ego": 0.004437133471171062, "sim_compute_robot_state-ego": 0.005523326396942138}}set_robot_commands_max 0.009670936266581215 set_robot_commands_mean 0.009156765460968018 set_robot_commands_median 0.009162846406300863 set_robot_commands_min 0.008690992991129557 sim_compute_performance-ego_max 0.004700158437093099 sim_compute_performance-ego_mean 0.00446977694829305 sim_compute_performance-ego_median 0.004427278836568196 sim_compute_performance-ego_min 0.004382890860239665 sim_compute_robot_state-ego_max 0.005772005716959635 sim_compute_robot_state-ego_mean 0.005535408496856689 sim_compute_robot_state-ego_median 0.005523326396942138 sim_compute_robot_state-ego_min 0.005415594577789307 sim_compute_sim_state_max 0.004193565050760905 sim_compute_sim_state_mean 0.003938324769337972 sim_compute_sim_state_median 0.003877971172332763 sim_compute_sim_state_min 0.0037564659118652344 sim_physics_max 0.0521248467763265 sim_physics_mean 0.049307304541269935 sim_physics_median 0.04888757546742757 sim_physics_min 0.04723353703816731 sim_render-ego_max 0.00714099645614624 sim_render-ego_mean 0.007017027060190836 sim_render-ego_median 0.006977479457855224 sim_render-ego_min 0.006927462418874105 simulation-passed 1 survival_time_max 14.950000000000076 survival_time_mean 14.950000000000076 survival_time_min 14.950000000000076
No reset possible 31101
6302
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation success yes MyEval
2020-02-13 11:33:38+00:00 2020-02-13 11:40:59+00:00 0:07:21 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 2.8837465483863416 survival_time_median 14.950000000000076 deviation-center-line_median 0.7656541604983518 in-drivable-lane_median 0
other stats agent_compute-ego_max 0.018660576343536375 agent_compute-ego_mean 0.017155965010325114 agent_compute-ego_median 0.017465937932332358 agent_compute-ego_min 0.01559280792872111 deviation-center-line_max 0.8620634690698115 deviation-center-line_mean 0.720186603848141 deviation-center-line_min 0.5207063014276943 deviation-heading_max 2.9892232325639556 deviation-heading_mean 2.308221829198647 deviation-heading_median 2.5204870992726915 deviation-heading_min 1.239634713370836 driven_any_max 3.7487480096654577 driven_any_mean 3.0302890941645684 driven_any_median 2.9556616511500047 driven_any_min 2.5015569337614987 driven_lanedir_consec_max 3.730322238358534 driven_lanedir_consec_mean 2.9743747852737257 driven_lanedir_consec_min 2.4205107344826367 driven_lanedir_max 3.730322238358534 driven_lanedir_mean 2.9743747852737257 driven_lanedir_median 2.8837465483863416 driven_lanedir_min 2.4205107344826367 in-drivable-lane_max 0 in-drivable-lane_mean 0 in-drivable-lane_min 0 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.5015569337614987, "sim_physics": 0.04740603685379028, "survival_time": 14.950000000000076, "driven_lanedir": 2.4205107344826367, "sim_render-ego": 0.006762684981028239, "in-drivable-lane": 0, "agent_compute-ego": 0.017796527544657388, "deviation-heading": 2.9892232325639556, "set_robot_commands": 0.008743464946746826, "deviation-center-line": 0.7656541604983518, "driven_lanedir_consec": 2.4205107344826367, "sim_compute_sim_state": 0.003745003541310628, "sim_compute_performance-ego": 0.004367448488871256, "sim_compute_robot_state-ego": 0.005372262795766194}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.9556616511500047, "sim_physics": 0.04696698506673177, "survival_time": 14.950000000000076, "driven_lanedir": 2.8837465483863416, "sim_render-ego": 0.006882426738739014, "in-drivable-lane": 0, "agent_compute-ego": 0.017465937932332358, "deviation-heading": 2.693399780298901, "set_robot_commands": 0.008577689329783122, "deviation-center-line": 0.8052193090832434, "driven_lanedir_consec": 2.8837465483863416, "sim_compute_sim_state": 0.003649803797403971, "sim_compute_performance-ego": 0.00427301565806071, "sim_compute_robot_state-ego": 0.005296006202697754}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 3.0615761525740544, "sim_physics": 0.046471025149027505, "survival_time": 14.950000000000076, "driven_lanedir": 2.988648985461994, "sim_render-ego": 0.006828200022379557, "in-drivable-lane": 0, "agent_compute-ego": 0.01559280792872111, "deviation-heading": 2.5204870992726915, "set_robot_commands": 0.008993215560913086, "deviation-center-line": 0.8620634690698115, "driven_lanedir_consec": 2.988648985461994, "sim_compute_sim_state": 0.003845154444376627, "sim_compute_performance-ego": 0.004300076961517334, "sim_compute_robot_state-ego": 0.005325399239857992}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 3.7487480096654577, "sim_physics": 0.04809330781300863, "survival_time": 14.950000000000076, "driven_lanedir": 3.730322238358534, "sim_render-ego": 0.006980834007263184, "in-drivable-lane": 0, "agent_compute-ego": 0.016263975302378338, "deviation-heading": 1.239634713370836, "set_robot_commands": 0.009037465254465738, "deviation-center-line": 0.5207063014276943, "driven_lanedir_consec": 3.730322238358534, "sim_compute_sim_state": 0.003841888109842936, "sim_compute_performance-ego": 0.004399154186248779, "sim_compute_robot_state-ego": 0.005519095261891683}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.8839027236718273, "sim_physics": 0.04833846966425578, "survival_time": 14.950000000000076, "driven_lanedir": 2.848645419679123, "sim_render-ego": 0.007107581297556559, "in-drivable-lane": 0, "agent_compute-ego": 0.018660576343536375, "deviation-heading": 2.098364320486851, "set_robot_commands": 0.008864172299702962, "deviation-center-line": 0.6472897791616044, "driven_lanedir_consec": 2.848645419679123, "sim_compute_sim_state": 0.003836554686228434, "sim_compute_performance-ego": 0.004481995105743408, "sim_compute_robot_state-ego": 0.005598954359690348}}set_robot_commands_max 0.009037465254465738 set_robot_commands_mean 0.008843201478322345 set_robot_commands_median 0.008864172299702962 set_robot_commands_min 0.008577689329783122 sim_compute_performance-ego_max 0.004481995105743408 sim_compute_performance-ego_mean 0.004364338080088298 sim_compute_performance-ego_median 0.004367448488871256 sim_compute_performance-ego_min 0.00427301565806071 sim_compute_robot_state-ego_max 0.005598954359690348 sim_compute_robot_state-ego_mean 0.005422343571980795 sim_compute_robot_state-ego_median 0.005372262795766194 sim_compute_robot_state-ego_min 0.005296006202697754 sim_compute_sim_state_max 0.003845154444376627 sim_compute_sim_state_mean 0.00378368091583252 sim_compute_sim_state_median 0.003836554686228434 sim_compute_sim_state_min 0.003649803797403971 sim_physics_max 0.04833846966425578 sim_physics_mean 0.047455164909362794 sim_physics_median 0.04740603685379028 sim_physics_min 0.046471025149027505 sim_render-ego_max 0.007107581297556559 sim_render-ego_mean 0.006912345409393311 sim_render-ego_median 0.006882426738739014 sim_render-ego_min 0.006762684981028239 simulation-passed 1 survival_time_max 14.950000000000076 survival_time_mean 14.950000000000076 survival_time_min 14.950000000000076
No reset possible 31094
6308
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation success yes MyEval
2020-02-13 11:25:54+00:00 2020-02-13 11:33:16+00:00 0:07:22 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 2.787792570059231 survival_time_median 14.950000000000076 deviation-center-line_median 0.887071824845663 in-drivable-lane_median 0
other stats agent_compute-ego_max 0.02000847180684407 agent_compute-ego_mean 0.019219390869140627 agent_compute-ego_median 0.019109227657318116 agent_compute-ego_min 0.01842610915501913 deviation-center-line_max 1.0644523866769888 deviation-center-line_mean 0.8542193495354276 deviation-center-line_min 0.6254732535936984 deviation-heading_max 3.252821516952374 deviation-heading_mean 2.9294176345504255 deviation-heading_median 2.84628413210395 deviation-heading_min 2.4741905848165535 driven_any_max 2.9209296448187443 driven_any_mean 2.804959637119363 driven_any_median 2.8587694425018157 driven_any_min 2.612750940742663 driven_lanedir_consec_max 2.8271512141300335 driven_lanedir_consec_mean 2.7299518324842835 driven_lanedir_consec_min 2.5257871257056945 driven_lanedir_max 2.8271512141300335 driven_lanedir_mean 2.7299518324842835 driven_lanedir_median 2.787792570059231 driven_lanedir_min 2.5257871257056945 in-drivable-lane_max 0 in-drivable-lane_mean 0 in-drivable-lane_min 0 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.612750940742663, "sim_physics": 0.048211587270100914, "survival_time": 14.950000000000076, "driven_lanedir": 2.5257871257056945, "sim_render-ego": 0.007036640644073487, "in-drivable-lane": 0, "agent_compute-ego": 0.01842610915501913, "deviation-heading": 3.252821516952374, "set_robot_commands": 0.008874514102935792, "deviation-center-line": 0.9910285597106778, "driven_lanedir_consec": 2.5257871257056945, "sim_compute_sim_state": 0.003690110047658285, "sim_compute_performance-ego": 0.0044718623161315914, "sim_compute_robot_state-ego": 0.005456126530965169}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.770180478929309, "sim_physics": 0.050099507967631025, "survival_time": 14.950000000000076, "driven_lanedir": 2.7031882035934904, "sim_render-ego": 0.007216227054595947, "in-drivable-lane": 0, "agent_compute-ego": 0.019104133447011312, "deviation-heading": 2.8247114869974967, "set_robot_commands": 0.008844610850016275, "deviation-center-line": 0.887071824845663, "driven_lanedir_consec": 2.7031882035934904, "sim_compute_sim_state": 0.0037657411893208823, "sim_compute_performance-ego": 0.0044766759872436526, "sim_compute_robot_state-ego": 0.005564608573913574}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.9209296448187443, "sim_physics": 0.05104555686314901, "survival_time": 14.950000000000076, "driven_lanedir": 2.8271512141300335, "sim_render-ego": 0.007547637621561686, "in-drivable-lane": 0, "agent_compute-ego": 0.019109227657318116, "deviation-heading": 3.249080451881752, "set_robot_commands": 0.009464761416117353, "deviation-center-line": 1.0644523866769888, "driven_lanedir_consec": 2.8271512141300335, "sim_compute_sim_state": 0.003976021607716878, "sim_compute_performance-ego": 0.00483592430750529, "sim_compute_robot_state-ego": 0.005965252717336019}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.8587694425018157, "sim_physics": 0.05170296748479208, "survival_time": 14.950000000000076, "driven_lanedir": 2.8058400489329687, "sim_render-ego": 0.007595949172973633, "in-drivable-lane": 0, "agent_compute-ego": 0.0194490122795105, "deviation-heading": 2.4741905848165535, "set_robot_commands": 0.00946940819422404, "deviation-center-line": 0.6254732535936984, "driven_lanedir_consec": 2.8058400489329687, "sim_compute_sim_state": 0.0040676164627075195, "sim_compute_performance-ego": 0.004823048909505208, "sim_compute_robot_state-ego": 0.006017458438873291}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.8621676786042816, "sim_physics": 0.05077802658081055, "survival_time": 14.950000000000076, "driven_lanedir": 2.787792570059231, "sim_render-ego": 0.00728938897450765, "in-drivable-lane": 0, "agent_compute-ego": 0.02000847180684407, "deviation-heading": 2.84628413210395, "set_robot_commands": 0.00920387585957845, "deviation-center-line": 0.70307072285011, "driven_lanedir_consec": 2.787792570059231, "sim_compute_sim_state": 0.004064633051554362, "sim_compute_performance-ego": 0.004595167636871338, "sim_compute_robot_state-ego": 0.0057439915339152015}}set_robot_commands_max 0.00946940819422404 set_robot_commands_mean 0.00917143408457438 set_robot_commands_median 0.00920387585957845 set_robot_commands_min 0.008844610850016275 sim_compute_performance-ego_max 0.00483592430750529 sim_compute_performance-ego_mean 0.004640535831451416 sim_compute_performance-ego_median 0.004595167636871338 sim_compute_performance-ego_min 0.0044718623161315914 sim_compute_robot_state-ego_max 0.006017458438873291 sim_compute_robot_state-ego_mean 0.005749487559000651 sim_compute_robot_state-ego_median 0.0057439915339152015 sim_compute_robot_state-ego_min 0.005456126530965169 sim_compute_sim_state_max 0.0040676164627075195 sim_compute_sim_state_mean 0.0039128244717915855 sim_compute_sim_state_median 0.003976021607716878 sim_compute_sim_state_min 0.003690110047658285 sim_physics_max 0.05170296748479208 sim_physics_mean 0.05036752923329672 sim_physics_median 0.05077802658081055 sim_physics_min 0.048211587270100914 sim_render-ego_max 0.007595949172973633 sim_render-ego_mean 0.007337168693542481 sim_render-ego_median 0.00728938897450765 sim_render-ego_min 0.007036640644073487 simulation-passed 1 survival_time_max 14.950000000000076 survival_time_mean 14.950000000000076 survival_time_min 14.950000000000076
No reset possible 31086
6314
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation success yes MyEval
2020-02-13 11:18:25+00:00 2020-02-13 11:25:45+00:00 0:07:20 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 3.076713635507828 survival_time_median 14.950000000000076 deviation-center-line_median 0.7145369124326418 in-drivable-lane_median 0
other stats agent_compute-ego_max 0.021255333423614502 agent_compute-ego_mean 0.019075394471486408 agent_compute-ego_median 0.01888407309850057 agent_compute-ego_min 0.01570566733678182 deviation-center-line_max 0.960601420726504 deviation-center-line_mean 0.7411394310355301 deviation-center-line_min 0.5891849152797769 deviation-heading_max 3.308378169231783 deviation-heading_mean 1.916747216174866 deviation-heading_median 1.605428557447187 deviation-heading_min 1.183970635131239 driven_any_max 3.486225119126711 driven_any_mean 3.0195417781453258 driven_any_median 3.0923930947042932 driven_any_min 2.593015414317566 driven_lanedir_consec_max 3.4725342663339167 driven_lanedir_consec_mean 2.9825584769893236 driven_lanedir_consec_min 2.5589401514369974 driven_lanedir_max 3.4725342663339167 driven_lanedir_mean 2.9825584769893236 driven_lanedir_median 3.076713635507828 driven_lanedir_min 2.5589401514369974 in-drivable-lane_max 0 in-drivable-lane_mean 0 in-drivable-lane_min 0 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.593015414317566, "sim_physics": 0.05101423104604085, "survival_time": 14.950000000000076, "driven_lanedir": 2.5589401514369974, "sim_render-ego": 0.00715985377629598, "in-drivable-lane": 0, "agent_compute-ego": 0.02080286105473836, "deviation-heading": 2.021321897315301, "set_robot_commands": 0.00906137466430664, "deviation-center-line": 0.8503953463080062, "driven_lanedir_consec": 2.5589401514369974, "sim_compute_sim_state": 0.003826134204864502, "sim_compute_performance-ego": 0.004678115049997966, "sim_compute_robot_state-ego": 0.005686423778533936}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 3.0923930947042932, "sim_physics": 0.05290771802266439, "survival_time": 14.950000000000076, "driven_lanedir": 3.076713635507828, "sim_render-ego": 0.007510777314503988, "in-drivable-lane": 0, "agent_compute-ego": 0.021255333423614502, "deviation-heading": 1.4646368217488193, "set_robot_commands": 0.00927293618520101, "deviation-center-line": 0.7145369124326418, "driven_lanedir_consec": 3.076713635507828, "sim_compute_sim_state": 0.004013783137003581, "sim_compute_performance-ego": 0.004707010587056478, "sim_compute_robot_state-ego": 0.00582356055577596}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.7076625024752787, "sim_physics": 0.046506327788035075, "survival_time": 14.950000000000076, "driven_lanedir": 2.6083438715961793, "sim_render-ego": 0.006950419743855794, "in-drivable-lane": 0, "agent_compute-ego": 0.01570566733678182, "deviation-heading": 3.308378169231783, "set_robot_commands": 0.009014410972595217, "deviation-center-line": 0.960601420726504, "driven_lanedir_consec": 2.6083438715961793, "sim_compute_sim_state": 0.00372053861618042, "sim_compute_performance-ego": 0.004390749136606852, "sim_compute_robot_state-ego": 0.005417338212331136}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 3.486225119126711, "sim_physics": 0.047238152821858725, "survival_time": 14.950000000000076, "driven_lanedir": 3.4725342663339167, "sim_render-ego": 0.00696592648824056, "in-drivable-lane": 0, "agent_compute-ego": 0.018729037443796795, "deviation-heading": 1.183970635131239, "set_robot_commands": 0.00853155771891276, "deviation-center-line": 0.5909785604307218, "driven_lanedir_consec": 3.4725342663339167, "sim_compute_sim_state": 0.0036406572659810382, "sim_compute_performance-ego": 0.00435383955637614, "sim_compute_robot_state-ego": 0.005441530545552572}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 3.218412760102779, "sim_physics": 0.04796937783559163, "survival_time": 14.950000000000076, "driven_lanedir": 3.1962604600716973, "sim_render-ego": 0.00710783322652181, "in-drivable-lane": 0, "agent_compute-ego": 0.01888407309850057, "deviation-heading": 1.605428557447187, "set_robot_commands": 0.00866278092066447, "deviation-center-line": 0.5891849152797769, "driven_lanedir_consec": 3.1962604600716973, "sim_compute_sim_state": 0.0037118903795878098, "sim_compute_performance-ego": 0.004441097577412923, "sim_compute_robot_state-ego": 0.005499724547068278}}set_robot_commands_max 0.00927293618520101 set_robot_commands_mean 0.008908612092336019 set_robot_commands_median 0.009014410972595217 set_robot_commands_min 0.00853155771891276 sim_compute_performance-ego_max 0.004707010587056478 sim_compute_performance-ego_mean 0.0045141623814900726 sim_compute_performance-ego_median 0.004441097577412923 sim_compute_performance-ego_min 0.00435383955637614 sim_compute_robot_state-ego_max 0.00582356055577596 sim_compute_robot_state-ego_mean 0.005573715527852376 sim_compute_robot_state-ego_median 0.005499724547068278 sim_compute_robot_state-ego_min 0.005417338212331136 sim_compute_sim_state_max 0.004013783137003581 sim_compute_sim_state_mean 0.0037826007207234697 sim_compute_sim_state_median 0.00372053861618042 sim_compute_sim_state_min 0.0036406572659810382 sim_physics_max 0.05290771802266439 sim_physics_mean 0.049127161502838135 sim_physics_median 0.04796937783559163 sim_physics_min 0.046506327788035075 sim_render-ego_max 0.007510777314503988 sim_render-ego_mean 0.007138962109883627 sim_render-ego_median 0.00710783322652181 sim_render-ego_min 0.006950419743855794 simulation-passed 1 survival_time_max 14.950000000000076 survival_time_mean 14.950000000000076 survival_time_min 14.950000000000076
No reset possible 31071
6322
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation success yes MyEval
2020-02-13 11:10:34+00:00 2020-02-13 11:17:58+00:00 0:07:24 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 3.3822661545444843 survival_time_median 14.950000000000076 deviation-center-line_median 0.9566875661775468 in-drivable-lane_median 0
other stats agent_compute-ego_max 0.02035071055094401 agent_compute-ego_mean 0.01926685333251953 agent_compute-ego_median 0.019600438276926677 agent_compute-ego_min 0.01767043352127075 deviation-center-line_max 1.055156914832329 deviation-center-line_mean 0.9566579522347376 deviation-center-line_min 0.8917917279991175 deviation-heading_max 3.537610156953064 deviation-heading_mean 3.00264776922766 deviation-heading_median 2.9570580584708765 deviation-heading_min 2.352402175086589 driven_any_max 4.131779463461953 driven_any_mean 3.5506791047776525 driven_any_median 3.4543675781173593 driven_any_min 2.986997837264857 driven_lanedir_consec_max 4.053582358731587 driven_lanedir_consec_mean 3.453883721614563 driven_lanedir_consec_min 2.877043900337325 driven_lanedir_max 4.053582358731587 driven_lanedir_mean 3.453883721614563 driven_lanedir_median 3.3822661545444843 driven_lanedir_min 2.877043900337325 in-drivable-lane_max 0 in-drivable-lane_mean 0 in-drivable-lane_min 0 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.986997837264857, "sim_physics": 0.05196542819341024, "survival_time": 14.950000000000076, "driven_lanedir": 2.877043900337325, "sim_render-ego": 0.007091030279795329, "in-drivable-lane": 0, "agent_compute-ego": 0.020119907855987548, "deviation-heading": 3.5098621520171056, "set_robot_commands": 0.009344953695933024, "deviation-center-line": 1.055156914832329, "driven_lanedir_consec": 2.877043900337325, "sim_compute_sim_state": 0.004062480926513672, "sim_compute_performance-ego": 0.004537130196889241, "sim_compute_robot_state-ego": 0.005598161220550537}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 3.4543675781173593, "sim_physics": 0.04988369226455688, "survival_time": 14.950000000000076, "driven_lanedir": 3.3822661545444843, "sim_render-ego": 0.007064514954884847, "in-drivable-lane": 0, "agent_compute-ego": 0.019600438276926677, "deviation-heading": 2.6563063036106636, "set_robot_commands": 0.009058087666829428, "deviation-center-line": 0.9566875661775468, "driven_lanedir_consec": 3.3822661545444843, "sim_compute_sim_state": 0.00407756487528483, "sim_compute_performance-ego": 0.004431037902832031, "sim_compute_robot_state-ego": 0.005561001300811767}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 4.131779463461953, "sim_physics": 0.050957260131835935, "survival_time": 14.950000000000076, "driven_lanedir": 4.053582358731587, "sim_render-ego": 0.007303698062896729, "in-drivable-lane": 0, "agent_compute-ego": 0.01767043352127075, "deviation-heading": 2.352402175086589, "set_robot_commands": 0.009603611628214518, "deviation-center-line": 0.8917917279991175, "driven_lanedir_consec": 4.053582358731587, "sim_compute_sim_state": 0.004088157018025716, "sim_compute_performance-ego": 0.004709060986836751, "sim_compute_robot_state-ego": 0.005860195954640706}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 3.830894575268508, "sim_physics": 0.05152835210164388, "survival_time": 14.950000000000076, "driven_lanedir": 3.720557640630595, "sim_render-ego": 0.0072316948572794595, "in-drivable-lane": 0, "agent_compute-ego": 0.01859277645746867, "deviation-heading": 2.9570580584708765, "set_robot_commands": 0.0094252347946167, "deviation-center-line": 0.9785948085667976, "driven_lanedir_consec": 3.720557640630595, "sim_compute_sim_state": 0.004075928529103597, "sim_compute_performance-ego": 0.004558758735656738, "sim_compute_robot_state-ego": 0.005727011362711589}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 3.3493560697755855, "sim_physics": 0.05094654719034831, "survival_time": 14.950000000000076, "driven_lanedir": 3.2359685538288216, "sim_render-ego": 0.007121841112772624, "in-drivable-lane": 0, "agent_compute-ego": 0.02035071055094401, "deviation-heading": 3.537610156953064, "set_robot_commands": 0.009029051462809246, "deviation-center-line": 0.9010587435978972, "driven_lanedir_consec": 3.2359685538288216, "sim_compute_sim_state": 0.00390167236328125, "sim_compute_performance-ego": 0.004561842282613119, "sim_compute_robot_state-ego": 0.0057741657892862955}}set_robot_commands_max 0.009603611628214518 set_robot_commands_mean 0.009292187849680583 set_robot_commands_median 0.009344953695933024 set_robot_commands_min 0.009029051462809246 sim_compute_performance-ego_max 0.004709060986836751 sim_compute_performance-ego_mean 0.004559566020965576 sim_compute_performance-ego_median 0.004558758735656738 sim_compute_performance-ego_min 0.004431037902832031 sim_compute_robot_state-ego_max 0.005860195954640706 sim_compute_robot_state-ego_mean 0.005704107125600179 sim_compute_robot_state-ego_median 0.005727011362711589 sim_compute_robot_state-ego_min 0.005561001300811767 sim_compute_sim_state_max 0.004088157018025716 sim_compute_sim_state_mean 0.004041160742441813 sim_compute_sim_state_median 0.004075928529103597 sim_compute_sim_state_min 0.00390167236328125 sim_physics_max 0.05196542819341024 sim_physics_mean 0.05105625597635906 sim_physics_median 0.050957260131835935 sim_physics_min 0.04988369226455688 sim_render-ego_max 0.007303698062896729 sim_render-ego_mean 0.007162555853525797 sim_render-ego_median 0.007121841112772624 sim_render-ego_min 0.007064514954884847 simulation-passed 1 survival_time_max 14.950000000000076 survival_time_mean 14.950000000000076 survival_time_min 14.950000000000076
No reset possible 31063
6329
YUNG-CHIEN CHOU 🇹🇼challenge-aido_LF-template-random aido3-off-LF-sim-validation
step1-simulation success yes MyEval
2020-02-13 11:07:30+00:00 2020-02-13 11:10:18+00:00 0:02:48 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 0.4838163264350306 survival_time_median 1.7000000000000008 deviation-center-line_median 0.08751730474052305 in-drivable-lane_median 0
other stats agent_compute-ego_max 0.014212085650517393 agent_compute-ego_mean 0.013367453782707688 agent_compute-ego_median 0.013197765630834244 agent_compute-ego_min 0.0125480055809021 deviation-center-line_max 0.10683019887032848 deviation-center-line_mean 0.08719800587294524 deviation-center-line_min 0.07065220523066396 deviation-heading_max 0.5050851849885492 deviation-heading_mean 0.37472646794797326 deviation-heading_median 0.3711400756311837 deviation-heading_min 0.242154099114489 driven_any_max 1.224736399271231 driven_any_mean 0.7001682289165533 driven_any_median 0.6334906616120762 driven_any_min 0.311949329092399 driven_lanedir_consec_max 0.6642632200434848 driven_lanedir_consec_mean 0.47031102562542887 driven_lanedir_consec_min 0.28855079430839536 driven_lanedir_max 0.6642632200434848 driven_lanedir_mean 0.47031102562542887 driven_lanedir_median 0.4838163264350306 driven_lanedir_min 0.28855079430839536 in-drivable-lane_max 1.5999999999999963 in-drivable-lane_mean 0.4499999999999989 in-drivable-lane_min 0 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0078896084088962, "sim_physics": 0.05147772568922777, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6642632200434848, "sim_render-ego": 0.007833274511190562, "in-drivable-lane": 0.6499999999999977, "agent_compute-ego": 0.014212085650517393, "deviation-heading": 0.5050851849885492, "set_robot_commands": 0.005973462875072773, "deviation-center-line": 0.07065220523066396, "driven_lanedir_consec": 0.6642632200434848, "sim_compute_sim_state": 0.003070803788992075, "sim_compute_performance-ego": 0.0049063379947955794, "sim_compute_robot_state-ego": 0.005852942283336933}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.311949329092399, "sim_physics": 0.05028897523880005, "survival_time": 1.0000000000000002, "driven_lanedir": 0.3030566281970668, "sim_render-ego": 0.007065272331237793, "in-drivable-lane": 0, "agent_compute-ego": 0.0125480055809021, "deviation-heading": 0.242154099114489, "set_robot_commands": 0.005612635612487793, "deviation-center-line": 0.08751730474052305, "driven_lanedir_consec": 0.3030566281970668, "sim_compute_sim_state": 0.0029543280601501466, "sim_compute_performance-ego": 0.004218459129333496, "sim_compute_robot_state-ego": 0.0051962494850158695}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.6334906616120762, "sim_physics": 0.04870745013741886, "survival_time": 1.7000000000000008, "driven_lanedir": 0.6118681591431665, "sim_render-ego": 0.007423583198996151, "in-drivable-lane": 0, "agent_compute-ego": 0.013197765630834244, "deviation-heading": 0.3711400756311837, "set_robot_commands": 0.005790310747483198, "deviation-center-line": 0.10683019887032848, "driven_lanedir_consec": 0.6118681591431665, "sim_compute_sim_state": 0.0030317236395443186, "sim_compute_performance-ego": 0.0044124687419218175, "sim_compute_robot_state-ego": 0.0054301794837502875}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.224736399271231, "sim_physics": 0.05109729141485496, "survival_time": 3.049999999999997, "driven_lanedir": 0.4838163264350306, "sim_render-ego": 0.007836478655455543, "in-drivable-lane": 1.5999999999999963, "agent_compute-ego": 0.01403570566021028, "deviation-heading": 0.29149367303980006, "set_robot_commands": 0.006136608905479556, "deviation-center-line": 0.10020114979294129, "driven_lanedir_consec": 0.4838163264350306, "sim_compute_sim_state": 0.0031950278360335553, "sim_compute_performance-ego": 0.00476673782848921, "sim_compute_robot_state-ego": 0.005893210895725938}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.32277514619816383, "sim_physics": 0.05185674537311901, "survival_time": 1.1000000000000003, "driven_lanedir": 0.28855079430839536, "sim_render-ego": 0.007243665781888095, "in-drivable-lane": 0, "agent_compute-ego": 0.01284370639107444, "deviation-heading": 0.4637593069658444, "set_robot_commands": 0.005725080316716974, "deviation-center-line": 0.07078917073026944, "driven_lanedir_consec": 0.28855079430839536, "sim_compute_sim_state": 0.0030702569267966533, "sim_compute_performance-ego": 0.004487124356356534, "sim_compute_robot_state-ego": 0.005600571632385254}}set_robot_commands_max 0.006136608905479556 set_robot_commands_mean 0.005847619691448059 set_robot_commands_median 0.005790310747483198 set_robot_commands_min 0.005612635612487793 sim_compute_performance-ego_max 0.0049063379947955794 sim_compute_performance-ego_mean 0.004558225610179328 sim_compute_performance-ego_median 0.004487124356356534 sim_compute_performance-ego_min 0.004218459129333496 sim_compute_robot_state-ego_max 0.005893210895725938 sim_compute_robot_state-ego_mean 0.005594630756042856 sim_compute_robot_state-ego_median 0.005600571632385254 sim_compute_robot_state-ego_min 0.0051962494850158695 sim_compute_sim_state_max 0.0031950278360335553 sim_compute_sim_state_mean 0.00306442805030335 sim_compute_sim_state_median 0.0030702569267966533 sim_compute_sim_state_min 0.0029543280601501466 sim_physics_max 0.05185674537311901 sim_physics_mean 0.05068563757068413 sim_physics_median 0.05109729141485496 sim_physics_min 0.04870745013741886 sim_render-ego_max 0.007836478655455543 sim_render-ego_mean 0.0074804548957536285 sim_render-ego_median 0.007423583198996151 sim_render-ego_min 0.007065272331237793 simulation-passed 1 survival_time_max 3.049999999999997 survival_time_mean 1.8899999999999997 survival_time_min 1.0000000000000002
No reset possible 31057
6345
Susanne Keller PredictorLast aido3-off-LF-sim-validation
step1-simulation error yes MyEval
2020-02-13 11:06:13+00:00 2020-02-13 11:06:55+00:00 0:00:42 Unexpected exception [...] Unexpected exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/zuper_nodes/compatibility.py", line 20, in check_compatible_protocol
raise IncompatibleProtocol(msg)
zuper_nodes.compatibility.IncompatibleProtocol: First protocol misses input "get_commands".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 676, in scoring_context
yield cie
File "experiment_manager.py", line 446, in <module>
wrap(cie)
File "experiment_manager.py", line 434, in wrap
main(cie, logdir, attempts)
File "experiment_manager.py", line 79, in main
agent_ci._get_node_protocol(timeout=config.timeout_initialization)
File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 88, in _get_node_protocol
check_compatible_protocol(self.node_protocol, self.expect_protocol)
File "/usr/local/lib/python3.7/site-packages/zuper_nodes/compatibility.py", line 43, in check_compatible_protocol
raise IncompatibleProtocol(msg) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
p1: |InteractionProtocol(description='An estimator receives a stream of values and must predict the next value.', inputs={'seed': <class 'int'>, 'observations': <class 'float'>, 'get_prediction': <class 'NoneType'>}, outputs={'prediction': <class 'float'>}, language='(in:seed)? ; ((in:observations | (in:get_prediction ; out:prediction))*)')
p2: |InteractionProtocol(description='Generic protocol for an agent that receives "observations" and responds \nwith "commands".\n\n"episode_start" marks the beginning of an episode.', inputs={'observations': typing.Any, 'seed': <class 'int'>, 'get_commands': <class 'NoneType'>, 'episode_start': <class 'aido_schemas.protocol_agent.EpisodeStart'>}, outputs={'commands': typing.Any}, language='(in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)')
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 31038
6295
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation error no MyEval
2020-02-13 10:51:59+00:00 2020-02-13 10:52:37+00:00 0:00:38 The container "simul [...] The container "simulator" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 31026
6300
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation error no MyEval
2020-02-13 10:51:11+00:00 2020-02-13 10:51:50+00:00 0:00:39 The container "simul [...] The container "simulator" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 31011
6303
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation error no MyEval
2020-02-13 10:50:15+00:00 2020-02-13 10:50:55+00:00 0:00:40 The container "simul [...] The container "simulator" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 31007
6320
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation error no MyEval
2020-02-13 10:49:14+00:00 2020-02-13 10:49:53+00:00 0:00:39 The container "simul [...] The container "simulator" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 30992
6321
Rey Wiyatno 🇨🇦challenge-aido_LF-baseline-duckietown aido3-off-LF-sim-validation
step1-simulation error no MyEval
2020-02-13 10:47:48+00:00 2020-02-13 10:48:46+00:00 0:00:58 The container "simul [...] The container "simulator" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 30980
6353
Amaury Camus 🇨🇭challenge-aido_LF-template-random aido3-off-LF-sim-validation
step1-simulation error no MyEval
2020-02-13 10:46:49+00:00 2020-02-13 10:47:25+00:00 0:00:36 The container "simul [...] The container "simulator" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 30969
6348
Susanne Keller PredictorLast aido3-off-LF-sim-validation
step1-simulation error no MyEval
2020-02-13 10:45:46+00:00 2020-02-13 10:46:26+00:00 0:00:40 The container "simul [...] The container "simulator" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible