Job ID submission user user label challenge step status up to date evaluator date started date completed duration message 4096510776Fernanda Custodio Pereira do Carmo exercise_ros_template aido5-LF-sim-validationLFv-sim timeout no reg01-bf67cb02b1e0-12020-11-07 18:51:13+00:00 - - - - No reset possible 4095410789Melisande Teng exercise_ros_template aido5-LF-sim-validationLFv-sim success no reg01-bf67cb02b1e0-12020-11-07 18:39:15+00:00 2020-11-07 18:50:45+00:00 0:11:30 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 1.7020693338472206 survival_time_median 14.950000000000076 deviation-center-line_median 0.4214714578694647 in-drivable-lane_median 3.975000000000057
other stats agent_compute-ego_max 0.06073477586110433 agent_compute-ego_mean 0.046690170116591864 agent_compute-ego_median 0.049744491577148435 agent_compute-ego_min 0.026536921450966283 complete-iteration_max 0.3338574520746867 complete-iteration_mean 0.30850430336960577 complete-iteration_median 0.3064216848214467 complete-iteration_min 0.28731639176084284 deviation-center-line_max 0.5977579992174556 deviation-center-line_mean 0.4111260354388767 deviation-center-line_min 0.20380322679912183 deviation-heading_max 2.3273724500644093 deviation-heading_mean 1.571283131591544 deviation-heading_median 1.553198407500801 deviation-heading_min 0.8513632613001647 driven_any_max 2.509024753729485 driven_any_mean 2.3859901648238204 driven_any_median 2.4257069362557173 driven_any_min 2.183522033054362 driven_lanedir_consec_max 2.1086323747573346 driven_lanedir_consec_mean 1.5387575553669377 driven_lanedir_consec_min 0.6422591790159742 driven_lanedir_max 2.1086323747573346 driven_lanedir_mean 1.5387575553669377 driven_lanedir_median 1.7020693338472206 driven_lanedir_min 0.6422591790159742 get_duckie_state_max 4.772554364120751e-06 get_duckie_state_mean 4.626962176540442e-06 get_duckie_state_median 4.647572835286458e-06 get_duckie_state_min 4.440148671468099e-06 get_robot_state_max 0.0243457275524474 get_robot_state_mean 0.023899597075947545 get_robot_state_median 0.02395285924275716 get_robot_state_min 0.023346942265828452 get_state_dump_max 0.02013141632080078 get_state_dump_mean 0.01947345219160381 get_state_dump_median 0.01942063892096804 get_state_dump_min 0.018921114603678384 get_ui_image_max 0.05131657361984253 get_ui_image_mean 0.049832095430608375 get_ui_image_median 0.0494650199538783 get_ui_image_min 0.04908176819483439 in-drivable-lane_max 10.750000000000083 in-drivable-lane_mean 5.187500000000057 in-drivable-lane_min 2.050000000000029 per-episodes details {"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.400127618690754, "get_ui_image": 0.04981541048016464, "step_physics": 0.11030804065235876, "survival_time": 14.250000000000068, "driven_lanedir": 1.387241479877683, "get_state_dump": 0.019562738820126185, "sim_render-ego": 0.009934064798187791, "get_robot_state": 0.0243457275524474, "get_duckie_state": 4.772554364120751e-06, "in-drivable-lane": 5.700000000000081, "agent_compute-ego": 0.026536921450966283, "deviation-heading": 1.3713883632150594, "complete-iteration": 0.28731639176084284, "set_robot_commands": 0.007508082138864618, "deviation-center-line": 0.32036821684758326, "driven_lanedir_consec": 1.387241479877683, "sim_compute_sim_state": 0.032266129945453845, "sim_compute_performance-ego": 0.006820774914925559}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.451286253820681, "get_ui_image": 0.04908176819483439, "step_physics": 0.11160491784413656, "survival_time": 14.950000000000076, "driven_lanedir": 2.016897187816758, "get_state_dump": 0.019278539021809895, "sim_render-ego": 0.009797093073527018, "get_robot_state": 0.02432796398798625, "get_duckie_state": 4.6261151631673175e-06, "in-drivable-lane": 2.250000000000032, "agent_compute-ego": 0.040871898333231606, "deviation-heading": 2.3273724500644093, "complete-iteration": 0.2996267795562744, "set_robot_commands": 0.007475472291310628, "deviation-center-line": 0.5977579992174556, "driven_lanedir_consec": 2.016897187816758, "sim_compute_sim_state": 0.030521777470906578, "sim_compute_performance-ego": 0.006450462341308594}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.183522033054362, "get_ui_image": 0.04911462942759196, "step_physics": 0.10639311154683433, "survival_time": 14.950000000000076, "driven_lanedir": 0.6422591790159742, "get_state_dump": 0.02013141632080078, "sim_render-ego": 0.010060513019561767, "get_robot_state": 0.023577754497528077, "get_duckie_state": 4.669030507405599e-06, "in-drivable-lane": 10.750000000000083, "agent_compute-ego": 0.058617084821065264, "deviation-heading": 0.8513632613001647, "complete-iteration": 0.31321659008661906, "set_robot_commands": 0.007752034664154053, "deviation-center-line": 0.20380322679912183, "driven_lanedir_consec": 0.6422591790159742, "sim_compute_sim_state": 0.030809296766916912, "sim_compute_performance-ego": 0.00653515100479126}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.509024753729485, "get_ui_image": 0.05131657361984253, "step_physics": 0.11982774416605632, "survival_time": 14.950000000000076, "driven_lanedir": 2.1086323747573346, "get_state_dump": 0.018921114603678384, "sim_render-ego": 0.009851245880126954, "get_robot_state": 0.023346942265828452, "get_duckie_state": 4.440148671468099e-06, "in-drivable-lane": 2.050000000000029, "agent_compute-ego": 0.06073477586110433, "deviation-heading": 1.7350084517865425, "complete-iteration": 0.3338574520746867, "set_robot_commands": 0.007516514460245768, "deviation-center-line": 0.5225746988913461, "driven_lanedir_consec": 2.1086323747573346, "sim_compute_sim_state": 0.03545466661453247, "sim_compute_performance-ego": 0.006674896081288656}}set_robot_commands_max 0.007752034664154053 set_robot_commands_mean 0.007563025888643766 set_robot_commands_median 0.007512298299555193 set_robot_commands_min 0.007475472291310628 sim_compute_performance-ego_max 0.006820774914925559 sim_compute_performance-ego_mean 0.0066203210855785176 sim_compute_performance-ego_median 0.006605023543039958 sim_compute_performance-ego_min 0.006450462341308594 sim_compute_sim_state_max 0.03545466661453247 sim_compute_sim_state_mean 0.03226296769945245 sim_compute_sim_state_median 0.03153771335618538 sim_compute_sim_state_min 0.030521777470906578 sim_render-ego_max 0.010060513019561767 sim_render-ego_mean 0.009910729192850885 sim_render-ego_median 0.009892655339157373 sim_render-ego_min 0.009797093073527018 simulation-passed 1 step_physics_max 0.11982774416605632 step_physics_mean 0.11203345355234648 step_physics_median 0.11095647924824764 step_physics_min 0.10639311154683433 survival_time_max 14.950000000000076 survival_time_mean 14.775000000000071 survival_time_min 14.250000000000068
No reset possible 4093810800Ayman Shams exercise_ros_template aido5-LF-sim-validationLFv-sim success no reg01-bf67cb02b1e0-12020-11-07 18:27:41+00:00 2020-11-07 18:39:00+00:00 0:11:19 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 0.0 survival_time_median 14.950000000000076 deviation-center-line_median 0.3071094952840746 in-drivable-lane_median 0.0
other stats agent_compute-ego_max 0.028213049570719403 agent_compute-ego_mean 0.02572279413541158 agent_compute-ego_median 0.025116876363754273 agent_compute-ego_min 0.024444374243418377 complete-iteration_max 0.3026611622174581 complete-iteration_mean 0.2859811057647069 complete-iteration_median 0.28593683282534277 complete-iteration_min 0.269389595190684 deviation-center-line_max 0.6863069918481528 deviation-center-line_mean 0.35998819605511767 deviation-center-line_min 0.13942680180416853 deviation-heading_max 3.7258518640401217 deviation-heading_mean 1.6586762301304627 deviation-heading_median 1.2113904039371066 deviation-heading_min 0.486072248607516 driven_any_max 1.3686924727581474e-13 driven_any_mean 6.950391439969185e-14 driven_any_median 7.057320516147632e-14 driven_any_min 0.0 driven_lanedir_consec_max 1.7821788787841797e-05 driven_lanedir_consec_mean 4.455447196960449e-06 driven_lanedir_consec_min 0.0 driven_lanedir_max 1.7821788787841797e-05 driven_lanedir_mean 4.455447196960449e-06 driven_lanedir_median 0.0 driven_lanedir_min 0.0 get_duckie_state_max 3.961722056070964e-06 get_duckie_state_mean 3.686944643656413e-06 get_duckie_state_median 3.7062168121337895e-06 get_duckie_state_min 3.3736228942871095e-06 get_robot_state_max 0.025777681668599447 get_robot_state_mean 0.02374495844046275 get_robot_state_median 0.023367119232813517 get_robot_state_min 0.02246791362762451 get_state_dump_max 0.02124795118967692 get_state_dump_mean 0.02018159826596578 get_state_dump_median 0.020238529443740848 get_state_dump_min 0.019001382986704507 get_ui_image_max 0.05055868148803711 get_ui_image_mean 0.0491702530781428 get_ui_image_median 0.0495436163743337 get_ui_image_min 0.0470350980758667 in-drivable-lane_max 0.0 in-drivable-lane_mean 0.0 in-drivable-lane_min 0.0 per-episodes details {"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.328474932138796e-13, "get_ui_image": 0.05055868148803711, "step_physics": 0.11154403845469156, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.020828116734822592, "sim_render-ego": 0.010862181186676026, "get_robot_state": 0.025777681668599447, "get_duckie_state": 3.8226445515950525e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.028213049570719403, "deviation-heading": 1.6198720948235377, "complete-iteration": 0.29343674182891843, "set_robot_commands": 0.00834139347076416, "deviation-center-line": 0.20880585936578705, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.030621883869171144, "sim_compute_performance-ego": 0.006466658115386963}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 1.3686924727581474e-13, "get_ui_image": 0.050111234188079834, "step_physics": 0.11056153456370035, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.019001382986704507, "sim_render-ego": 0.009215250015258789, "get_robot_state": 0.02309253692626953, "get_duckie_state": 3.3736228942871095e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.024600716431935628, "deviation-heading": 3.7258518640401217, "complete-iteration": 0.27843692382176716, "set_robot_commands": 0.0074542935689290366, "deviation-center-line": 0.13942680180416853, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.028090522289276124, "sim_compute_performance-ego": 0.006094572544097901}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 0.0, "get_ui_image": 0.0470350980758667, "step_physics": 0.09974363088607788, "survival_time": 14.950000000000076, "driven_lanedir": 1.7821788787841797e-05, "get_state_dump": 0.02124795118967692, "sim_render-ego": 0.009008407592773438, "get_robot_state": 0.02246791362762451, "get_duckie_state": 3.961722056070964e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.02563303629557292, "deviation-heading": 0.8029087130506755, "complete-iteration": 0.269389595190684, "set_robot_commands": 0.006859365304311116, "deviation-center-line": 0.4054131312023622, "driven_lanedir_consec": 1.7821788787841797e-05, "sim_compute_sim_state": 0.030760353406270344, "sim_compute_performance-ego": 0.006393109957377116}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 8.298917109073045e-15, "get_ui_image": 0.04897599856058756, "step_physics": 0.13015229066212972, "survival_time": 14.950000000000076, "driven_lanedir": 0.0, "get_state_dump": 0.0196489421526591, "sim_render-ego": 0.009323709805806478, "get_robot_state": 0.023641701539357504, "get_duckie_state": 3.589789072672526e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.024444374243418377, "deviation-heading": 0.486072248607516, "complete-iteration": 0.3026611622174581, "set_robot_commands": 0.0073120021820068356, "deviation-center-line": 0.6863069918481528, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.032640419006347655, "sim_compute_performance-ego": 0.006298956871032715}}set_robot_commands_max 0.00834139347076416 set_robot_commands_mean 0.007491763631502787 set_robot_commands_median 0.007383147875467936 set_robot_commands_min 0.006859365304311116 sim_compute_performance-ego_max 0.006466658115386963 sim_compute_performance-ego_mean 0.006313324371973674 sim_compute_performance-ego_median 0.006346033414204916 sim_compute_performance-ego_min 0.006094572544097901 sim_compute_sim_state_max 0.032640419006347655 sim_compute_sim_state_mean 0.030528294642766317 sim_compute_sim_state_median 0.030691118637720744 sim_compute_sim_state_min 0.028090522289276124 sim_render-ego_max 0.010862181186676026 sim_render-ego_mean 0.009602387150128682 sim_render-ego_median 0.009269479910532633 sim_render-ego_min 0.009008407592773438 simulation-passed 1 step_physics_max 0.13015229066212972 step_physics_mean 0.11300037364164987 step_physics_median 0.11105278650919596 step_physics_min 0.09974363088607788 survival_time_max 14.950000000000076 survival_time_mean 14.950000000000076 survival_time_min 14.950000000000076
No reset possible 4092610807Fernanda Custodio Pereira do Carmo exercise_ros_template aido5-LF-sim-validationLFv-sim success no reg01-bf67cb02b1e0-12020-11-07 18:16:02+00:00 2020-11-07 18:27:13+00:00 0:11:11 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 2.3213688932177527 survival_time_median 14.950000000000076 deviation-center-line_median 0.6640571995201576 in-drivable-lane_median 1.7500000000000249
other stats agent_compute-ego_max 0.026449485619862875 agent_compute-ego_mean 0.02595819572607676 agent_compute-ego_median 0.0261172616481781 agent_compute-ego_min 0.025148773988087972 complete-iteration_max 0.291322918732961 complete-iteration_mean 0.2864209699630737 complete-iteration_median 0.28568818728129064 complete-iteration_min 0.2829845865567525 deviation-center-line_max 0.8517578657467176 deviation-center-line_mean 0.6626372442025897 deviation-center-line_min 0.4706767120233263 deviation-heading_max 2.122599892238641 deviation-heading_mean 1.1770030179969249 deviation-heading_median 1.0043036893633157 deviation-heading_min 0.5768048010224267 driven_any_max 2.65473329633004 driven_any_mean 2.6525569351778877 driven_any_median 2.6547115631418903 driven_any_min 2.646071318097729 driven_lanedir_consec_max 2.642813563877903 driven_lanedir_consec_mean 2.20848452044875 driven_lanedir_consec_min 1.548386731481592 driven_lanedir_max 2.642813563877903 driven_lanedir_mean 2.20848452044875 driven_lanedir_median 2.3213688932177527 driven_lanedir_min 1.548386731481592 get_duckie_state_max 3.892580668131511e-06 get_duckie_state_mean 3.7626425425211583e-06 get_duckie_state_median 3.763039906819661e-06 get_duckie_state_min 3.631909688313802e-06 get_robot_state_max 0.024436184565226237 get_robot_state_mean 0.023827934463818867 get_robot_state_median 0.02375885486602783 get_robot_state_min 0.02335784355799357 get_state_dump_max 0.019823622703552247 get_state_dump_mean 0.019483696222305297 get_state_dump_median 0.01943528930346171 get_state_dump_min 0.01924058357874553 get_ui_image_max 0.05022683143615723 get_ui_image_mean 0.04872335731983185 get_ui_image_median 0.04861858447392782 get_ui_image_min 0.047429428895314533 in-drivable-lane_max 5.600000000000056 in-drivable-lane_mean 2.275000000000026 in-drivable-lane_min 0.0 per-episodes details {"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.646071318097729, "get_ui_image": 0.04847506284713745, "step_physics": 0.11023833831151326, "survival_time": 14.950000000000076, "driven_lanedir": 2.632311477605001, "get_state_dump": 0.01935118516286214, "sim_render-ego": 0.00945765733718872, "get_robot_state": 0.023849207560221353, "get_duckie_state": 3.892580668131511e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.02602512836456299, "deviation-heading": 0.8926991618372412, "complete-iteration": 0.283099881807963, "set_robot_commands": 0.00748215119043986, "deviation-center-line": 0.4706767120233263, "driven_lanedir_consec": 2.632311477605001, "sim_compute_sim_state": 0.03173143227895101, "sim_compute_performance-ego": 0.00625587542851766}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.6547167281041704, "get_ui_image": 0.04876210610071818, "step_physics": 0.1126335334777832, "survival_time": 14.950000000000076, "driven_lanedir": 2.642813563877903, "get_state_dump": 0.01951939344406128, "sim_render-ego": 0.009187094370524089, "get_robot_state": 0.023668502171834308, "get_duckie_state": 3.883043924967448e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.025148773988087972, "deviation-heading": 1.11590821688939, "complete-iteration": 0.2829845865567525, "set_robot_commands": 0.00712014118830363, "deviation-center-line": 0.533541278141163, "driven_lanedir_consec": 2.642813563877903, "sim_compute_sim_state": 0.030444605350494383, "sim_compute_performance-ego": 0.006274937788645426}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.65470639817961, "get_ui_image": 0.05022683143615723, "step_physics": 0.1151404062906901, "survival_time": 14.950000000000076, "driven_lanedir": 1.548386731481592, "get_state_dump": 0.01924058357874553, "sim_render-ego": 0.009296123186747234, "get_robot_state": 0.02335784355799357, "get_duckie_state": 3.643035888671875e-06, "in-drivable-lane": 5.600000000000056, "agent_compute-ego": 0.026209394931793212, "deviation-heading": 2.122599892238641, "complete-iteration": 0.2882764927546183, "set_robot_commands": 0.00715723196665446, "deviation-center-line": 0.7945731208991521, "driven_lanedir_consec": 1.548386731481592, "sim_compute_sim_state": 0.03132377465565999, "sim_compute_performance-ego": 0.006094442208607991}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.65473329633004, "get_ui_image": 0.047429428895314533, "step_physics": 0.11253474394480388, "survival_time": 14.950000000000076, "driven_lanedir": 2.010426308830505, "get_state_dump": 0.019823622703552247, "sim_render-ego": 0.009765415191650391, "get_robot_state": 0.024436184565226237, "get_duckie_state": 3.631909688313802e-06, "in-drivable-lane": 3.5000000000000497, "agent_compute-ego": 0.026449485619862875, "deviation-heading": 0.5768048010224267, "complete-iteration": 0.291322918732961, "set_robot_commands": 0.00800888458887736, "deviation-center-line": 0.8517578657467176, "driven_lanedir_consec": 2.010426308830505, "sim_compute_sim_state": 0.03597175041834513, "sim_compute_performance-ego": 0.006660452683766683}}set_robot_commands_max 0.00800888458887736 set_robot_commands_mean 0.007442102233568827 set_robot_commands_median 0.00731969157854716 set_robot_commands_min 0.00712014118830363 sim_compute_performance-ego_max 0.006660452683766683 sim_compute_performance-ego_mean 0.00632142702738444 sim_compute_performance-ego_median 0.006265406608581543 sim_compute_performance-ego_min 0.006094442208607991 sim_compute_sim_state_max 0.03597175041834513 sim_compute_sim_state_mean 0.032367890675862626 sim_compute_sim_state_median 0.0315276034673055 sim_compute_sim_state_min 0.030444605350494383 sim_render-ego_max 0.009765415191650391 sim_render-ego_mean 0.009426572521527609 sim_render-ego_median 0.009376890261967976 sim_render-ego_min 0.009187094370524089 simulation-passed 1 step_physics_max 0.1151404062906901 step_physics_mean 0.11263675550619762 step_physics_median 0.11258413871129357 step_physics_min 0.11023833831151326 survival_time_max 14.950000000000076 survival_time_mean 14.950000000000076 survival_time_min 14.950000000000076
No reset possible 4091710812Ayman Shams exercise_ros_template aido5-LF-sim-validationLFv-sim failed no reg01-bf67cb02b1e0-12020-11-07 18:12:01+00:00 2020-11-07 18:15:41+00:00 0:03:40 InvalidSubmission:
T [...] InvalidSubmission:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
others = read_until_over(fpout, timeout=timeout, nickname=nickname)
File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "experiment_manager.py", line 540, in run_episode
r: MsgReceived = await loop.run_in_executor(executor, f)
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 65 messages. Expected topic "commands".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
yield cie
File "experiment_manager.py", line 683, in go
wrap(cie)
File "experiment_manager.py", line 668, in wrap
asyncio.run(main(cie, logdir, attempts), debug=True)
File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "experiment_manager.py", line 281, in main
length_s = await run_episode(
File "experiment_manager.py", line 544, in run_episode
raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
Fernanda Custodio Pereira do Carmo 🇨🇦
Melisande Teng
Ayman Shams 🇨🇦