Job ID submission user user label challenge step status up to date evaluator date started date completed duration message 246533246Brian Chuang PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 14:00:07+00:00 2019-10-31 14:00:32+00:00 0:00:25 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.08959391735214417 error_L2 0.012643451947346562
No reset possible 246493265Liu Sam challenge-aido_LF-template-random aido2-PREDstep1-simulation host-error yes idsc-rudolf-88682019-10-31 13:59:48+00:00 2019-10-31 13:59:58+00:00 0:00:10 Error while running [...]
stderr | Pulling solution ...
stderr | Pulling evaluation ... done
stderr |
stderr | ERROR: for solution pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 246363276Timothius Victorio Yasin challenge-aido_LF-template-random aido2-PREDstep1-simulation error yes idsc-rudolf-88682019-10-31 13:59:07+00:00 2019-10-31 13:59:32+00:00 0:00:25 Unexpected exception [...] Unexpected exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/zuper_nodes/compatibility.py", line 25, in check_compatible_protocol
raise IncompatibleProtocol(msg)
zuper_nodes.compatibility.IncompatibleProtocol: For input "observations", cannot use type <class 'aido_schemas.schemas.Duckiebot1Observations'> as <class 'float'>: CanBeUsed(result=False, why="Type <class 'aido_schemas.schemas.Duckiebot1Observations'> (93990019201488) \n is not a subclass of <class 'float'> (140484963475584) \n is : False", M=Matches(m={}), matches={}, reasons={})
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
yield cie
File "experiment_manager.py", line 77, in <module>
main(cie)
File "experiment_manager.py", line 26, in main
agent_ci._get_node_protocol()
File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 88, in _get_node_protocol
check_compatible_protocol(self.node_protocol, self.expect_protocol)
File "/usr/local/lib/python3.7/site-packages/zuper_nodes/compatibility.py", line 43, in check_compatible_protocol
raise IncompatibleProtocol(msg) from e
zuper_nodes.compatibility.IncompatibleProtocol: Cannot say that p1 is a sub-protocol of p2
p1: |InteractionProtocol(description='Particularization for Duckiebot1 observations and commands.', inputs={'observations': <class 'aido_schemas.schemas.Duckiebot1Observations'>, 'seed': <class 'int'>, 'get_commands': <class 'NoneType'>, 'episode_start': <class 'aido_schemas.protocol_agent.EpisodeStart'>}, outputs={'commands': <class 'aido_schemas.schemas.Duckiebot1Commands'>}, language='(in:seed)? ; ((in:episode_start ; ((in:observations | (in:get_commands ; out:commands))*))*)')
p2: |InteractionProtocol(description='An estimator receives a stream of values and must predict the next value.', inputs={'observations': <class 'float'>, 'seed': <class 'int'>, 'get_prediction': <class 'NoneType'>}, outputs={'prediction': <class 'float'>}, language='(in:seed)? ; ((in:observations | (in:get_prediction ; out:prediction))*)')
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 246253815Philippe Laferriere PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:58:23+00:00 2019-10-31 13:58:52+00:00 0:00:29 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.11525320003501364 error_L2 0.020745912403286857
No reset possible 246183819Philippe Laferriere PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:57:34+00:00 2019-10-31 13:58:04+00:00 0:00:30 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.11425602180091862 error_L2 0.020434236588160706
No reset possible 246094030Frank (Chude) Qian PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:56:48+00:00 2019-10-31 13:57:13+00:00 0:00:25 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.10698297671072496 error_L2 0.019062983771216666
No reset possible 246004055Zoltán L?rincz PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:56:08+00:00 2019-10-31 13:56:33+00:00 0:00:25 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.12171806319625152 error_L2 0.022636627663454797
No reset possible 245924066Rekha Sundararajan PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:55:22+00:00 2019-10-31 13:55:48+00:00 0:00:26 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.12130616183557652 error_L2 0.02239975289033344
No reset possible 245844072Linus Lingg PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:54:50+00:00 2019-10-31 13:55:15+00:00 0:00:25 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.1189262781497887 error_L2 0.020583461569340895
No reset possible 245794086Christian Leopoldseder PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:53:52+00:00 2019-10-31 13:54:18+00:00 0:00:26 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.11296878879325428 error_L2 0.020068097363494845
No reset possible 245724087Carmen Scheidemann PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:53:19+00:00 2019-10-31 13:53:46+00:00 0:00:27 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.11541410310314054 error_L2 0.020946921192807527
No reset possible 245684099Fidel Esquivel Estay PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:52:35+00:00 2019-10-31 13:52:59+00:00 0:00:24 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.10922076093461262 error_L2 0.01870186310635291
No reset possible 245554105Etienne Walther PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:52:02+00:00 2019-10-31 13:52:26+00:00 0:00:24 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.10577167518576884 error_L2 0.017269536598197723
No reset possible 245324131Wencan Huang PredictorLast aido2-PREDstep1-simulation success yes idsc-rudolf-88682019-10-31 13:50:13+00:00 2019-10-31 13:51:30+00:00 0:01:17 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L1 0.10830785620056375 error_L2 0.01831790890783593
No reset possible 245194282Liam Paull challenge-aido_LF-template-random aido3-LF-sim-validationstep1-simulation success no idsc-rudolf-88682019-10-31 12:29:46+00:00 2019-10-31 12:33:21+00:00 0:03:35 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 0.5723310224856362 survival_time_median 2.549999999999999 deviation-center-line_median 0.13005513204193506 in-drivable-lane_median 0.29999999999999893
other stats agent_compute-ego_max 0.02555534418891458 agent_compute-ego_mean 0.024762181125819818 agent_compute-ego_median 0.02517929560021509 agent_compute-ego_min 0.023322860399882 deviation-center-line_max 0.2072381278624381 deviation-center-line_mean 0.13189395308989765 deviation-center-line_min 0.0788839760023766 deviation-heading_max 0.5289022616823588 deviation-heading_mean 0.3206717506075888 deviation-heading_median 0.23606825392187256 deviation-heading_min 0.19401296908406307 driven_any_max 1.6182144577009443 driven_any_mean 0.861378585933269 driven_any_median 0.9933867744747804 driven_any_min 0.23626560261560617 driven_lanedir_consec_max 0.8010406009604816 driven_lanedir_consec_mean 0.5597645764431625 driven_lanedir_consec_min 0.2264965947033466 driven_lanedir_max 0.8010406009604816 driven_lanedir_mean 0.5597645764431625 driven_lanedir_median 0.5723310224856362 driven_lanedir_min 0.2264965947033466 in-drivable-lane_max 1.7999999999999936 in-drivable-lane_mean 0.619999999999998 in-drivable-lane_min 0 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429704, "sim_physics": 0.05871741860001176, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.01136347982618544, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.023322860399882, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.0071008337868584525, "deviation-center-line": 0.15907928712407876, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.004614194234212239, "sim_compute_performance-ego": 0.00675921086911802, "sim_compute_robot_state-ego": 0.007805290045561614}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.06459218847985361, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.012502805859434838, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.02452388464235792, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.00784607494578642, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.0051582934809666055, "sim_compute_performance-ego": 0.007469672782748353, "sim_compute_robot_state-ego": 0.00916889134575339}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.07096440651837517, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.013677344602697036, "in-drivable-lane": 0, "agent_compute-ego": 0.02555534418891458, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.008701661053825827, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.005544942968031939, "sim_compute_performance-ego": 0.007825977662030388, "sim_compute_robot_state-ego": 0.0095216807197122}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.06652591705322265, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.01278400421142578, "in-drivable-lane": 0, "agent_compute-ego": 0.025229520797729492, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.008015327453613282, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.005195703506469727, "sim_compute_performance-ego": 0.007489471435546875, "sim_compute_robot_state-ego": 0.009023494720458984}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.6182144577009443, "sim_physics": 0.06388512744179255, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.012407417538799818, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.02517929560021509, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.00812130336520038, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.005225151400022869, "sim_compute_performance-ego": 0.007401297364053847, "sim_compute_robot_state-ego": 0.00889188428468342}}set_robot_commands_max 0.008701661053825827 set_robot_commands_mean 0.007957040121056871 set_robot_commands_median 0.008015327453613282 set_robot_commands_min 0.0071008337868584525 sim_compute_performance-ego_max 0.007825977662030388 sim_compute_performance-ego_mean 0.007389126022699496 sim_compute_performance-ego_median 0.007469672782748353 sim_compute_performance-ego_min 0.00675921086911802 sim_compute_robot_state-ego_max 0.0095216807197122 sim_compute_robot_state-ego_mean 0.008882248223233922 sim_compute_robot_state-ego_median 0.009023494720458984 sim_compute_robot_state-ego_min 0.007805290045561614 sim_compute_sim_state_max 0.005544942968031939 sim_compute_sim_state_mean 0.005147657117940676 sim_compute_sim_state_median 0.005195703506469727 sim_compute_sim_state_min 0.004614194234212239 sim_physics_max 0.07096440651837517 sim_physics_mean 0.06493701161865115 sim_physics_median 0.06459218847985361 sim_physics_min 0.05871741860001176 sim_render-ego_max 0.013677344602697036 sim_render-ego_mean 0.01254701040770858 sim_render-ego_median 0.012502805859434838 sim_render-ego_min 0.01136347982618544 simulation-passed 1 survival_time_max 3.949999999999994 survival_time_mean 2.2599999999999985 survival_time_min 0.8500000000000002
No reset possible 244894262Gianmarco Bernasconi challenge-aido_LF-baseline-duckietown aido3-LF-sim-validationstep1-simulation success no idsc-rudolf-88682019-10-29 17:53:47+00:00 2019-10-29 18:01:14+00:00 0:07:27 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 1.133720595485579 survival_time_median 14.950000000000076 deviation-center-line_median 0.6913271365925806 in-drivable-lane_median 3.650000000000052
other stats agent_compute-ego_max 0.027313378651936848 agent_compute-ego_mean 0.02530402354654515 agent_compute-ego_median 0.025815062522888184 agent_compute-ego_min 0.022621756646691298 deviation-center-line_max 0.9914697454209812 deviation-center-line_mean 0.6047225840119916 deviation-center-line_min 0.19115964802579016 deviation-heading_max 5.475613210104982 deviation-heading_mean 3.863977969914535 deviation-heading_median 4.623776004655106 deviation-heading_min 1.673030937175625 driven_any_max 2.3488313227650077 driven_any_mean 1.6754748725137052 driven_any_median 2.3486129446216193 driven_any_min 0.5976044718925265 driven_lanedir_consec_max 1.3628180818368838 driven_lanedir_consec_mean 0.858161561780236 driven_lanedir_consec_min 0.12569821439782658 driven_lanedir_max 1.515424349601733 driven_lanedir_mean 1.002443210854517 driven_lanedir_median 1.2162232033693468 driven_lanedir_min 0.3833182606287888 in-drivable-lane_max 5.650000000000052 in-drivable-lane_mean 2.9100000000000237 in-drivable-lane_min 0.29999999999999893 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.5976044718925265, "sim_physics": 0.054187123368426064, "survival_time": 4.099999999999993, "driven_lanedir": 0.45234771381154326, "sim_render-ego": 0.0100187179518909, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.022621756646691298, "deviation-heading": 1.673030937175625, "set_robot_commands": 0.006732696440161728, "deviation-center-line": 0.19115964802579016, "driven_lanedir_consec": 0.45234771381154326, "sim_compute_sim_state": 0.004272594684507788, "sim_compute_performance-ego": 0.005983806237941835, "sim_compute_robot_state-ego": 0.007017403114132765}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.7336670566893181, "sim_physics": 0.05644861046148806, "survival_time": 4.899999999999991, "driven_lanedir": 0.3833182606287888, "sim_render-ego": 0.01140565045025884, "in-drivable-lane": 1.1499999999999984, "agent_compute-ego": 0.02436039642411835, "deviation-heading": 2.496396568770012, "set_robot_commands": 0.007252732101751834, "deviation-center-line": 0.1966939567463896, "driven_lanedir_consec": 0.12569821439782658, "sim_compute_sim_state": 0.004673490718919404, "sim_compute_performance-ego": 0.00658687280148876, "sim_compute_robot_state-ego": 0.00803424630846296}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.3488313227650077, "sim_physics": 0.05866737604141235, "survival_time": 14.950000000000076, "driven_lanedir": 1.2162232033693468, "sim_render-ego": 0.011288337707519532, "in-drivable-lane": 5.650000000000052, "agent_compute-ego": 0.026409523487091063, "deviation-heading": 4.623776004655106, "set_robot_commands": 0.00801376740137736, "deviation-center-line": 0.9529624332742168, "driven_lanedir_consec": 1.2162232033693468, "sim_compute_sim_state": 0.005008149941762289, "sim_compute_performance-ego": 0.006666404406229655, "sim_compute_robot_state-ego": 0.00818397839864095}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.3486129446216193, "sim_physics": 0.05865092754364014, "survival_time": 14.950000000000076, "driven_lanedir": 1.4449025268611733, "sim_render-ego": 0.011494139035542804, "in-drivable-lane": 3.8000000000000176, "agent_compute-ego": 0.027313378651936848, "deviation-heading": 5.475613210104982, "set_robot_commands": 0.008018902937571208, "deviation-center-line": 0.9914697454209812, "driven_lanedir_consec": 1.133720595485579, "sim_compute_sim_state": 0.004960605303446452, "sim_compute_performance-ego": 0.006707332928975423, "sim_compute_robot_state-ego": 0.008233056863149008}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.3486585666000543, "sim_physics": 0.05772132953008016, "survival_time": 14.950000000000076, "driven_lanedir": 1.515424349601733, "sim_render-ego": 0.011042314370473226, "in-drivable-lane": 3.650000000000052, "agent_compute-ego": 0.025815062522888184, "deviation-heading": 5.051073128866949, "set_robot_commands": 0.00789182742436727, "deviation-center-line": 0.6913271365925806, "driven_lanedir_consec": 1.3628180818368838, "sim_compute_sim_state": 0.004909818172454834, "sim_compute_performance-ego": 0.006498893896738688, "sim_compute_robot_state-ego": 0.008019154866536458}}set_robot_commands_max 0.008018902937571208 set_robot_commands_mean 0.00758198526104588 set_robot_commands_median 0.00789182742436727 set_robot_commands_min 0.006732696440161728 sim_compute_performance-ego_max 0.006707332928975423 sim_compute_performance-ego_mean 0.006488662054274873 sim_compute_performance-ego_median 0.00658687280148876 sim_compute_performance-ego_min 0.005983806237941835 sim_compute_robot_state-ego_max 0.008233056863149008 sim_compute_robot_state-ego_mean 0.007897567910184427 sim_compute_robot_state-ego_median 0.00803424630846296 sim_compute_robot_state-ego_min 0.007017403114132765 sim_compute_sim_state_max 0.005008149941762289 sim_compute_sim_state_mean 0.004764931764218153 sim_compute_sim_state_median 0.004909818172454834 sim_compute_sim_state_min 0.004272594684507788 sim_physics_max 0.05866737604141235 sim_physics_mean 0.05713507338900936 sim_physics_median 0.05772132953008016 sim_physics_min 0.054187123368426064 sim_render-ego_max 0.011494139035542804 sim_render-ego_mean 0.01104983190313706 sim_render-ego_median 0.011288337707519532 sim_render-ego_min 0.0100187179518909 simulation-passed 1 survival_time_max 14.950000000000076 survival_time_mean 10.770000000000042 survival_time_min 4.099999999999993
No reset possible 244864258Anastasiya Nikolskaya beta aido3-LF-sim-testingstep1-simulation success no idsc-rudolf-88682019-10-28 12:15:19+00:00 2019-10-28 12:39:03+00:00 0:23:44 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 0.6231063017121015 survival_time_median 9.850000000000003 deviation-center-line_median 0.5072026204557948 in-drivable-lane_median 1.5000000000000009
other stats agent_compute-ego_max 0.10037231595261292 agent_compute-ego_mean 0.09840961163766145 agent_compute-ego_median 0.09854400944588754 agent_compute-ego_min 0.09657644581150364 deviation-center-line_max 0.9578765081453648 deviation-center-line_mean 0.575878653093167 deviation-center-line_min 0.28686246510843705 deviation-heading_max 3.7760720187631818 deviation-heading_mean 1.720478571957843 deviation-heading_median 1.5428323362316263 deviation-heading_min 0.5970335207486406 driven_any_max 1.786742746047968 driven_any_mean 1.2421571676351515 driven_any_median 1.1754244350237415 driven_any_min 0.5979840834149236 driven_lanedir_consec_max 1.6008525711555903 driven_lanedir_consec_mean 0.920432494652434 driven_lanedir_consec_min 0.4801095388439971 driven_lanedir_max 1.6008525711555903 driven_lanedir_mean 0.9679680290907052 driven_lanedir_median 0.8519036796649044 driven_lanedir_min 0.4801095388439971 in-drivable-lane_max 8.500000000000092 in-drivable-lane_mean 1.8233333333333444 in-drivable-lane_min 0 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0074030026974847, "sim_physics": 0.0679231272502379, "survival_time": 8.79999999999999, "driven_lanedir": 0.6183360637706854, "sim_render-ego": 0.01343160325830633, "in-drivable-lane": 2.400000000000005, "agent_compute-ego": 0.09917052090168, "deviation-heading": 2.328005935445923, "set_robot_commands": 0.009789916602048006, "deviation-center-line": 0.4432226632011767, "driven_lanedir_consec": 0.5950516713354923, "sim_compute_sim_state": 0.005823480811986056, "sim_compute_performance-ego": 0.008143765005198393, "sim_compute_robot_state-ego": 0.00962264971299605}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.5884011922291097, "sim_physics": 0.06717128679156303, "survival_time": 12.800000000000049, "driven_lanedir": 1.5630714026976635, "sim_render-ego": 0.013452442362904549, "in-drivable-lane": 0, "agent_compute-ego": 0.09865349251776934, "deviation-heading": 1.8614000628956795, "set_robot_commands": 0.00990860816091299, "deviation-center-line": 0.8313531302733334, "driven_lanedir_consec": 1.5630714026976635, "sim_compute_sim_state": 0.0057664960622787476, "sim_compute_performance-ego": 0.00790213793516159, "sim_compute_robot_state-ego": 0.00969110056757927}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.0314135450556898, "sim_physics": 0.06687618512187513, "survival_time": 9.850000000000003, "driven_lanedir": 0.6104587201124119, "sim_render-ego": 0.013412837449669233, "in-drivable-lane": 3.35000000000002, "agent_compute-ego": 0.09854400944588754, "deviation-heading": 1.5428323362316263, "set_robot_commands": 0.010011626984262225, "deviation-center-line": 0.5946060698090587, "driven_lanedir_consec": 0.6104587201124119, "sim_compute_sim_state": 0.0058896904669437315, "sim_compute_performance-ego": 0.007667350284944331, "sim_compute_robot_state-ego": 0.00943237149775936}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.868131193187374, "sim_physics": 0.06779668975050432, "survival_time": 6.849999999999984, "driven_lanedir": 0.8519036796649044, "sim_render-ego": 0.013502787499532211, "in-drivable-lane": 0, "agent_compute-ego": 0.09721822808258725, "deviation-heading": 1.1185954678480718, "set_robot_commands": 0.00985897196470386, "deviation-center-line": 0.6208630020073503, "driven_lanedir_consec": 0.8519036796649044, "sim_compute_sim_state": 0.005825792785978666, "sim_compute_performance-ego": 0.0077180409953542, "sim_compute_robot_state-ego": 0.009468381422279524}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.648801876255428, "sim_physics": 0.06507125337615267, "survival_time": 13.100000000000051, "driven_lanedir": 1.171671428784811, "sim_render-ego": 0.013423208061975375, "in-drivable-lane": 3.900000000000055, "agent_compute-ego": 0.09809881494245457, "deviation-heading": 0.962171218183121, "set_robot_commands": 0.009923452639397774, "deviation-center-line": 0.4838797126967088, "driven_lanedir_consec": 1.171671428784811, "sim_compute_sim_state": 0.00585851960509788, "sim_compute_performance-ego": 0.007749217157145493, "sim_compute_robot_state-ego": 0.009484471255586348}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.4920704557132538, "sim_physics": 0.0659600814970413, "survival_time": 10.100000000000009, "driven_lanedir": 1.2352099170234556, "sim_render-ego": 0.013505549714116766, "in-drivable-lane": 1.699999999999994, "agent_compute-ego": 0.09876712596062388, "deviation-heading": 1.5370567492611846, "set_robot_commands": 0.010045581524915036, "deviation-center-line": 0.9578765081453648, "driven_lanedir_consec": 1.2352099170234556, "sim_compute_sim_state": 0.005806826128818021, "sim_compute_performance-ego": 0.007846709525231088, "sim_compute_robot_state-ego": 0.00954938525020486}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.1754244350237415, "sim_physics": 0.06610316456975164, "survival_time": 9.249999999999996, "driven_lanedir": 0.9272292894059192, "sim_render-ego": 0.013578061799745302, "in-drivable-lane": 1.5000000000000009, "agent_compute-ego": 0.09657644581150364, "deviation-heading": 2.812909579206577, "set_robot_commands": 0.009933114696193386, "deviation-center-line": 0.43472302127793233, "driven_lanedir_consec": 0.6231063017121015, "sim_compute_sim_state": 0.005898019429799672, "sim_compute_performance-ego": 0.00781836767454405, "sim_compute_robot_state-ego": 0.009589277731405724}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.5979840834149236, "sim_physics": 0.06594984872000557, "survival_time": 4.199999999999993, "driven_lanedir": 0.5904408959522636, "sim_render-ego": 0.013429661591847738, "in-drivable-lane": 0, "agent_compute-ego": 0.09916854756219048, "deviation-heading": 0.5970335207486406, "set_robot_commands": 0.010045605046408517, "deviation-center-line": 0.4782144833183858, "driven_lanedir_consec": 0.5904408959522636, "sim_compute_sim_state": 0.005892617361886161, "sim_compute_performance-ego": 0.007722252891177223, "sim_compute_robot_state-ego": 0.00945244516645159}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.60411525015778, "sim_physics": 0.0658280889193217, "survival_time": 14.950000000000076, "driven_lanedir": 1.6008525711555903, "sim_render-ego": 0.013981095949808757, "in-drivable-lane": 0, "agent_compute-ego": 0.09771855115890504, "deviation-heading": 0.7360735418496934, "set_robot_commands": 0.010029582977294922, "deviation-center-line": 0.8449195851657476, "driven_lanedir_consec": 1.6008525711555903, "sim_compute_sim_state": 0.005885067780812581, "sim_compute_performance-ego": 0.007813436190287272, "sim_compute_robot_state-ego": 0.009501206080118814}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.9615943013672088, "sim_physics": 0.06959590072152, "survival_time": 7.94999999999998, "driven_lanedir": 0.4801095388439971, "sim_render-ego": 0.01411240355773542, "in-drivable-lane": 2.9999999999999893, "agent_compute-ego": 0.10037231595261292, "deviation-heading": 1.2323608451719148, "set_robot_commands": 0.009953122468864394, "deviation-center-line": 0.28686246510843705, "driven_lanedir_consec": 0.4801095388439971, "sim_compute_sim_state": 0.005901549597206355, "sim_compute_performance-ego": 0.008121369019994195, "sim_compute_robot_state-ego": 0.00995050736193387}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.786742746047968, "sim_physics": 0.0646817175547282, "survival_time": 14.950000000000076, "driven_lanedir": 1.5664221051930738, "sim_render-ego": 0.013417781988779704, "in-drivable-lane": 0.70000000000001, "agent_compute-ego": 0.0974856440226237, "deviation-heading": 3.7760720187631818, "set_robot_commands": 0.009883310794830322, "deviation-center-line": 0.5789967300749801, "driven_lanedir_consec": 1.2170039224332287, "sim_compute_sim_state": 0.005759592851003011, "sim_compute_performance-ego": 0.007704372406005859, "sim_compute_robot_state-ego": 0.009433097044626871}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.0809058723560914, "sim_physics": 0.06552664344832743, "survival_time": 8.449999999999985, "driven_lanedir": 0.6414261157830948, "sim_render-ego": 0.013569431192070774, "in-drivable-lane": 2.299999999999999, "agent_compute-ego": 0.09889537907211032, "deviation-heading": 2.971993634208915, "set_robot_commands": 0.009904644193028556, "deviation-center-line": 0.47054587208246945, "driven_lanedir_consec": 0.6052186620978813, "sim_compute_sim_state": 0.00578444808192507, "sim_compute_performance-ego": 0.007818340549807576, "sim_compute_robot_state-ego": 0.009568015499227853}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.5405443339498766, "sim_physics": 0.06453356027603149, "survival_time": 14.950000000000076, "driven_lanedir": 1.5255717363280383, "sim_render-ego": 0.013353127638498942, "in-drivable-lane": 0, "agent_compute-ego": 0.09841353416442872, "deviation-heading": 1.7084926020774212, "set_robot_commands": 0.009883206685384114, "deviation-center-line": 0.5072026204557948, "driven_lanedir_consec": 1.5255717363280383, "sim_compute_sim_state": 0.005772269566853841, "sim_compute_performance-ego": 0.007729671796162923, "sim_compute_robot_state-ego": 0.009405972162882487}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.6324477480941404, "sim_physics": 0.06570566054618004, "survival_time": 5.04999999999999, "driven_lanedir": 0.6207149833727801, "sim_render-ego": 0.013491521967519628, "in-drivable-lane": 0, "agent_compute-ego": 0.09734010224295137, "deviation-heading": 0.8575535240499732, "set_robot_commands": 0.010040009375846032, "deviation-center-line": 0.4588856824310454, "driven_lanedir_consec": 0.6207149833727801, "sim_compute_sim_state": 0.005713200805210831, "sim_compute_performance-ego": 0.007636551809783029, "sim_compute_robot_state-ego": 0.009352282722397603}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 1.616377478977198, "sim_physics": 0.06500123659769694, "survival_time": 14.950000000000076, "driven_lanedir": 0.5161019882718927, "sim_render-ego": 0.01333592176437378, "in-drivable-lane": 8.500000000000092, "agent_compute-ego": 0.09972146272659302, "deviation-heading": 1.7646275434257197, "set_robot_commands": 0.009996163050333658, "deviation-center-line": 0.6460282503497207, "driven_lanedir_consec": 0.5161019882718927, "sim_compute_sim_state": 0.005770269234975179, "sim_compute_performance-ego": 0.007692787647247314, "sim_compute_robot_state-ego": 0.009421974817911785}}set_robot_commands_max 0.010045605046408517 set_robot_commands_mean 0.009947127810961586 set_robot_commands_median 0.009933114696193386 set_robot_commands_min 0.009789916602048006 sim_compute_performance-ego_max 0.008143765005198393 sim_compute_performance-ego_mean 0.007805624725869636 sim_compute_performance-ego_median 0.007749217157145493 sim_compute_performance-ego_min 0.007636551809783029 sim_compute_robot_state-ego_max 0.00995050736193387 sim_compute_robot_state-ego_mean 0.009528209219557469 sim_compute_robot_state-ego_median 0.009484471255586348 sim_compute_robot_state-ego_min 0.009352282722397603 sim_compute_sim_state_max 0.005901549597206355 sim_compute_sim_state_mean 0.005823189371385053 sim_compute_sim_state_median 0.005823480811986056 sim_compute_sim_state_min 0.005713200805210831 sim_physics_max 0.06959590072152 sim_physics_mean 0.06624829634272916 sim_physics_median 0.06594984872000557 sim_physics_min 0.06453356027603149 sim_render-ego_max 0.01411240355773542 sim_render-ego_mean 0.013533162386458969 sim_render-ego_median 0.013452442362904549 sim_render-ego_min 0.01333592176437378 simulation-passed 1 survival_time_max 14.950000000000076 survival_time_mean 10.413333333333355 survival_time_min 4.199999999999993
No reset possible 244534247Manfred Diaz challenge-aido_LF-template-tensorflow aido3-LF-sim-validationstep1-simulation failed no idsc-rudolf-88682019-10-25 04:50:39+00:00 2019-10-25 04:53:03+00:00 0:02:24 InvalidSubmission:
T [...] InvalidSubmission:
Traceback (most recent call last):
File "experiment_manager.py", line 299, in run_episode
r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
nickname=self.nickname)
File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
others = read_until_over(fpout, timeout=timeout, nickname=nickname)
File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:
error in agent |Exception while handling a message on topic "get_commands".
|
|| Traceback (most recent call last):
|| File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
|| handle_message_node(parsed, receiver0, context0)
|| File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
|| call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
|| File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
|| f(**kwargs)
|| File "solution.py", line 42, in on_received_get_commands
|| pwm_commands = PWMCommands(motor_left=pwm_left, motor_right=pwm_right)
|| File "<string>", line 4, in __init__
|| File "/usr/local/lib/python3.7/dist-packages/aido_schemas/schemas.py", line 41, in __post_init__
|| raise ValueError(msg)
|| ValueError: Expected values to be between -1 and 1. Obtained 1.1147898435592651, -0.26043227314949036
||
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
yield cie
File "experiment_manager.py", line 444, in <module>
wrap(cie)
File "experiment_manager.py", line 432, in wrap
main(cie, logdir, attempts)
File "experiment_manager.py", line 136, in main
physics_dt=config.physics_dt)
File "experiment_manager.py", line 303, in run_episode
raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 244334231Andrea Censi challenge-aido_LF-template-pytorch aido3-LF-sim-validationstep1-simulation error no idsc-rudolf-88682019-10-24 17:13:56+00:00 2019-10-24 17:15:44+00:00 0:01:48 The container "solut [...] The container "solution" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 244184223Manfred Diaz Baseline-IL-logs-tensorflow aido3-LF-sim-validationstep1-simulation error no idsc-rudolf-88682019-10-24 06:36:44+00:00 2019-10-24 06:37:36+00:00 0:00:52 The container "solut [...] The container "solution" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 244124218Manfred Diaz Baseline-IL-logs-tensorflow aido3-LFV-sim-validationstep1-simulation error no idsc-rudolf-88682019-10-24 06:26:58+00:00 2019-10-24 06:29:24+00:00 0:02:26 The container "solut [...] The container "solution" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 243814189Liam Paull challenge-aido_LF-template-ros - Template solution using ROS aido3-LFV-sim-testingstep1-simulation error no idsc-rudolf-88682019-10-23 19:50:52+00:00 2019-10-23 20:02:16+00:00 0:11:24 The result file is n [...] The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido3-LFV-sim-testing/submission4189/step1-simulation-idsc-rudolf-8868-job24381:
File '/tmp/duckietown/DT18/evaluator/executions/aido3-LFV-sim-testing/submission4189/step1-simulation-idsc-rudolf-8868-job24381/challenge-results/challenge_results.yaml' does not exist.
This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 243724182Liam Paull challenge-aido_LF-baseline-duckietown aido3-LFV-sim-validationstep1-simulation error no idsc-rudolf-88682019-10-23 16:54:30+00:00 2019-10-23 16:57:05+00:00 0:02:35 The container "solut [...] The container "solution" exited with code 1.
Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
Andrea Censi 🇨🇭
