AI Driving Olympics Home Challenges Submissions Jobs

Submission 2681

Submission2681
Competingyes
Challengeaido2-LFV-sim-validation
UserRamiNaim
Date submitted
Completecomplete
Resulterror
Jobsstep1-simulation: 22828
Next
User labelchallenge-aido_LF-baseline-duckietown
Admin priority50
User priority50

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
228282681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerroryes-0:30:27
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
(hidden)
210942681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationsuccessno-0:35:49(hidden)
driven_lanedir_consec_median0.9957937578101996
survival_time_median13.70000000000006
deviation-center-line_median0.8919033096715387
in-drivable-lane_median3.2500000000000213


other stats
agent_compute-ego_max0.09059424877166748
agent_compute-ego_mean0.08936551493322044
agent_compute-ego_median0.08912953701648084
agent_compute-ego_min0.08839945157368978
deviation-center-line_max1.2581235091803151
deviation-center-line_mean0.8751610563365022
deviation-center-line_min0.36764988746677113
deviation-heading_max5.904874199503731
deviation-heading_mean4.161807181581326
deviation-heading_median4.764497426634542
deviation-heading_min0.9239633058573166
driven_any_max2.3489458661644
driven_any_mean1.9324367141189296
driven_any_median2.1476765385276213
driven_any_min0.6936907434607329
driven_lanedir_consec_max1.3207412666109584
driven_lanedir_consec_mean0.9981694774969612
driven_lanedir_consec_min0.6781641145269627
driven_lanedir_max2.0358244540105064
driven_lanedir_mean1.3642280822487858
driven_lanedir_median1.238397712208239
driven_lanedir_min0.6781641145269627
in-drivable-lane_max3.8000000000000016
in-drivable-lane_mean2.2000000000000104
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.1229560319859595, "sim_physics": 0.21011557771172715, "survival_time": 13.65000000000006, "driven_lanedir": 1.2145077467941174, "sim_render-ego": 0.07190062187530183, "in-drivable-lane": 3.8000000000000016, "agent_compute-ego": 0.08912953701648084, "deviation-heading": 4.764497426634542, "set_robot_commands": 0.11433945907341254, "deviation-center-line": 0.8221030822709682, "driven_lanedir_consec": 0.9957937578101996, "sim_compute_sim_state": 0.044265807329953374, "sim_compute_performance-ego": 0.08004131596603672, "sim_compute_robot_state-ego": 0.08986503649980594, "sim_compute_robot_state-npc0": 0.07997728442097758, "sim_compute_robot_state-npc1": 0.07906715861170283, "sim_compute_robot_state-npc2": 0.0803154964586754, "sim_compute_robot_state-npc3": 0.0788285391671317}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.3489143904559344, "sim_physics": 0.2255681729316711, "survival_time": 14.950000000000076, "driven_lanedir": 1.6542463837041033, "sim_render-ego": 0.07176458676656088, "in-drivable-lane": 3.2500000000000213, "agent_compute-ego": 0.08839945157368978, "deviation-heading": 4.1894688604145625, "set_robot_commands": 0.11573743343353272, "deviation-center-line": 0.8919033096715387, "driven_lanedir_consec": 1.1677571855209754, "sim_compute_sim_state": 0.04508675575256347, "sim_compute_performance-ego": 0.08008323351542156, "sim_compute_robot_state-ego": 0.08945188363393147, "sim_compute_robot_state-npc0": 0.08048590262730916, "sim_compute_robot_state-npc1": 0.08019828001658122, "sim_compute_robot_state-npc2": 0.0808284584681193, "sim_compute_robot_state-npc3": 0.07900531768798828}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.3489458661644, "sim_physics": 0.21248416503270467, "survival_time": 14.950000000000076, "driven_lanedir": 2.0358244540105064, "sim_render-ego": 0.07080610116322836, "in-drivable-lane": 0.5500000000000078, "agent_compute-ego": 0.09059424877166748, "deviation-heading": 5.026232115496474, "set_robot_commands": 0.11218767166137696, "deviation-center-line": 1.2581235091803151, "driven_lanedir_consec": 1.3207412666109584, "sim_compute_sim_state": 0.04452215433120728, "sim_compute_performance-ego": 0.0790914527575175, "sim_compute_robot_state-ego": 0.08758938074111938, "sim_compute_robot_state-npc0": 0.08054750204086304, "sim_compute_robot_state-npc1": 0.07888628005981445, "sim_compute_robot_state-npc2": 0.07966272751490275, "sim_compute_robot_state-npc3": 0.0802730123202006}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.1476765385276213, "sim_physics": 0.2251732897584456, "survival_time": 13.70000000000006, "driven_lanedir": 1.238397712208239, "sim_render-ego": 0.07075115973061889, "in-drivable-lane": 3.400000000000021, "agent_compute-ego": 0.08866259477434367, "deviation-heading": 5.904874199503731, "set_robot_commands": 0.11650410098751096, "deviation-center-line": 1.036025493092918, "driven_lanedir_consec": 0.8283910630157103, "sim_compute_sim_state": 0.044406065975662566, "sim_compute_performance-ego": 0.07991726033008882, "sim_compute_robot_state-ego": 0.08882726021926768, "sim_compute_robot_state-npc0": 0.08082061788461505, "sim_compute_robot_state-npc1": 0.07981448451968005, "sim_compute_robot_state-npc2": 0.08024440542624815, "sim_compute_robot_state-npc3": 0.07941135437819209}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.6936907434607329, "sim_physics": 0.21167700008679463, "survival_time": 4.6499999999999915, "driven_lanedir": 0.6781641145269627, "sim_render-ego": 0.07015076760322816, "in-drivable-lane": 0, "agent_compute-ego": 0.09004174252992035, "deviation-heading": 0.9239633058573166, "set_robot_commands": 0.11689859308222288, "deviation-center-line": 0.36764988746677113, "driven_lanedir_consec": 0.6781641145269627, "sim_compute_sim_state": 0.04441224631442819, "sim_compute_performance-ego": 0.07967128292206795, "sim_compute_robot_state-ego": 0.09049624012362573, "sim_compute_robot_state-npc0": 0.07915715761082147, "sim_compute_robot_state-npc1": 0.07775285936171009, "sim_compute_robot_state-npc2": 0.07675842315919938, "sim_compute_robot_state-npc3": 0.07786886153682586}}
set_robot_commands_max0.11689859308222288
set_robot_commands_mean0.1151334516476112
set_robot_commands_median0.11573743343353272
set_robot_commands_min0.11218767166137696
sim_compute_performance-ego_max0.08008323351542156
sim_compute_performance-ego_mean0.0797609090982265
sim_compute_performance-ego_median0.07991726033008882
sim_compute_performance-ego_min0.0790914527575175
sim_compute_robot_state-ego_max0.09049624012362573
sim_compute_robot_state-ego_mean0.08924596024355004
sim_compute_robot_state-ego_median0.08945188363393147
sim_compute_robot_state-ego_min0.08758938074111938
sim_compute_robot_state-npc0_max0.08082061788461505
sim_compute_robot_state-npc0_mean0.08019769291691727
sim_compute_robot_state-npc0_median0.08048590262730916
sim_compute_robot_state-npc0_min0.07915715761082147
sim_compute_robot_state-npc1_max0.08019828001658122
sim_compute_robot_state-npc1_mean0.07914381251389772
sim_compute_robot_state-npc1_median0.07906715861170283
sim_compute_robot_state-npc1_min0.07775285936171009
sim_compute_robot_state-npc2_max0.0808284584681193
sim_compute_robot_state-npc2_mean0.079561902205429
sim_compute_robot_state-npc2_median0.08024440542624815
sim_compute_robot_state-npc2_min0.07675842315919938
sim_compute_robot_state-npc3_max0.0802730123202006
sim_compute_robot_state-npc3_mean0.07907741701806771
sim_compute_robot_state-npc3_median0.07900531768798828
sim_compute_robot_state-npc3_min0.07786886153682586
sim_compute_sim_state_max0.04508675575256347
sim_compute_sim_state_mean0.04453860594076298
sim_compute_sim_state_median0.04441224631442819
sim_compute_sim_state_min0.044265807329953374
sim_physics_max0.2255681729316711
sim_physics_mean0.21700364110426865
sim_physics_median0.21248416503270467
sim_physics_min0.21011557771172715
sim_render-ego_max0.07190062187530183
sim_render-ego_mean0.07107464742778762
sim_render-ego_median0.07080610116322836
sim_render-ego_min0.07015076760322816
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean12.380000000000052
survival_time_min4.6499999999999915
203812681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerrorno-0:01:40
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 87, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 87, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 > 
(hidden)
199052681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerrorno-0:01:00
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 250, in run_episode
    sim_ci.write_topic_and_expect_zero('episode_start', EpisodeStart(episode_name))
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "episode_start".
                   |
                   || Traceback (most recent call last):
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "gym_bridge.py", line 209, in on_received_episode_start
                   ||     self.update_observations(context)
                   ||   File "gym_bridge.py", line 274, in update_observations
                   ||     jpg_data = rgb2jpg(obs)
                   ||   File "gym_bridge.py", line 352, in rgb2jpg
                   ||     bgr = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
                   || cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<1>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
                   || > Unsupported depth of input image:
                   || >     'VDepth::contains(depth)'
                   || > where
                   || >     'depth' is 6 (CV_64F)
                   ||
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 135, in main
 >     physics_dt=config.physics_dt)
 >   File "experiment_manager.py", line 250, in run_episode
 >     sim_ci.write_topic_and_expect_zero('episode_start', EpisodeStart(episode_name))
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
 >     others = read_until_over(fpout, timeout=timeout, nickname=nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
 >     raise RemoteNodeAborted(m)
 > zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
 > 
 > error in simulator |Exception while handling a message on topic "episode_start".
 >                    |
 >                    || Traceback (most recent call last):
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
 >                    ||     handle_message_node(parsed, receiver0, context0)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
 >                    ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
 >                    ||     f(**kwargs)
 >                    ||   File "gym_bridge.py", line 209, in on_received_episode_start
 >                    ||     self.update_observations(context)
 >                    ||   File "gym_bridge.py", line 274, in update_observations
 >                    ||     jpg_data = rgb2jpg(obs)
 >                    ||   File "gym_bridge.py", line 352, in rgb2jpg
 >                    ||     bgr = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
 >                    || cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<1>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
 >                    || > Unsupported depth of input image:
 >                    || >     'VDepth::contains(depth)'
 >                    || > where
 >                    || >     'depth' is 6 (CV_64F)
 >                    ||
 >                    ||
 > 
(hidden)
194332681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationsuccessno-0:13:21(hidden)
driven_lanedir_consec_median1.017763155263947
survival_time_median6.8999999999999835
deviation-center-line_median0.5866433865895332
in-drivable-lane_median0.4000000000000022


other stats
agent_compute-ego_max0.06325483640034993
agent_compute-ego_mean0.057459995697102875
agent_compute-ego_median0.05653932455814246
agent_compute-ego_min0.055294156074523926
deviation-center-line_max1.069032505900824
deviation-center-line_mean0.5731469410438331
deviation-center-line_min0.2121155800982467
deviation-heading_max6.259006909916686
deviation-heading_mean2.7766080489243463
deviation-heading_median2.250565177970314
deviation-heading_min1.2356908987578057
driven_any_max2.355121861464702
driven_any_mean1.221931866794732
driven_any_median1.0619157891698765
driven_any_min0.6895073285045462
driven_lanedir_consec_max1.1187885275170286
driven_lanedir_consec_mean0.783042928960264
driven_lanedir_consec_min0.19626064887495875
driven_lanedir_max1.5458747850927324
driven_lanedir_mean0.9359266745813536
driven_lanedir_median1.017763155263947
driven_lanedir_min0.4326392236096508
in-drivable-lane_max2.700000000000008
in-drivable-lane_mean0.8300000000000016
in-drivable-lane_min0.09999999999999964
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.6895073285045462, "sim_physics": 0.026791116242767664, "survival_time": 4.6499999999999915, "driven_lanedir": 0.5645676814234095, "sim_render-ego": 0.05035425514303228, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.05668962386346633, "deviation-heading": 1.6629397356700248, "set_robot_commands": 0.07044533760316911, "deviation-center-line": 0.2121155800982467, "driven_lanedir_consec": 0.5645676814234095, "sim_compute_sim_state": 0.03097235002825337, "sim_compute_performance-ego": 0.05371942315050351, "sim_compute_robot_state-ego": 0.05558983484903971, "sim_compute_robot_state-npc0": 0.05716032879326933, "sim_compute_robot_state-npc1": 0.05486120972582089, "sim_compute_robot_state-npc2": 0.05428396758212838, "sim_compute_robot_state-npc3": 0.054584103245888985}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.724288933224702, "sim_physics": 0.02614654352267583, "survival_time": 4.799999999999991, "driven_lanedir": 0.4326392236096508, "sim_render-ego": 0.04876981923977534, "in-drivable-lane": 0.6499999999999995, "agent_compute-ego": 0.055294156074523926, "deviation-heading": 2.474837522306902, "set_robot_commands": 0.0727498655517896, "deviation-center-line": 0.379069780479775, "driven_lanedir_consec": 0.19626064887495875, "sim_compute_sim_state": 0.030224576592445377, "sim_compute_performance-ego": 0.05281063914299011, "sim_compute_robot_state-ego": 0.05484995742638906, "sim_compute_robot_state-npc0": 0.05642807483673096, "sim_compute_robot_state-npc1": 0.054060690104961395, "sim_compute_robot_state-npc2": 0.053903400897979736, "sim_compute_robot_state-npc3": 0.05305881301561991}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.2788254216098338, "sim_physics": 0.02472358616915616, "survival_time": 8.249999999999982, "driven_lanedir": 1.1187885275170286, "sim_render-ego": 0.04817676110701127, "in-drivable-lane": 0.4000000000000022, "agent_compute-ego": 0.05653932455814246, "deviation-heading": 2.250565177970314, "set_robot_commands": 0.07027101227731415, "deviation-center-line": 0.5866433865895332, "driven_lanedir_consec": 1.1187885275170286, "sim_compute_sim_state": 0.030476906805327445, "sim_compute_performance-ego": 0.053174433563694806, "sim_compute_robot_state-ego": 0.055187193552652995, "sim_compute_robot_state-npc0": 0.05541540203672467, "sim_compute_robot_state-npc1": 0.05392079497828628, "sim_compute_robot_state-npc2": 0.05281954389629942, "sim_compute_robot_state-npc3": 0.05357650699037494}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.0619157891698765, "sim_physics": 0.024643999942834827, "survival_time": 6.8999999999999835, "driven_lanedir": 1.017763155263947, "sim_render-ego": 0.04989517426145249, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.05552203758903171, "deviation-heading": 1.2356908987578057, "set_robot_commands": 0.07011309050131535, "deviation-center-line": 0.6188734521507868, "driven_lanedir_consec": 1.017763155263947, "sim_compute_sim_state": 0.03068729241689046, "sim_compute_performance-ego": 0.05317961996880131, "sim_compute_robot_state-ego": 0.0542839160863904, "sim_compute_robot_state-npc0": 0.05618366123973459, "sim_compute_robot_state-npc1": 0.05448964367742124, "sim_compute_robot_state-npc2": 0.05388034081113511, "sim_compute_robot_state-npc3": 0.055089230122773544}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.355121861464702, "sim_physics": 0.027077728112538654, "survival_time": 14.950000000000076, "driven_lanedir": 1.5458747850927324, "sim_render-ego": 0.05255229791005452, "in-drivable-lane": 2.700000000000008, "agent_compute-ego": 0.06325483640034993, "deviation-heading": 6.259006909916686, "set_robot_commands": 0.07487359682718912, "deviation-center-line": 1.069032505900824, "driven_lanedir_consec": 1.0178346317219755, "sim_compute_sim_state": 0.03281435966491699, "sim_compute_performance-ego": 0.056205549240112306, "sim_compute_robot_state-ego": 0.058726726373036706, "sim_compute_robot_state-npc0": 0.05996860106786092, "sim_compute_robot_state-npc1": 0.05874718983968099, "sim_compute_robot_state-npc2": 0.057382389704386395, "sim_compute_robot_state-npc3": 0.05736568053563436}}
set_robot_commands_max0.07487359682718912
set_robot_commands_mean0.07169058055215546
set_robot_commands_median0.07044533760316911
set_robot_commands_min0.07011309050131535
sim_compute_performance-ego_max0.056205549240112306
sim_compute_performance-ego_mean0.05381793301322041
sim_compute_performance-ego_median0.05317961996880131
sim_compute_performance-ego_min0.05281063914299011
sim_compute_robot_state-ego_max0.058726726373036706
sim_compute_robot_state-ego_mean0.055727525657501775
sim_compute_robot_state-ego_median0.055187193552652995
sim_compute_robot_state-ego_min0.0542839160863904
sim_compute_robot_state-npc0_max0.05996860106786092
sim_compute_robot_state-npc0_mean0.05703121359486409
sim_compute_robot_state-npc0_median0.05642807483673096
sim_compute_robot_state-npc0_min0.05541540203672467
sim_compute_robot_state-npc1_max0.05874718983968099
sim_compute_robot_state-npc1_mean0.055215905665234155
sim_compute_robot_state-npc1_median0.05448964367742124
sim_compute_robot_state-npc1_min0.05392079497828628
sim_compute_robot_state-npc2_max0.057382389704386395
sim_compute_robot_state-npc2_mean0.05445392857838581
sim_compute_robot_state-npc2_median0.053903400897979736
sim_compute_robot_state-npc2_min0.05281954389629942
sim_compute_robot_state-npc3_max0.05736568053563436
sim_compute_robot_state-npc3_mean0.05473486678205834
sim_compute_robot_state-npc3_median0.054584103245888985
sim_compute_robot_state-npc3_min0.05305881301561991
sim_compute_sim_state_max0.03281435966491699
sim_compute_sim_state_mean0.03103509710156673
sim_compute_sim_state_median0.03068729241689046
sim_compute_sim_state_min0.030224576592445377
sim_physics_max0.027077728112538654
sim_physics_mean0.025876594797994628
sim_physics_median0.02614654352267583
sim_physics_min0.024643999942834827
sim_render-ego_max0.05255229791005452
sim_render-ego_mean0.04994966153226518
sim_render-ego_median0.04989517426145249
sim_render-ego_min0.04817676110701127
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean7.910000000000005
survival_time_min4.6499999999999915
194242681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationhost-errorno-0:12:37
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job19424-360383', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job19424-360383', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job19424-360383', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
(hidden)
186842681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerrorno-0:21:10
Waited 1201 seconds [...]
Waited 1201 seconds for container to finish. Giving up. 
(hidden)
183462681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationsuccessno-0:18:55(hidden)
driven_lanedir_consec_median1.1099178216504757
survival_time_median9.4
deviation-center-line_median0.6891850501480127
in-drivable-lane_median0.34999999999999876


other stats
agent_compute-ego_max0.04579476912816365
agent_compute-ego_mean0.045258548409244766
agent_compute-ego_median0.04534661374385888
agent_compute-ego_min0.04450549826716745
deviation-center-line_max1.094628842665842
deviation-center-line_mean0.6545654322508427
deviation-center-line_min0.2569150387489945
deviation-heading_max4.721511817928979
deviation-heading_mean2.706393786088841
deviation-heading_median2.1806889868169073
deviation-heading_min1.4022927119026225
driven_any_max2.3555294183517015
driven_any_mean1.469508400409695
driven_any_median1.4636647597625811
driven_any_min0.713613268734168
driven_lanedir_consec_max1.8340473946900513
driven_lanedir_consec_mean1.2122662729683198
driven_lanedir_consec_min0.6599278909573126
driven_lanedir_max1.8340473946900513
driven_lanedir_mean1.2574605952788829
driven_lanedir_median1.3358894332032905
driven_lanedir_min0.6599278909573126
in-drivable-lane_max1.6500000000000234
in-drivable-lane_mean0.5600000000000052
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.713613268734168, "sim_physics": 0.01851624498764674, "survival_time": 4.799999999999991, "driven_lanedir": 0.6599278909573126, "sim_render-ego": 0.0360946257909139, "in-drivable-lane": 0, "agent_compute-ego": 0.04561645785967509, "deviation-heading": 1.4022927119026225, "set_robot_commands": 0.05256840089956919, "deviation-center-line": 0.2569150387489945, "driven_lanedir_consec": 0.6599278909573126, "sim_compute_sim_state": 0.022585158546765644, "sim_compute_performance-ego": 0.03729526946942011, "sim_compute_robot_state-ego": 0.04336390147606531, "sim_compute_robot_state-npc0": 0.04030268142620722, "sim_compute_robot_state-npc1": 0.03920900324980418, "sim_compute_robot_state-npc2": 0.03852697958548864, "sim_compute_robot_state-npc3": 0.039198197424411774}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.6483609851898042, "sim_physics": 0.01765415114814071, "survival_time": 10.550000000000017, "driven_lanedir": 1.3358894332032905, "sim_render-ego": 0.0365279127636227, "in-drivable-lane": 0.8000000000000034, "agent_compute-ego": 0.04534661374385888, "deviation-heading": 3.5101866226493437, "set_robot_commands": 0.052120989532832285, "deviation-center-line": 0.6891850501480127, "driven_lanedir_consec": 1.1099178216504757, "sim_compute_sim_state": 0.022813438804228724, "sim_compute_performance-ego": 0.0373716569059833, "sim_compute_robot_state-ego": 0.04393601530536091, "sim_compute_robot_state-npc0": 0.040678261580625416, "sim_compute_robot_state-npc1": 0.03936681363254927, "sim_compute_robot_state-npc2": 0.039079926025245994, "sim_compute_robot_state-npc3": 0.03953034504894962}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.3555294183517015, "sim_physics": 0.016871001720428467, "survival_time": 14.950000000000076, "driven_lanedir": 1.8340473946900513, "sim_render-ego": 0.035764563083648684, "in-drivable-lane": 1.6500000000000234, "agent_compute-ego": 0.04579476912816365, "deviation-heading": 4.721511817928979, "set_robot_commands": 0.05217934449513753, "deviation-center-line": 1.094628842665842, "driven_lanedir_consec": 1.8340473946900513, "sim_compute_sim_state": 0.02270143429438273, "sim_compute_performance-ego": 0.0372372031211853, "sim_compute_robot_state-ego": 0.04262589454650879, "sim_compute_robot_state-npc0": 0.04053692976633707, "sim_compute_robot_state-npc1": 0.03920315345128377, "sim_compute_robot_state-npc2": 0.03860588709513346, "sim_compute_robot_state-npc3": 0.03903048515319824}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.4636647597625811, "sim_physics": 0.017992143935345588, "survival_time": 9.4, "driven_lanedir": 1.3992453660332962, "sim_render-ego": 0.03614132581873143, "in-drivable-lane": 0, "agent_compute-ego": 0.04502940304735874, "deviation-heading": 2.1806889868169073, "set_robot_commands": 0.051850984705255385, "deviation-center-line": 0.7914825498926447, "driven_lanedir_consec": 1.3992453660332962, "sim_compute_sim_state": 0.0227378378523157, "sim_compute_performance-ego": 0.0373641366654254, "sim_compute_robot_state-ego": 0.04299791569405414, "sim_compute_robot_state-npc0": 0.04101489579423945, "sim_compute_robot_state-npc1": 0.03911208725990133, "sim_compute_robot_state-npc2": 0.0390538344991968, "sim_compute_robot_state-npc3": 0.03945530348635735}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.1663735700102198, "sim_physics": 0.017463286191422418, "survival_time": 7.549999999999981, "driven_lanedir": 1.058192891510463, "sim_render-ego": 0.03657416476319168, "in-drivable-lane": 0.34999999999999876, "agent_compute-ego": 0.04450549826716745, "deviation-heading": 1.7172887911463517, "set_robot_commands": 0.051746930507634646, "deviation-center-line": 0.44061567979872, "driven_lanedir_consec": 1.058192891510463, "sim_compute_sim_state": 0.02284563929829377, "sim_compute_performance-ego": 0.03776296085079774, "sim_compute_robot_state-ego": 0.04293792610926344, "sim_compute_robot_state-npc0": 0.04048167001332669, "sim_compute_robot_state-npc1": 0.039403024888196525, "sim_compute_robot_state-npc2": 0.03896521574614064, "sim_compute_robot_state-npc3": 0.039079884030171576}}
set_robot_commands_max0.05256840089956919
set_robot_commands_mean0.052093330028085805
set_robot_commands_median0.052120989532832285
set_robot_commands_min0.051746930507634646
sim_compute_performance-ego_max0.03776296085079774
sim_compute_performance-ego_mean0.03740624540256237
sim_compute_performance-ego_median0.0373641366654254
sim_compute_performance-ego_min0.0372372031211853
sim_compute_robot_state-ego_max0.04393601530536091
sim_compute_robot_state-ego_mean0.04317233062625052
sim_compute_robot_state-ego_median0.04299791569405414
sim_compute_robot_state-ego_min0.04262589454650879
sim_compute_robot_state-npc0_max0.04101489579423945
sim_compute_robot_state-npc0_mean0.040602887716147174
sim_compute_robot_state-npc0_median0.04053692976633707
sim_compute_robot_state-npc0_min0.04030268142620722
sim_compute_robot_state-npc1_max0.039403024888196525
sim_compute_robot_state-npc1_mean0.039258816496347014
sim_compute_robot_state-npc1_median0.03920900324980418
sim_compute_robot_state-npc1_min0.03911208725990133
sim_compute_robot_state-npc2_max0.039079926025245994
sim_compute_robot_state-npc2_mean0.03884636859024111
sim_compute_robot_state-npc2_median0.03896521574614064
sim_compute_robot_state-npc2_min0.03852697958548864
sim_compute_robot_state-npc3_max0.03953034504894962
sim_compute_robot_state-npc3_mean0.03925884302861771
sim_compute_robot_state-npc3_median0.039198197424411774
sim_compute_robot_state-npc3_min0.03903048515319824
sim_compute_sim_state_max0.02284563929829377
sim_compute_sim_state_mean0.02273670175919731
sim_compute_sim_state_median0.0227378378523157
sim_compute_sim_state_min0.022585158546765644
sim_physics_max0.01851624498764674
sim_physics_mean0.017699365596596787
sim_physics_median0.01765415114814071
sim_physics_min0.016871001720428467
sim_render-ego_max0.03657416476319168
sim_render-ego_mean0.036220518444021674
sim_render-ego_median0.03614132581873143
sim_render-ego_min0.035764563083648684
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean9.450000000000014
survival_time_min4.799999999999991
180632681RamiNaimchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationsuccessno-0:18:02(hidden)
driven_lanedir_consec_median0.7322998488987488
survival_time_median3.649999999999995
deviation-center-line_median0.2243841654427191
in-drivable-lane_median0.2499999999999991


other stats
agent_compute-ego_max0.09507989088694256
agent_compute-ego_mean0.0829111340936868
agent_compute-ego_median0.08580351727349418
agent_compute-ego_min0.06654473526837075
deviation-center-line_max0.5460781579714795
deviation-center-line_mean0.2454386270980437
deviation-center-line_min0.07184242826360611
deviation-heading_max2.6009978653614185
deviation-heading_mean1.0875822232254675
deviation-heading_median0.7301029801806983
deviation-heading_min0.29493603329448104
driven_any_max2.2447786276145085
driven_any_mean1.2287432726675598
driven_any_median1.172333180328872
driven_any_min0.44993925369855226
driven_lanedir_consec_max1.055367425136
driven_lanedir_consec_mean0.7426813271613203
driven_lanedir_consec_min0.4390706498431785
driven_lanedir_max1.1973725684683487
driven_lanedir_mean0.8356958710752403
driven_lanedir_median0.8734905941477327
driven_lanedir_min0.4390706498431785
in-drivable-lane_max2.1999999999999953
in-drivable-lane_mean0.8499999999999979
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.44993925369855226, "sim_physics": 0.037595486640930174, "survival_time": 1.5000000000000009, "driven_lanedir": 0.4390706498431785, "sim_render-ego": 0.07450776100158692, "in-drivable-lane": 0, "agent_compute-ego": 0.09507989088694256, "deviation-heading": 0.29493603329448104, "set_robot_commands": 0.1104652961095174, "deviation-center-line": 0.11150855469386496, "driven_lanedir_consec": 0.4390706498431785, "sim_compute_sim_state": 0.04684177239735921, "sim_compute_performance-ego": 0.07834045092264812, "sim_compute_robot_state-ego": 0.08484856287638347, "sim_compute_robot_state-npc0": 0.07730731964111329, "sim_compute_robot_state-npc1": 0.075327467918396, "sim_compute_robot_state-npc2": 0.08039259115854899, "sim_compute_robot_state-npc3": 0.08289966583251954}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.2447786276145085, "sim_physics": 0.04045419182096209, "survival_time": 6.999999999999983, "driven_lanedir": 1.1973725684683487, "sim_render-ego": 0.07213619095938546, "in-drivable-lane": 2.1999999999999953, "agent_compute-ego": 0.08580351727349418, "deviation-heading": 2.6009978653614185, "set_robot_commands": 0.10883025441850935, "deviation-center-line": 0.5460781579714795, "driven_lanedir_consec": 0.7322998488987488, "sim_compute_sim_state": 0.044165228094373433, "sim_compute_performance-ego": 0.07548517840249198, "sim_compute_robot_state-ego": 0.08241915362221854, "sim_compute_robot_state-npc0": 0.07623380933489118, "sim_compute_robot_state-npc1": 0.0750100919178554, "sim_compute_robot_state-npc2": 0.07514847346714565, "sim_compute_robot_state-npc3": 0.07580923863819668}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.8838547392064144, "sim_physics": 0.04260615002025257, "survival_time": 2.7499999999999982, "driven_lanedir": 0.8734905941477327, "sim_render-ego": 0.07353438030589711, "in-drivable-lane": 0, "agent_compute-ego": 0.08886837959289551, "deviation-heading": 0.3484759887351704, "set_robot_commands": 0.11966739134355024, "deviation-center-line": 0.27337982911854875, "driven_lanedir_consec": 0.8734905941477327, "sim_compute_sim_state": 0.04431064779108221, "sim_compute_performance-ego": 0.07750679796392268, "sim_compute_robot_state-ego": 0.08653693632646041, "sim_compute_robot_state-npc0": 0.07736833745783026, "sim_compute_robot_state-npc1": 0.07830434279008346, "sim_compute_robot_state-npc2": 0.07924770441922274, "sim_compute_robot_state-npc3": 0.07854521491310813}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.392810562489452, "sim_physics": 0.03689907337057179, "survival_time": 4.3499999999999925, "driven_lanedir": 0.6131781177809414, "sim_render-ego": 0.0677457513480351, "in-drivable-lane": 1.799999999999995, "agent_compute-ego": 0.07825914744673104, "deviation-heading": 1.4633982485555703, "set_robot_commands": 0.099174400855755, "deviation-center-line": 0.07184242826360611, "driven_lanedir_consec": 0.6131781177809414, "sim_compute_sim_state": 0.03963475391782563, "sim_compute_performance-ego": 0.06998989225804121, "sim_compute_robot_state-ego": 0.07436155176710808, "sim_compute_robot_state-npc0": 0.06933127874615548, "sim_compute_robot_state-npc1": 0.07169905750230811, "sim_compute_robot_state-npc2": 0.07049196341942096, "sim_compute_robot_state-npc3": 0.07416401512321384}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.172333180328872, "sim_physics": 0.03244059053185868, "survival_time": 3.649999999999995, "driven_lanedir": 1.055367425136, "sim_render-ego": 0.054260155926012014, "in-drivable-lane": 0.2499999999999991, "agent_compute-ego": 0.06654473526837075, "deviation-heading": 0.7301029801806983, "set_robot_commands": 0.08243542174770407, "deviation-center-line": 0.2243841654427191, "driven_lanedir_consec": 1.055367425136, "sim_compute_sim_state": 0.030537170906589455, "sim_compute_performance-ego": 0.05743455560240027, "sim_compute_robot_state-ego": 0.05895422582756983, "sim_compute_robot_state-npc0": 0.05526537437961526, "sim_compute_robot_state-npc1": 0.05455454081705172, "sim_compute_robot_state-npc2": 0.056002950015133376, "sim_compute_robot_state-npc3": 0.054765381225167886}}
set_robot_commands_max0.11966739134355024
set_robot_commands_mean0.1041145528950072
set_robot_commands_median0.10883025441850935
set_robot_commands_min0.08243542174770407
sim_compute_performance-ego_max0.07834045092264812
sim_compute_performance-ego_mean0.07175137502990085
sim_compute_performance-ego_median0.07548517840249198
sim_compute_performance-ego_min0.05743455560240027
sim_compute_robot_state-ego_max0.08653693632646041
sim_compute_robot_state-ego_mean0.07742408608394806
sim_compute_robot_state-ego_median0.08241915362221854
sim_compute_robot_state-ego_min0.05895422582756983
sim_compute_robot_state-npc0_max0.07736833745783026
sim_compute_robot_state-npc0_mean0.0711012239119211
sim_compute_robot_state-npc0_median0.07623380933489118
sim_compute_robot_state-npc0_min0.05526537437961526
sim_compute_robot_state-npc1_max0.07830434279008346
sim_compute_robot_state-npc1_mean0.07097910018913893
sim_compute_robot_state-npc1_median0.0750100919178554
sim_compute_robot_state-npc1_min0.05455454081705172
sim_compute_robot_state-npc2_max0.08039259115854899
sim_compute_robot_state-npc2_mean0.07225673649589434
sim_compute_robot_state-npc2_median0.07514847346714565
sim_compute_robot_state-npc2_min0.056002950015133376
sim_compute_robot_state-npc3_max0.08289966583251954
sim_compute_robot_state-npc3_mean0.07323670314644122
sim_compute_robot_state-npc3_median0.07580923863819668
sim_compute_robot_state-npc3_min0.054765381225167886
sim_compute_sim_state_max0.04684177239735921
sim_compute_sim_state_mean0.04109791462144598
sim_compute_sim_state_median0.044165228094373433
sim_compute_sim_state_min0.030537170906589455
sim_physics_max0.04260615002025257
sim_physics_mean0.03799909847691506
sim_physics_median0.037595486640930174
sim_physics_min0.03244059053185868
sim_render-ego_max0.07450776100158692
sim_render-ego_mean0.06843684790818333
sim_render-ego_median0.07213619095938546
sim_render-ego_min0.054260155926012014
simulation-passed1
survival_time_max6.999999999999983
survival_time_mean3.8499999999999943
survival_time_min1.5000000000000009