AI Driving Olympics Home Challenges Submissions Jobs

Submission 2861

Submission2861
Competingyes
Challengeaido2-LFV-sim-validation
UserAlexKaravaev
Date submitted
Completecomplete
Resultsuccess
Jobsstep1-simulation: 22734
Next
User labelchallenge-aido_LF-baseline-duckietown
Admin priority50
User priority50

22734

Click the images to see detailed statistics about the episode.

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
227342861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationsuccessyes-0:14:29(hidden)
driven_lanedir_consec_median0.47836629091548977
survival_time_median6.999999999999983
deviation-center-line_median0.20433366980906725
in-drivable-lane_median0.5999999999999979


other stats
agent_compute-ego_max0.09199162661019018
agent_compute-ego_mean0.08828414076225752
agent_compute-ego_median0.08755681276321411
agent_compute-ego_min0.08486901124318441
deviation-center-line_max0.28844415094366993
deviation-center-line_mean0.2239502979920882
deviation-center-line_min0.1624146827548989
deviation-heading_max1.9220264916359744
deviation-heading_mean1.7590353372017369
deviation-heading_median1.8527944533600644
deviation-heading_min1.4242247726612312
driven_any_max1.1707162632485568
driven_any_mean0.5686705564943999
driven_any_median0.5356965595175749
driven_any_min0.21022919986359032
driven_lanedir_consec_max0.495496351283172
driven_lanedir_consec_mean0.3752627679813157
driven_lanedir_consec_min0.17424171691909784
driven_lanedir_max0.495496351283172
driven_lanedir_mean0.3752627679813157
driven_lanedir_median0.47836629091548977
driven_lanedir_min0.17424171691909784
in-drivable-lane_max7.800000000000094
in-drivable-lane_mean1.8400000000000176
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.1707162632485568, "sim_physics": 0.10158046086629231, "survival_time": 14.950000000000076, "driven_lanedir": 0.495496351283172, "sim_render-ego": 0.06051742076873779, "in-drivable-lane": 7.800000000000094, "agent_compute-ego": 0.08755681276321411, "deviation-heading": 1.8527944533600644, "set_robot_commands": 0.07726306041081746, "deviation-center-line": 0.28844415094366993, "driven_lanedir_consec": 0.495496351283172, "sim_compute_sim_state": 0.0360496441523234, "sim_compute_performance-ego": 0.06226936022440593, "sim_compute_robot_state-ego": 0.06228128353754679, "sim_compute_robot_state-npc0": 0.06931014458338419, "sim_compute_robot_state-npc1": 0.0647650154431661, "sim_compute_robot_state-npc2": 0.06398614088694254, "sim_compute_robot_state-npc3": 0.06397000789642333}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.3508483993889199, "sim_physics": 0.10416061827476988, "survival_time": 4.699999999999991, "driven_lanedir": 0.23682562715238875, "sim_render-ego": 0.05845534040572795, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.08564724820725461, "deviation-heading": 1.711197107599585, "set_robot_commands": 0.0786243753230318, "deviation-center-line": 0.20093089734150887, "driven_lanedir_consec": 0.23682562715238875, "sim_compute_sim_state": 0.03629558137122621, "sim_compute_performance-ego": 0.06339593897474573, "sim_compute_robot_state-ego": 0.06326496601104736, "sim_compute_robot_state-npc0": 0.06761231574606388, "sim_compute_robot_state-npc1": 0.06090057657120076, "sim_compute_robot_state-npc2": 0.0617787685800106, "sim_compute_robot_state-npc3": 0.06363348250693464}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.5758623604533578, "sim_physics": 0.10391751607259114, "survival_time": 7.499999999999981, "driven_lanedir": 0.47836629091548977, "sim_render-ego": 0.05686238447825114, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.08486901124318441, "deviation-heading": 1.8849338607518291, "set_robot_commands": 0.07573312282562256, "deviation-center-line": 0.20433366980906725, "driven_lanedir_consec": 0.47836629091548977, "sim_compute_sim_state": 0.034448825518290205, "sim_compute_performance-ego": 0.060205397605895994, "sim_compute_robot_state-ego": 0.06039220650990804, "sim_compute_robot_state-npc0": 0.06685725053151449, "sim_compute_robot_state-npc1": 0.06173037687937419, "sim_compute_robot_state-npc2": 0.06210443973541259, "sim_compute_robot_state-npc3": 0.062376969655354816}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.5356965595175749, "sim_physics": 0.10589817762374878, "survival_time": 6.999999999999983, "driven_lanedir": 0.4913838536364299, "sim_render-ego": 0.06528695140566144, "in-drivable-lane": 0, "agent_compute-ego": 0.0913560049874442, "deviation-heading": 1.9220264916359744, "set_robot_commands": 0.07713988678795951, "deviation-center-line": 0.26362808911129604, "driven_lanedir_consec": 0.4913838536364299, "sim_compute_sim_state": 0.037812404973166326, "sim_compute_performance-ego": 0.06638285602842059, "sim_compute_robot_state-ego": 0.06438945872443062, "sim_compute_robot_state-npc0": 0.07351224422454834, "sim_compute_robot_state-npc1": 0.06887644018445696, "sim_compute_robot_state-npc2": 0.06686156136648995, "sim_compute_robot_state-npc3": 0.06684942075184414}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.21022919986359032, "sim_physics": 0.1032134799633996, "survival_time": 2.9499999999999975, "driven_lanedir": 0.17424171691909784, "sim_render-ego": 0.06337611958131951, "in-drivable-lane": 0, "agent_compute-ego": 0.09199162661019018, "deviation-heading": 1.4242247726612312, "set_robot_commands": 0.07640050225338693, "deviation-center-line": 0.1624146827548989, "driven_lanedir_consec": 0.17424171691909784, "sim_compute_sim_state": 0.03771425909915213, "sim_compute_performance-ego": 0.0672359870652021, "sim_compute_robot_state-ego": 0.06650322170580848, "sim_compute_robot_state-npc0": 0.07463279821104922, "sim_compute_robot_state-npc1": 0.0698024580034159, "sim_compute_robot_state-npc2": 0.06923918400780629, "sim_compute_robot_state-npc3": 0.06759670225240416}}
set_robot_commands_max0.0786243753230318
set_robot_commands_mean0.07703218952016366
set_robot_commands_median0.07713988678795951
set_robot_commands_min0.07573312282562256
sim_compute_performance-ego_max0.0672359870652021
sim_compute_performance-ego_mean0.06389790797973408
sim_compute_performance-ego_median0.06339593897474573
sim_compute_performance-ego_min0.060205397605895994
sim_compute_robot_state-ego_max0.06650322170580848
sim_compute_robot_state-ego_mean0.06336622729774825
sim_compute_robot_state-ego_median0.06326496601104736
sim_compute_robot_state-ego_min0.06039220650990804
sim_compute_robot_state-npc0_max0.07463279821104922
sim_compute_robot_state-npc0_mean0.07038495065931202
sim_compute_robot_state-npc0_median0.06931014458338419
sim_compute_robot_state-npc0_min0.06685725053151449
sim_compute_robot_state-npc1_max0.0698024580034159
sim_compute_robot_state-npc1_mean0.06521497341632279
sim_compute_robot_state-npc1_median0.0647650154431661
sim_compute_robot_state-npc1_min0.06090057657120076
sim_compute_robot_state-npc2_max0.06923918400780629
sim_compute_robot_state-npc2_mean0.0647940189153324
sim_compute_robot_state-npc2_median0.06398614088694254
sim_compute_robot_state-npc2_min0.0617787685800106
sim_compute_robot_state-npc3_max0.06759670225240416
sim_compute_robot_state-npc3_mean0.06488531661259221
sim_compute_robot_state-npc3_median0.06397000789642333
sim_compute_robot_state-npc3_min0.062376969655354816
sim_compute_sim_state_max0.037812404973166326
sim_compute_sim_state_mean0.03646414302283165
sim_compute_sim_state_median0.03629558137122621
sim_compute_sim_state_min0.034448825518290205
sim_physics_max0.10589817762374878
sim_physics_mean0.10375405056016034
sim_physics_median0.10391751607259114
sim_physics_min0.10158046086629231
sim_render-ego_max0.06528695140566144
sim_render-ego_mean0.06089964332793957
sim_render-ego_median0.06051742076873779
sim_render-ego_min0.05686238447825114
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean7.420000000000006
survival_time_min2.9499999999999975
209592861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationsuccessno-0:18:01(hidden)
driven_lanedir_consec_median0.4442836953793563
survival_time_median6.5999999999999845
deviation-center-line_median0.23011717945840268
in-drivable-lane_median0.14999999999999947


other stats
agent_compute-ego_max0.12275256315867106
agent_compute-ego_mean0.10480534467338098
agent_compute-ego_median0.10146036292567398
agent_compute-ego_min0.09774773567914964
deviation-center-line_max0.2677965905311012
deviation-center-line_mean0.2240018824772453
deviation-center-line_min0.1676578222570646
deviation-heading_max1.9590729116640704
deviation-heading_mean1.718855273685324
deviation-heading_median1.759743530959963
deviation-heading_min1.4542236037360072
driven_any_max1.1707030898390385
driven_any_mean0.5517917304513403
driven_any_median0.5035446157600671
driven_any_min0.21826570258829023
driven_lanedir_consec_max0.49543157849784336
driven_lanedir_consec_mean0.363214735545378
driven_lanedir_consec_min0.18106093948028257
driven_lanedir_max0.49543157849784336
driven_lanedir_mean0.363214735545378
driven_lanedir_median0.4442836953793563
driven_lanedir_min0.18106093948028257
in-drivable-lane_max7.9000000000000945
in-drivable-lane_mean1.7800000000000178
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.1707030898390385, "sim_physics": 0.15399993658065797, "survival_time": 14.950000000000076, "driven_lanedir": 0.49543157849784336, "sim_render-ego": 0.05325824419657389, "in-drivable-lane": 7.9000000000000945, "agent_compute-ego": 0.12275256315867106, "deviation-heading": 1.637814574719438, "set_robot_commands": 0.0816234024365743, "deviation-center-line": 0.2677965905311012, "driven_lanedir_consec": 0.49543157849784336, "sim_compute_sim_state": 0.03412662744522095, "sim_compute_performance-ego": 0.059219028949737545, "sim_compute_robot_state-ego": 0.06724810202916463, "sim_compute_robot_state-npc0": 0.06217306137084961, "sim_compute_robot_state-npc1": 0.0603417444229126, "sim_compute_robot_state-npc2": 0.06085826794306437, "sim_compute_robot_state-npc3": 0.06146678924560547}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.35888276677719, "sim_physics": 0.17299936215082803, "survival_time": 4.799999999999991, "driven_lanedir": 0.24748331544262123, "sim_render-ego": 0.0627503792444865, "in-drivable-lane": 0.7499999999999973, "agent_compute-ego": 0.09774773567914964, "deviation-heading": 1.759743530959963, "set_robot_commands": 0.0982886254787445, "deviation-center-line": 0.2070233828025143, "driven_lanedir_consec": 0.24748331544262123, "sim_compute_sim_state": 0.03909137099981308, "sim_compute_performance-ego": 0.06644412875175476, "sim_compute_robot_state-ego": 0.07793417821327846, "sim_compute_robot_state-npc0": 0.0721106082201004, "sim_compute_robot_state-npc1": 0.07072754700978597, "sim_compute_robot_state-npc2": 0.0694346899787585, "sim_compute_robot_state-npc3": 0.06805821011463802}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.5075624772921158, "sim_physics": 0.18783265307433625, "survival_time": 6.649999999999984, "driven_lanedir": 0.4478141489267866, "sim_render-ego": 0.062262959946367075, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.10391545654239512, "deviation-heading": 1.7834217473471417, "set_robot_commands": 0.10066343967179608, "deviation-center-line": 0.24741443733714355, "driven_lanedir_consec": 0.4478141489267866, "sim_compute_sim_state": 0.03942739156852091, "sim_compute_performance-ego": 0.07062899797482598, "sim_compute_robot_state-ego": 0.07568726145235219, "sim_compute_robot_state-npc0": 0.0719773375002065, "sim_compute_robot_state-npc1": 0.07020208889380433, "sim_compute_robot_state-npc2": 0.07261384519419276, "sim_compute_robot_state-npc3": 0.07338236686878634}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.5035446157600671, "sim_physics": 0.1844956116242842, "survival_time": 6.5999999999999845, "driven_lanedir": 0.4442836953793563, "sim_render-ego": 0.06146908528877027, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.10146036292567398, "deviation-heading": 1.9590729116640704, "set_robot_commands": 0.09928793979413582, "deviation-center-line": 0.23011717945840268, "driven_lanedir_consec": 0.4442836953793563, "sim_compute_sim_state": 0.04002102757945205, "sim_compute_performance-ego": 0.0695431087956284, "sim_compute_robot_state-ego": 0.07357061992992055, "sim_compute_robot_state-npc0": 0.06915734392223936, "sim_compute_robot_state-npc1": 0.07379083561174797, "sim_compute_robot_state-npc2": 0.0708716529788393, "sim_compute_robot_state-npc3": 0.07259641452269121}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.21826570258829023, "sim_physics": 0.17394244084592725, "survival_time": 3.049999999999997, "driven_lanedir": 0.18106093948028257, "sim_render-ego": 0.06939181734303959, "in-drivable-lane": 0, "agent_compute-ego": 0.09815060506101514, "deviation-heading": 1.4542236037360072, "set_robot_commands": 0.0966396918062304, "deviation-center-line": 0.1676578222570646, "driven_lanedir_consec": 0.18106093948028257, "sim_compute_sim_state": 0.03979993648216373, "sim_compute_performance-ego": 0.0699092833722224, "sim_compute_robot_state-ego": 0.07434421289162557, "sim_compute_robot_state-npc0": 0.07001878003605076, "sim_compute_robot_state-npc1": 0.06707228598047475, "sim_compute_robot_state-npc2": 0.07076052368664351, "sim_compute_robot_state-npc3": 0.07029433328597272}}
set_robot_commands_max0.10066343967179608
set_robot_commands_mean0.0953006198374962
set_robot_commands_median0.0982886254787445
set_robot_commands_min0.0816234024365743
sim_compute_performance-ego_max0.07062899797482598
sim_compute_performance-ego_mean0.06714890956883382
sim_compute_performance-ego_median0.0695431087956284
sim_compute_performance-ego_min0.059219028949737545
sim_compute_robot_state-ego_max0.07793417821327846
sim_compute_robot_state-ego_mean0.07375687490326828
sim_compute_robot_state-ego_median0.07434421289162557
sim_compute_robot_state-ego_min0.06724810202916463
sim_compute_robot_state-npc0_max0.0721106082201004
sim_compute_robot_state-npc0_mean0.06908742620988934
sim_compute_robot_state-npc0_median0.07001878003605076
sim_compute_robot_state-npc0_min0.06217306137084961
sim_compute_robot_state-npc1_max0.07379083561174797
sim_compute_robot_state-npc1_mean0.06842690038374513
sim_compute_robot_state-npc1_median0.07020208889380433
sim_compute_robot_state-npc1_min0.0603417444229126
sim_compute_robot_state-npc2_max0.07261384519419276
sim_compute_robot_state-npc2_mean0.0689077959562997
sim_compute_robot_state-npc2_median0.07076052368664351
sim_compute_robot_state-npc2_min0.06085826794306437
sim_compute_robot_state-npc3_max0.07338236686878634
sim_compute_robot_state-npc3_mean0.06915962280753876
sim_compute_robot_state-npc3_median0.07029433328597272
sim_compute_robot_state-npc3_min0.06146678924560547
sim_compute_sim_state_max0.04002102757945205
sim_compute_sim_state_mean0.03849327081503415
sim_compute_sim_state_median0.03942739156852091
sim_compute_sim_state_min0.03412662744522095
sim_physics_max0.18783265307433625
sim_physics_mean0.17465400085520671
sim_physics_median0.17394244084592725
sim_physics_min0.15399993658065797
sim_render-ego_max0.06939181734303959
sim_render-ego_mean0.06182649720384746
sim_render-ego_median0.062262959946367075
sim_render-ego_min0.05325824419657389
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean7.210000000000006
survival_time_min3.049999999999997
202712861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerrorno-0:01:42
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 87, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 87, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 > 
(hidden)
198772861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerrorno-0:00:58
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 250, in run_episode
    sim_ci.write_topic_and_expect_zero('episode_start', EpisodeStart(episode_name))
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "episode_start".
                   |
                   || Traceback (most recent call last):
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "gym_bridge.py", line 209, in on_received_episode_start
                   ||     self.update_observations(context)
                   ||   File "gym_bridge.py", line 274, in update_observations
                   ||     jpg_data = rgb2jpg(obs)
                   ||   File "gym_bridge.py", line 352, in rgb2jpg
                   ||     bgr = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
                   || cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<1>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
                   || > Unsupported depth of input image:
                   || >     'VDepth::contains(depth)'
                   || > where
                   || >     'depth' is 6 (CV_64F)
                   ||
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 135, in main
 >     physics_dt=config.physics_dt)
 >   File "experiment_manager.py", line 250, in run_episode
 >     sim_ci.write_topic_and_expect_zero('episode_start', EpisodeStart(episode_name))
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
 >     others = read_until_over(fpout, timeout=timeout, nickname=nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
 >     raise RemoteNodeAborted(m)
 > zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
 > 
 > error in simulator |Exception while handling a message on topic "episode_start".
 >                    |
 >                    || Traceback (most recent call last):
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
 >                    ||     handle_message_node(parsed, receiver0, context0)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
 >                    ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
 >                    ||     f(**kwargs)
 >                    ||   File "gym_bridge.py", line 209, in on_received_episode_start
 >                    ||     self.update_observations(context)
 >                    ||   File "gym_bridge.py", line 274, in update_observations
 >                    ||     jpg_data = rgb2jpg(obs)
 >                    ||   File "gym_bridge.py", line 352, in rgb2jpg
 >                    ||     bgr = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
 >                    || cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<1>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
 >                    || > Unsupported depth of input image:
 >                    || >     'VDepth::contains(depth)'
 >                    || > where
 >                    || >     'depth' is 6 (CV_64F)
 >                    ||
 >                    ||
 > 
(hidden)
192992861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationsuccessno-0:14:10(hidden)
driven_lanedir_consec_median0.4859810225669756
survival_time_median7.049999999999983
deviation-center-line_median0.17426736990173092
in-drivable-lane_median0.14999999999999947


other stats
agent_compute-ego_max0.08927338150725968
agent_compute-ego_mean0.08413082261537143
agent_compute-ego_median0.0833220396723066
agent_compute-ego_min0.0796888178967415
deviation-center-line_max0.25019790342318043
deviation-center-line_mean0.16339124489541643
deviation-center-line_min0.08619052063981153
deviation-heading_max2.040125562297356
deviation-heading_mean1.6687227758791956
deviation-heading_median1.632814462929599
deviation-heading_min1.415158341640376
driven_any_max1.176679831743056
driven_any_mean0.6351935225345445
driven_any_median0.5430208237082748
driven_any_min0.326034872298141
driven_lanedir_consec_max0.6189005306864326
driven_lanedir_consec_mean0.4560433452336109
driven_lanedir_consec_min0.2671153515255509
driven_lanedir_max0.6189005306864326
driven_lanedir_mean0.4560433452336109
driven_lanedir_median0.4859810225669756
driven_lanedir_min0.2671153515255509
in-drivable-lane_max7.750000000000095
in-drivable-lane_mean1.6600000000000186
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.176679831743056, "sim_physics": 0.025833771228790284, "survival_time": 14.950000000000076, "driven_lanedir": 0.4859810225669756, "sim_render-ego": 0.05364890416463216, "in-drivable-lane": 7.750000000000095, "agent_compute-ego": 0.08581033229827881, "deviation-heading": 2.040125562297356, "set_robot_commands": 0.07483145236968994, "deviation-center-line": 0.17426736990173092, "driven_lanedir_consec": 0.4859810225669756, "sim_compute_sim_state": 0.03351983626683553, "sim_compute_performance-ego": 0.05907703717549642, "sim_compute_robot_state-ego": 0.06101979811986288, "sim_compute_robot_state-npc0": 0.06203738927841187, "sim_compute_robot_state-npc1": 0.05948402325312296, "sim_compute_robot_state-npc2": 0.05908257007598877, "sim_compute_robot_state-npc3": 0.05989089488983154}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.326034872298141, "sim_physics": 0.03088687206136769, "survival_time": 4.3499999999999925, "driven_lanedir": 0.2671153515255509, "sim_render-ego": 0.05432322655601063, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.08927338150725968, "deviation-heading": 1.5868703988442918, "set_robot_commands": 0.07348061978131876, "deviation-center-line": 0.11748392774218525, "driven_lanedir_consec": 0.2671153515255509, "sim_compute_sim_state": 0.033193486860428736, "sim_compute_performance-ego": 0.05840928801174822, "sim_compute_robot_state-ego": 0.05990292286050731, "sim_compute_robot_state-npc0": 0.06487345695495605, "sim_compute_robot_state-npc1": 0.058400565180285226, "sim_compute_robot_state-npc2": 0.05775143634313824, "sim_compute_robot_state-npc3": 0.058503970332529354}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.6515117869338354, "sim_physics": 0.026311821880794707, "survival_time": 8.399999999999984, "driven_lanedir": 0.6189005306864326, "sim_render-ego": 0.05270246522767203, "in-drivable-lane": 0, "agent_compute-ego": 0.0833220396723066, "deviation-heading": 1.632814462929599, "set_robot_commands": 0.07354648766063508, "deviation-center-line": 0.25019790342318043, "driven_lanedir_consec": 0.6189005306864326, "sim_compute_sim_state": 0.032537265902473814, "sim_compute_performance-ego": 0.05647909925097511, "sim_compute_robot_state-ego": 0.05840747838928586, "sim_compute_robot_state-npc0": 0.06061296519779023, "sim_compute_robot_state-npc1": 0.05892115263711838, "sim_compute_robot_state-npc2": 0.05706012390908741, "sim_compute_robot_state-npc3": 0.056719198113396055}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.47872029798941546, "sim_physics": 0.027160001754760743, "survival_time": 6.249999999999986, "driven_lanedir": 0.4041798365370513, "sim_render-ego": 0.05318840980529785, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.0825595417022705, "deviation-heading": 1.415158341640376, "set_robot_commands": 0.07244341468811036, "deviation-center-line": 0.08619052063981153, "driven_lanedir_consec": 0.4041798365370513, "sim_compute_sim_state": 0.033066534042358396, "sim_compute_performance-ego": 0.06001057815551758, "sim_compute_robot_state-ego": 0.058784074783325195, "sim_compute_robot_state-npc0": 0.06156061172485352, "sim_compute_robot_state-npc1": 0.05889935684204101, "sim_compute_robot_state-npc2": 0.05691688346862793, "sim_compute_robot_state-npc3": 0.0582545223236084}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5430208237082748, "sim_physics": 0.026050303844695397, "survival_time": 7.049999999999983, "driven_lanedir": 0.5040399848520439, "sim_render-ego": 0.05330480244142789, "in-drivable-lane": 0, "agent_compute-ego": 0.0796888178967415, "deviation-heading": 1.6686451136843543, "set_robot_commands": 0.07068343703628432, "deviation-center-line": 0.18881650277017423, "driven_lanedir_consec": 0.5040399848520439, "sim_compute_sim_state": 0.0328058604653, "sim_compute_performance-ego": 0.058831661305529005, "sim_compute_robot_state-ego": 0.05997629875832416, "sim_compute_robot_state-npc0": 0.05914426019005742, "sim_compute_robot_state-npc1": 0.058139476370304186, "sim_compute_robot_state-npc2": 0.05691138734208777, "sim_compute_robot_state-npc3": 0.057878172989432694}}
set_robot_commands_max0.07483145236968994
set_robot_commands_mean0.0729970823072077
set_robot_commands_median0.07348061978131876
set_robot_commands_min0.07068343703628432
sim_compute_performance-ego_max0.06001057815551758
sim_compute_performance-ego_mean0.058561532779853265
sim_compute_performance-ego_median0.058831661305529005
sim_compute_performance-ego_min0.05647909925097511
sim_compute_robot_state-ego_max0.06101979811986288
sim_compute_robot_state-ego_mean0.059618114582261086
sim_compute_robot_state-ego_median0.05990292286050731
sim_compute_robot_state-ego_min0.05840747838928586
sim_compute_robot_state-npc0_max0.06487345695495605
sim_compute_robot_state-npc0_mean0.06164573666921382
sim_compute_robot_state-npc0_median0.06156061172485352
sim_compute_robot_state-npc0_min0.05914426019005742
sim_compute_robot_state-npc1_max0.05948402325312296
sim_compute_robot_state-npc1_mean0.05876891485657435
sim_compute_robot_state-npc1_median0.05889935684204101
sim_compute_robot_state-npc1_min0.058139476370304186
sim_compute_robot_state-npc2_max0.05908257007598877
sim_compute_robot_state-npc2_mean0.057544480227786035
sim_compute_robot_state-npc2_median0.05706012390908741
sim_compute_robot_state-npc2_min0.05691138734208777
sim_compute_robot_state-npc3_max0.05989089488983154
sim_compute_robot_state-npc3_mean0.05824935172975961
sim_compute_robot_state-npc3_median0.0582545223236084
sim_compute_robot_state-npc3_min0.056719198113396055
sim_compute_sim_state_max0.03351983626683553
sim_compute_sim_state_mean0.033024596707479294
sim_compute_sim_state_median0.033066534042358396
sim_compute_sim_state_min0.032537265902473814
sim_physics_max0.03088687206136769
sim_physics_mean0.02724855415408177
sim_physics_median0.026311821880794707
sim_physics_min0.025833771228790284
sim_render-ego_max0.05432322655601063
sim_render-ego_mean0.05343356163900811
sim_render-ego_median0.05330480244142789
sim_render-ego_min0.05270246522767203
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean8.200000000000005
survival_time_min4.3499999999999925
185912861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerrorno-0:21:36
Waited 1201 seconds [...]
Waited 1201 seconds for container to finish. Giving up. 
(hidden)
185882861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationhost-errorno-0:00:06
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 1041, in download_artefacts
    get_file_from_cache(fn, sha256hex)
  File "/project/src/duckietown_challenges_runner/runner_cache.py", line 27, in get_file_from_cache
    raise KeyError(msg)
KeyError: 'Hash not in cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 518, in run_single
    prepare_dir(wd, aws_config, steps2artefacts)
  File "/project/src/duckietown_challenges_runner/runner.py", line 785, in prepare_dir
    download_artefacts(aws_config, steps2artefacts, previous_steps_dir)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1062, in download_artefacts
    raise ValueError(msg)
ValueError: Corrupt cache or download for {'rpath': 'tmp/tmpkcoyldy0/attempts/ETHZ_autolab_technical_track-0-0.attempt0/log.gs2.cbor.tmp', 'mime_type': 'binary/octet-stream', 'sha256hex': '352e46035a8805bdf34665f8b50beeb9db3a1a2867eff8732ff286606285cf7d', 'size': 29822541, 'storage': {'s3': {'bucket_name': 'duckietown-ai-driving-olympics-1', 'object_key': 'v3/frankfurt/by-value/sha256/352e46035a8805bdf34665f8b50beeb9db3a1a2867eff8732ff286606285cf7d', 'url': 'http://duckietown-ai-driving-olympics-1.s3.amazonaws.com/v3/frankfurt/by-value/sha256/352e46035a8805bdf34665f8b50beeb9db3a1a2867eff8732ff286606285cf7d'}}} at /tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-validation/submission2861/step1-simulation-ip-172-31-25-98-9205-job18588/previous-steps/step1-simulation/tmp/tmpkcoyldy0/attempts/ETHZ_autolab_technical_track-0-0.attempt0/log.gs2.cbor.tmp.
(hidden)
185822861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationhost-errorno-0:02:36
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 691, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 862, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/container.py", line 755, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/containers/a9b78c1313bb4482e274c5ab8ec0779473d8b8f3df2fe977fd4f8c5c3fcab1eb/json
(hidden)
185682861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerrorno-0:24:45
Waited 1203 seconds [...]
Waited 1203 seconds for container to finish. Giving up. 
(hidden)
185642861AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFV-sim-validationstep1-simulationerrorno-0:21:28
Waited 1201 seconds [...]
Waited 1201 seconds for container to finish. Giving up. 
(hidden)