Duckietown Challenges Home Challenges Submissions

Submission 2587

Submission2587
Competingyes
Challengeaido2-LF-sim-validation
UserJulian Zilly
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22879
Next
User labelbaseline-IL-logs-tensorflow
Admin priority50
Blessingn/a
User priority50

22879

Click the images to see detailed statistics about the episode.

ETHZ_autolab_technical_track-0-0

ETHZ_autolab_technical_track-1-0

ETHZ_autolab_technical_track-2-0

ETHZ_autolab_technical_track-3-0

ETHZ_autolab_technical_track-4-0

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22879step1-simulationsuccessyes0:10:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.2608531657552158
survival_time_median6.199999999999986
deviation-center-line_median0.3140927400160046
in-drivable-lane_median0


other stats
agent_compute-ego_max0.23311424255371097
agent_compute-ego_mean0.21803120361556155
agent_compute-ego_median0.2169200162418553
agent_compute-ego_min0.19458247314799915
deviation-center-line_max0.36890251993379
deviation-center-line_mean0.3226659090997216
deviation-center-line_min0.2820011001806122
deviation-heading_max3.072135380403356
deviation-heading_mean2.3717584399359843
deviation-heading_median2.3921792858421393
deviation-heading_min1.4120486728376898
driven_any_max0.4300251850417715
driven_any_mean0.3057202478001917
driven_any_median0.32117061367009514
driven_any_min0.1661086451733553
driven_lanedir_consec_max0.38981315101197966
driven_lanedir_consec_mean0.2597740268424526
driven_lanedir_consec_min0.13425774532728818
driven_lanedir_max0.38981315101197966
driven_lanedir_mean0.2597740268424526
driven_lanedir_median0.2608531657552158
driven_lanedir_min0.13425774532728818
in-drivable-lane_max0.14999999999999947
in-drivable-lane_mean0.029999999999999895
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.32117061367009514, "sim_physics": 0.1117906762707618, "survival_time": 6.199999999999986, "driven_lanedir": 0.2608531657552158, "sim_render-ego": 0.06367051985956007, "in-drivable-lane": 0, "agent_compute-ego": 0.2305625004153098, "deviation-heading": 3.072135380403356, "set_robot_commands": 0.09852241316149311, "deviation-center-line": 0.36890251993379, "driven_lanedir_consec": 0.2608531657552158, "sim_compute_sim_state": 0.03968852373861497, "sim_compute_performance-ego": 0.07078293446571596, "sim_compute_robot_state-ego": 0.07795177159770843}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4300251850417715, "sim_physics": 0.1161406557810934, "survival_time": 7.599999999999981, "driven_lanedir": 0.38981315101197966, "sim_render-ego": 0.06311391686138354, "in-drivable-lane": 0, "agent_compute-ego": 0.23311424255371097, "deviation-heading": 2.3921792858421393, "set_robot_commands": 0.0981003406800722, "deviation-center-line": 0.33887719483445133, "driven_lanedir_consec": 0.38981315101197966, "sim_compute_sim_state": 0.03990696135320162, "sim_compute_performance-ego": 0.07072456259476512, "sim_compute_robot_state-ego": 0.07800968383487902}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.1661086451733553, "sim_physics": 0.08842905623013855, "survival_time": 3.049999999999997, "driven_lanedir": 0.13425774532728818, "sim_render-ego": 0.05961603023966805, "in-drivable-lane": 0.14999999999999947, "agent_compute-ego": 0.2169200162418553, "deviation-heading": 1.4120486728376898, "set_robot_commands": 0.09328927759264335, "deviation-center-line": 0.30945599053374967, "driven_lanedir_consec": 0.13425774532728818, "sim_compute_sim_state": 0.03769779987022525, "sim_compute_performance-ego": 0.0673617925800261, "sim_compute_robot_state-ego": 0.06996400629887815}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.25446675747671194, "sim_physics": 0.09483230926773766, "survival_time": 4.399999999999992, "driven_lanedir": 0.19977709420858816, "sim_render-ego": 0.05463953180746599, "in-drivable-lane": 0, "agent_compute-ego": 0.19458247314799915, "deviation-heading": 2.6337125011940135, "set_robot_commands": 0.08275702866640958, "deviation-center-line": 0.2820011001806122, "driven_lanedir_consec": 0.19977709420858816, "sim_compute_sim_state": 0.03373221917585893, "sim_compute_performance-ego": 0.059759340502999046, "sim_compute_robot_state-ego": 0.06350489638068459}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.3568300376390246, "sim_physics": 0.0922338796216388, "survival_time": 6.449999999999985, "driven_lanedir": 0.3141689779091914, "sim_render-ego": 0.05373262804608012, "in-drivable-lane": 0, "agent_compute-ego": 0.21497678571893264, "deviation-heading": 2.348716359402723, "set_robot_commands": 0.09434817003649336, "deviation-center-line": 0.3140927400160046, "driven_lanedir_consec": 0.3141689779091914, "sim_compute_sim_state": 0.0406203732010006, "sim_compute_performance-ego": 0.07696408079576122, "sim_compute_robot_state-ego": 0.06447291743847751}}
set_robot_commands_max0.09852241316149311
set_robot_commands_mean0.09340344602742232
set_robot_commands_median0.09434817003649336
set_robot_commands_min0.08275702866640958
sim_compute_performance-ego_max0.07696408079576122
sim_compute_performance-ego_mean0.06911854218785349
sim_compute_performance-ego_median0.07072456259476512
sim_compute_performance-ego_min0.059759340502999046
sim_compute_robot_state-ego_max0.07800968383487902
sim_compute_robot_state-ego_mean0.07078065511012553
sim_compute_robot_state-ego_median0.06996400629887815
sim_compute_robot_state-ego_min0.06350489638068459
sim_compute_sim_state_max0.0406203732010006
sim_compute_sim_state_mean0.03832917546778027
sim_compute_sim_state_median0.03968852373861497
sim_compute_sim_state_min0.03373221917585893
sim_physics_max0.1161406557810934
sim_physics_mean0.10068531543427404
sim_physics_median0.09483230926773766
sim_physics_min0.08842905623013855
sim_render-ego_max0.06367051985956007
sim_render-ego_mean0.05895452536283156
sim_render-ego_median0.05961603023966805
sim_render-ego_min0.05373262804608012
simulation-passed1
survival_time_max7.599999999999981
survival_time_mean5.5399999999999885
survival_time_min3.049999999999997
No reset possible
21155step1-simulationerrorno0:02:03
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 87, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 87, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 > 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20463step1-simulationsuccessno0:11:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19964step1-simulationerrorno0:00:58
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 250, in run_episode
    sim_ci.write_topic_and_expect_zero('episode_start', EpisodeStart(episode_name))
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "episode_start".
                   |
                   || Traceback (most recent call last):
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "gym_bridge.py", line 209, in on_received_episode_start
                   ||     self.update_observations(context)
                   ||   File "gym_bridge.py", line 281, in update_observations
                   ||     jpg_data = rgb2jpg(obs)
                   ||   File "gym_bridge.py", line 359, in rgb2jpg
                   ||     bgr = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
                   || cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
                   || > Unsupported depth of input image:
                   || >     'VDepth::contains(depth)'
                   || > where
                   || >     'depth' is 6 (CV_64F)
                   ||
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 135, in main
 >     physics_dt=config.physics_dt)
 >   File "experiment_manager.py", line 250, in run_episode
 >     sim_ci.write_topic_and_expect_zero('episode_start', EpisodeStart(episode_name))
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
 >     others = read_until_over(fpout, timeout=timeout, nickname=nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
 >     raise RemoteNodeAborted(m)
 > zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
 > 
 > error in simulator |Exception while handling a message on topic "episode_start".
 >                    |
 >                    || Traceback (most recent call last):
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
 >                    ||     handle_message_node(parsed, receiver0, context0)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
 >                    ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
 >                    ||     f(**kwargs)
 >                    ||   File "gym_bridge.py", line 209, in on_received_episode_start
 >                    ||     self.update_observations(context)
 >                    ||   File "gym_bridge.py", line 281, in update_observations
 >                    ||     jpg_data = rgb2jpg(obs)
 >                    ||   File "gym_bridge.py", line 359, in rgb2jpg
 >                    ||     bgr = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
 >                    || cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
 >                    || > Unsupported depth of input image:
 >                    || >     'VDepth::contains(depth)'
 >                    || > where
 >                    || >     'depth' is 6 (CV_64F)
 >                    ||
 >                    ||
 > 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19497step1-simulationsuccessno0:11:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18749step1-simulationsuccessno0:17:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18401step1-simulationsuccessno0:10:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18011step1-simulationsuccessno0:08:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible