Job ID step status up to date date started date completed duration message 22691
step1-simulation success yes 2019-05-17 00:31:50+00:00 2019-05-17 00:47:26+00:00 0:15:36 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median 0.32860646146617434 survival_time_median 5.599999999999988 deviation-center-line_median 0.2306472078071921 in-drivable-lane_median 1.999999999999993
other stats agent_compute-ego_max 0.2166734571967806 agent_compute-ego_mean 0.20513334711994224 agent_compute-ego_median 0.20290352238549125 agent_compute-ego_min 0.19705575367189804 deviation-center-line_max 0.37186798553077816 deviation-center-line_mean 0.25301593827813823 deviation-center-line_min 0.17867772524329653 deviation-heading_max 1.994732723406737 deviation-heading_mean 1.4720731931657152 deviation-heading_median 1.4003785013314385 deviation-heading_min 0.7005659636761089 driven_any_max 1.174378191651163 driven_any_mean 0.7492248936631178 driven_any_median 0.655969518607507 driven_any_min 0.5912358650185877 driven_lanedir_consec_max 0.637203514140166 driven_lanedir_consec_mean 0.3646106621718051 driven_lanedir_consec_min 0.23442696827770337 driven_lanedir_max 0.6681650233489003 driven_lanedir_mean 0.37119036249252335 driven_lanedir_median 0.3299001600753342 driven_lanedir_min 0.23442696827770337 in-drivable-lane_max 2.6000000000000076 in-drivable-lane_mean 2.0799999999999956 in-drivable-lane_min 1.699999999999994 per-episodes details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.174378191651163, "sim_physics": 0.15162389278411864, "survival_time": 8.999999999999993, "driven_lanedir": 0.6681650233489003, "sim_render-ego": 0.05883565213945177, "in-drivable-lane": 2.6000000000000076, "agent_compute-ego": 0.20290352238549125, "deviation-heading": 1.994732723406737, "set_robot_commands": 0.0904865304629008, "deviation-center-line": 0.37186798553077816, "driven_lanedir_consec": 0.637203514140166, "sim_compute_sim_state": 0.03661284711625841, "sim_compute_performance-ego": 0.06542532841364543, "sim_compute_robot_state-ego": 0.07084732717937893, "sim_compute_robot_state-npc0": 0.06540977292590672, "sim_compute_robot_state-npc1": 0.06671299669477675, "sim_compute_robot_state-npc2": 0.06428450875812107, "sim_compute_robot_state-npc3": 0.06433829201592339}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.5912358650185877, "sim_physics": 0.1790919355724169, "survival_time": 4.599999999999992, "driven_lanedir": 0.23442696827770337, "sim_render-ego": 0.05929586939189745, "in-drivable-lane": 2.249999999999992, "agent_compute-ego": 0.20739213798357092, "deviation-heading": 0.7005659636761089, "set_robot_commands": 0.10065415631169858, "deviation-center-line": 0.2306472078071921, "driven_lanedir_consec": 0.23442696827770337, "sim_compute_sim_state": 0.038109359533890434, "sim_compute_performance-ego": 0.06407859791880069, "sim_compute_robot_state-ego": 0.06931202307991359, "sim_compute_robot_state-npc0": 0.06793118559795877, "sim_compute_robot_state-npc1": 0.06792880659518034, "sim_compute_robot_state-npc2": 0.07072120386621226, "sim_compute_robot_state-npc3": 0.06761044263839722}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.655969518607507, "sim_physics": 0.14527067723481552, "survival_time": 5.749999999999988, "driven_lanedir": 0.3299001600753342, "sim_render-ego": 0.05741614673448645, "in-drivable-lane": 1.8499999999999936, "agent_compute-ego": 0.20164186436197032, "deviation-heading": 1.9176606914803176, "set_robot_commands": 0.08712980021601138, "deviation-center-line": 0.26831674694934954, "driven_lanedir_consec": 0.32860646146617434, "sim_compute_sim_state": 0.036653842096743376, "sim_compute_performance-ego": 0.06698507433352263, "sim_compute_robot_state-ego": 0.06959588424019192, "sim_compute_robot_state-npc0": 0.06576625575190005, "sim_compute_robot_state-npc1": 0.06511066063590672, "sim_compute_robot_state-npc2": 0.06168859937916631, "sim_compute_robot_state-npc3": 0.0628614093946374}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.7108706335637784, "sim_physics": 0.1669948250055313, "survival_time": 5.599999999999988, "driven_lanedir": 0.33698129422838763, "sim_render-ego": 0.06173392065933773, "in-drivable-lane": 1.999999999999993, "agent_compute-ego": 0.2166734571967806, "deviation-heading": 1.3470280859339745, "set_robot_commands": 0.09088939428329468, "deviation-center-line": 0.215570025860075, "driven_lanedir_consec": 0.33698129422838763, "sim_compute_sim_state": 0.03920985545430865, "sim_compute_performance-ego": 0.06684909973825727, "sim_compute_robot_state-ego": 0.07082424419266838, "sim_compute_robot_state-npc0": 0.07506326479571206, "sim_compute_robot_state-npc1": 0.07086211655821119, "sim_compute_robot_state-npc2": 0.06838269531726837, "sim_compute_robot_state-npc3": 0.06837057215826851}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.6136702594745531, "sim_physics": 0.15324626103887018, "survival_time": 5.299999999999989, "driven_lanedir": 0.2864783665322913, "sim_render-ego": 0.057843977550290665, "in-drivable-lane": 1.699999999999994, "agent_compute-ego": 0.19705575367189804, "deviation-heading": 1.4003785013314385, "set_robot_commands": 0.08935346468439642, "deviation-center-line": 0.17867772524329653, "driven_lanedir_consec": 0.2858350727465945, "sim_compute_sim_state": 0.03396160872477406, "sim_compute_performance-ego": 0.06258328905645406, "sim_compute_robot_state-ego": 0.06400582250559105, "sim_compute_robot_state-npc0": 0.06549601959732343, "sim_compute_robot_state-npc1": 0.06452866995109702, "sim_compute_robot_state-npc2": 0.060291915569665296, "sim_compute_robot_state-npc3": 0.05949511617984412}}set_robot_commands_max 0.10065415631169858 set_robot_commands_mean 0.09170266919166035 set_robot_commands_median 0.0904865304629008 set_robot_commands_min 0.08712980021601138 sim_compute_performance-ego_max 0.06698507433352263 sim_compute_performance-ego_mean 0.06518427789213602 sim_compute_performance-ego_median 0.06542532841364543 sim_compute_performance-ego_min 0.06258328905645406 sim_compute_robot_state-ego_max 0.07084732717937893 sim_compute_robot_state-ego_mean 0.06891706023954877 sim_compute_robot_state-ego_median 0.06959588424019192 sim_compute_robot_state-ego_min 0.06400582250559105 sim_compute_robot_state-npc0_max 0.07506326479571206 sim_compute_robot_state-npc0_mean 0.06793329973376021 sim_compute_robot_state-npc0_median 0.06576625575190005 sim_compute_robot_state-npc0_min 0.06540977292590672 sim_compute_robot_state-npc1_max 0.07086211655821119 sim_compute_robot_state-npc1_mean 0.0670286500870344 sim_compute_robot_state-npc1_median 0.06671299669477675 sim_compute_robot_state-npc1_min 0.06452866995109702 sim_compute_robot_state-npc2_max 0.07072120386621226 sim_compute_robot_state-npc2_mean 0.06507378457808666 sim_compute_robot_state-npc2_median 0.06428450875812107 sim_compute_robot_state-npc2_min 0.060291915569665296 sim_compute_robot_state-npc3_max 0.06837057215826851 sim_compute_robot_state-npc3_mean 0.06453516647741413 sim_compute_robot_state-npc3_median 0.06433829201592339 sim_compute_robot_state-npc3_min 0.05949511617984412 sim_compute_sim_state_max 0.03920985545430865 sim_compute_sim_state_mean 0.036909502585194987 sim_compute_sim_state_median 0.036653842096743376 sim_compute_sim_state_min 0.03396160872477406 sim_physics_max 0.1790919355724169 sim_physics_mean 0.1592455183271505 sim_physics_median 0.15324626103887018 sim_physics_min 0.14527067723481552 sim_render-ego_max 0.06173392065933773 sim_render-ego_mean 0.059025113295092815 sim_render-ego_median 0.05883565213945177 sim_render-ego_min 0.05741614673448645 simulation-passed 1 survival_time_max 8.999999999999993 survival_time_mean 6.04999999999999 survival_time_min 4.599999999999992
No reset possible 20927
step1-simulation success no 2019-04-27 15:03:36+00:00 2019-04-27 15:21:35+00:00 0:17:59 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 20906
step1-simulation host-error no 2019-04-27 14:56:40+00:00 2019-04-27 15:00:08+00:00 0:03:28 Uncaught exception:
[...] Uncaught exception:
Traceback (most recent call last):
File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
submission_id=submission_id, timeout_sec=timeout_sec)
File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
next(self.gen)
File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
convert(f_stdout)
File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-validation/submission2961/step1-simulation-ip-172-31-42-167-7227-job20906/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 20230
step1-simulation success no 2019-04-25 10:20:47+00:00 2019-04-25 10:43:21+00:00 0:22:34 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 19859
step1-simulation error no 2019-04-25 08:31:58+00:00 2019-04-25 08:32:54+00:00 0:00:56 InvalidEvaluator:
Tr [...] InvalidEvaluator:
Traceback (most recent call last):
File "experiment_manager.py", line 135, in main
physics_dt=config.physics_dt)
File "experiment_manager.py", line 250, in run_episode
sim_ci.write_topic_and_expect_zero('episode_start', EpisodeStart(episode_name))
File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
nickname=self.nickname)
File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
others = read_until_over(fpout, timeout=timeout, nickname=nickname)
File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
error in simulator |Exception while handling a message on topic "episode_start".
|
|| Traceback (most recent call last):
|| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
|| handle_message_node(parsed, receiver0, context0)
|| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
|| call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
|| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
|| f(**kwargs)
|| File "gym_bridge.py", line 209, in on_received_episode_start
|| self.update_observations(context)
|| File "gym_bridge.py", line 274, in update_observations
|| jpg_data = rgb2jpg(obs)
|| File "gym_bridge.py", line 352, in rgb2jpg
|| bgr = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
|| cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<1>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
|| > Unsupported depth of input image:
|| > 'VDepth::contains(depth)'
|| > where
|| > 'depth' is 6 (CV_64F)
||
||
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
yield cie
File "experiment_manager.py", line 439, in <module>
wrap(cie)
File "experiment_manager.py", line 427, in wrap
main(cie, logdir, attempts)
File "experiment_manager.py", line 186, in main
raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
> Traceback (most recent call last):
> File "experiment_manager.py", line 135, in main
> physics_dt=config.physics_dt)
> File "experiment_manager.py", line 250, in run_episode
> sim_ci.write_topic_and_expect_zero('episode_start', EpisodeStart(episode_name))
> File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
> nickname=self.nickname)
> File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
> others = read_until_over(fpout, timeout=timeout, nickname=nickname)
> File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
> raise RemoteNodeAborted(m)
> zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
>
> error in simulator |Exception while handling a message on topic "episode_start".
> |
> || Traceback (most recent call last):
> || File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
> || handle_message_node(parsed, receiver0, context0)
> || File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
> || call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
> || File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
> || f(**kwargs)
> || File "gym_bridge.py", line 209, in on_received_episode_start
> || self.update_observations(context)
> || File "gym_bridge.py", line 274, in update_observations
> || jpg_data = rgb2jpg(obs)
> || File "gym_bridge.py", line 352, in rgb2jpg
> || bgr = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)
> || cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<1>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2u; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> || > Unsupported depth of input image:
> || > 'VDepth::contains(depth)'
> || > where
> || > 'depth' is 6 (CV_64F)
> ||
> ||
>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 19208
step1-simulation success no 2019-04-24 12:27:36+00:00 2019-04-24 12:45:14+00:00 0:17:38 Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 19139
step1-simulation error no 2019-04-24 09:02:31+00:00 2019-04-24 09:25:42+00:00 0:23:11 Waited 1200 seconds [...] Waited 1200 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 19135
step1-simulation host-error no 2019-04-24 08:57:40+00:00 2019-04-24 08:59:10+00:00 0:01:30 Error while running [...]
stderr | Pulling simulator ...
stderr | Pulling solution ...
stderr | Pulling evaluator ...
stderr | Pulling scenario_maker ... error
stderr |
stderr | ERROR: for simulator Get https://registry-1.docker.io/v2/andreacensi/aido2-lfv-sim-validation-step1-simulation-simulator/manifests/2019_04_20_10_15_30: Get https://auth.docker.io/token?scope=repository%3Aandreacensi%2Faido2-lfv-sim-validation-step1-simulation-simulator%3Apull&service=registry.docker.io: net/http: TLS handshake timeout
stderr |
stderr | ERROR: for solution Get https://registry-1.docker.io/v2/julian121266/aido-submissions/manifests/2019_04_24_07_40_09: Get https://auth.docker.io/token?scope=repository%3Ajulian121266%2Faido-submissions%3Apull&service=registry.docker.io: net/http: TLS handshake timeout
stderr |
stderr | ERROR: for evaluator Get https://registry-1.docker.io/v2/andreacensi/aido2-lfv-sim-validation-step1-simulation-evaluator/manifests/2019_04_20_10_15_18: Get https://auth.docker.io/token?scope=repository%3Aandreacensi%2Faido2-lfv-sim-validation-step1-simulation-evaluator%3Apull&service=registry.docker.io: net/http: TLS handshake timeout
stderr |
stderr | ERROR: for scenario_maker Get https://registry-1.docker.io/v2/andreacensi/aido2-lfv-sim-validation-step1-simulation-scenario_maker/manifests/2019_04_20_10_15_59: Get https://auth.docker.io/token?scope=repository%3Aandreacensi%2Faido2-lfv-sim-validation-step1-simulation-scenario_maker%3Apull&service=registry.docker.io: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
stderr | Get https://registry-1.docker.io/v2/andreacensi/aido2-lfv-sim-validation-step1-simulation-simulator/manifests/2019_04_20_10_15_30: Get https://auth.docker.io/token?scope=repository%3Aandreacensi%2Faido2-lfv-sim-validation-step1-simulation-simulator%3Apull&service=registry.docker.io: net/http: TLS handshake timeout
stderr | Get https://registry-1.docker.io/v2/julian121266/aido-submissions/manifests/2019_04_24_07_40_09: Get https://auth.docker.io/token?scope=repository%3Ajulian121266%2Faido-submissions%3Apull&service=registry.docker.io: net/http: TLS handshake timeout
stderr | Get https://registry-1.docker.io/v2/andreacensi/aido2-lfv-sim-validation-step1-simulation-evaluator/manifests/2019_04_20_10_15_18: Get https://auth.docker.io/token?scope=repository%3Aandreacensi%2Faido2-lfv-sim-validation-step1-simulation-evaluator%3Apull&service=registry.docker.io: net/http: TLS handshake timeout
stderr | Get https://registry-1.docker.io/v2/andreacensi/aido2-lfv-sim-validation-step1-simulation-scenario_maker/manifests/2019_04_20_10_15_59: Get https://auth.docker.io/token?scope=repository%3Aandreacensi%2Faido2-lfv-sim-validation-step1-simulation-scenario_maker%3Apull&service=registry.docker.io: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 19130
step1-simulation timeout no 2019-04-24 06:31:07+00:00 2019-04-24 06:46:45+00:00 0:15:38 Timeout because eval [...] Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 19127
step1-simulation error no 2019-04-24 06:00:43+00:00 2019-04-24 06:21:28+00:00 0:20:45 Waited 1202 seconds [...] Waited 1202 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 19125
step1-simulation host-error no 2019-04-24 05:41:29+00:00 2019-04-24 06:00:06+00:00 0:18:37 Uncaught exception:
[...] Uncaught exception:
Traceback (most recent call last):
File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job19125-374845', 'ps', '-q', 'solution']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
submission_id=submission_id, timeout_sec=timeout_sec)
File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
write_logs(wd, project, services=config['services'])
File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
container_id = get_id_for_service(wd, project, service)
File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
o = run_docker(wd, project, cmd, get_output=True)
File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job19125-374845', 'ps', '-q', 'solution']:
> Command '['docker-compose', '-p', 'job19125-374845', 'ps', '-q', 'solution']' returned non-zero exit status 1.
stdout |
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible 19118
step1-simulation error no 2019-04-24 05:40:55+00:00 2019-04-24 06:02:15+00:00 0:21:20 Waited 1204 seconds [...] Waited 1204 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible