AI Driving Olympics Home Challenges Submissions Jobs

Submission 3101

Submission3101
Competingyes
Challengeaido2-LFV-sim-validation
Userawoo424
Date submitted
Completecomplete
Resulterror
Jobsstep1-simulation: 22616
Next
User labelHKU Duckietown Project
Admin priority50
User priority50

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
226163101awoo424HKU Duckietown Projectaido2-LFV-sim-validationstep1-simulationerroryes-0:26:40
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
(hidden)
226133101awoo424HKU Duckietown Projectaido2-LFV-sim-validationstep1-simulationhost-erroryes-0:01:20
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job22613-278309', 'pull']:

   >  Command '['docker-compose', '-p', 'job22613-278309', 'pull']' returned non-zero exit status 1.

stdout | 

stderr | Some networks were defined but are not used by any service: evaluation
stderr | Pulling evaluator      ...
stderr | Pulling scenario_maker ...
stderr | Pulling simulator      ...
stderr | Pulling solution       ...
stderr | 
stderr | Pulling scenario_maker ... error
stderr | 
stderr | Pulling solution       ... error
stderr | 
stderr | Pulling simulator      ... done
stderr | 
stderr | Pulling evaluator      ... done
stderr | 
stderr | ERROR: for scenario_maker  Get https://registry-1.docker.io/v2/andreacensi/aido2-lfv-sim-validation-step1-simulation-scenario_maker/manifests/2019_05_16_15_49_26: Get https://auth.docker.io/token?scope=repository%3Aandreacensi%2Faido2-lfv-sim-validation-step1-simulation-scenario_maker%3Apull&service=registry.docker.io: net/http: TLS handshake timeout
stderr | 
stderr | ERROR: for solution  Get https://registry-1.docker.io/v2/duckiet/aido-submissions/manifests/2019_04_27_20_33_14: Get https://auth.docker.io/token?scope=repository%3Aduckiet%2Faido-submissions%3Apull&service=registry.docker.io: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
stderr | Get https://registry-1.docker.io/v2/andreacensi/aido2-lfv-sim-validation-step1-simulation-scenario_maker/manifests/2019_05_16_15_49_26: Get https://auth.docker.io/token?scope=repository%3Aandreacensi%2Faido2-lfv-sim-validation-step1-simulation-scenario_maker%3Apull&service=registry.docker.io: net/http: TLS handshake timeout
stderr | Get https://registry-1.docker.io/v2/duckiet/aido-submissions/manifests/2019_04_27_20_33_14: Get https://auth.docker.io/token?scope=repository%3Aduckiet%2Faido-submissions%3Apull&service=registry.docker.io: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
stderr | 
(hidden)
208093101awoo424HKU Duckietown Projectaido2-LFV-sim-validationstep1-simulationsuccessno-0:36:01(hidden)
driven_lanedir_consec_median0.439925096833917
survival_time_median13.250000000000052
deviation-center-line_median0.6239853399057752
in-drivable-lane_median4.9000000000000465


other stats
agent_compute-ego_max0.2342921382976028
agent_compute-ego_mean0.22661939502614417
agent_compute-ego_median0.2277700996398926
agent_compute-ego_min0.21907610647668144
deviation-center-line_max0.7435127051255117
deviation-center-line_mean0.595602707926662
deviation-center-line_min0.4065453237131485
deviation-heading_max3.231178580537095
deviation-heading_mean2.453660223543085
deviation-heading_median2.405225835484903
deviation-heading_min1.165103464359677
driven_any_max0.9487107508352632
driven_any_mean0.855995194707601
driven_any_median0.8333307977381974
driven_any_min0.7372626499997462
driven_lanedir_consec_max0.8275161517633438
driven_lanedir_consec_mean0.5709451062193592
driven_lanedir_consec_min0.38940065476844854
driven_lanedir_max0.8275161517633438
driven_lanedir_mean0.5709451062193592
driven_lanedir_median0.439925096833917
driven_lanedir_min0.38940065476844854
in-drivable-lane_max6.600000000000069
in-drivable-lane_mean3.990000000000046
in-drivable-lane_min1.2500000000000178
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.8333307977381974, "sim_physics": 0.189065043431408, "survival_time": 13.250000000000052, "driven_lanedir": 0.3992873378221544, "sim_render-ego": 0.06551824245812758, "in-drivable-lane": 6.600000000000069, "agent_compute-ego": 0.2342921382976028, "deviation-heading": 1.165103464359677, "set_robot_commands": 0.1042845959933299, "deviation-center-line": 0.7076613834832143, "driven_lanedir_consec": 0.3992873378221544, "sim_compute_sim_state": 0.04017598763951716, "sim_compute_performance-ego": 0.07253786752808769, "sim_compute_robot_state-ego": 0.07914047151241663, "sim_compute_robot_state-npc0": 0.07179236322079065, "sim_compute_robot_state-npc1": 0.072026911321676, "sim_compute_robot_state-npc2": 0.07137072401226692, "sim_compute_robot_state-npc3": 0.07298133058368035}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9464030616250384, "sim_physics": 0.1873590024312337, "survival_time": 14.950000000000076, "driven_lanedir": 0.8275161517633438, "sim_render-ego": 0.06409026861190796, "in-drivable-lane": 1.2500000000000178, "agent_compute-ego": 0.2277700996398926, "deviation-heading": 3.231178580537095, "set_robot_commands": 0.10107333660125732, "deviation-center-line": 0.6239853399057752, "driven_lanedir_consec": 0.8275161517633438, "sim_compute_sim_state": 0.04072420597076416, "sim_compute_performance-ego": 0.06961399714152018, "sim_compute_robot_state-ego": 0.07678272565205892, "sim_compute_robot_state-npc0": 0.07237754821777344, "sim_compute_robot_state-npc1": 0.07232784986495971, "sim_compute_robot_state-npc2": 0.07189279397328695, "sim_compute_robot_state-npc3": 0.07230599244435629}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.8142687133397596, "sim_physics": 0.182348615769651, "survival_time": 13.150000000000052, "driven_lanedir": 0.439925096833917, "sim_render-ego": 0.06401432693684508, "in-drivable-lane": 5.500000000000071, "agent_compute-ego": 0.22327109797372563, "deviation-heading": 2.405225835484903, "set_robot_commands": 0.10001238036065048, "deviation-center-line": 0.496308787405661, "driven_lanedir_consec": 0.439925096833917, "sim_compute_sim_state": 0.04009717226934977, "sim_compute_performance-ego": 0.07003783091846075, "sim_compute_robot_state-ego": 0.07639892744927351, "sim_compute_robot_state-npc0": 0.07287078665236557, "sim_compute_robot_state-npc1": 0.0715619584000156, "sim_compute_robot_state-npc2": 0.07122658504732661, "sim_compute_robot_state-npc3": 0.07193901239692485}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.7372626499997462, "sim_physics": 0.16914276094395717, "survival_time": 11.65000000000003, "driven_lanedir": 0.38940065476844854, "sim_render-ego": 0.05982533340290381, "in-drivable-lane": 4.9000000000000465, "agent_compute-ego": 0.21907610647668144, "deviation-heading": 2.3072659701668887, "set_robot_commands": 0.09573908220544904, "deviation-center-line": 0.4065453237131485, "driven_lanedir_consec": 0.38940065476844854, "sim_compute_sim_state": 0.036069920134646695, "sim_compute_performance-ego": 0.06608295747650539, "sim_compute_robot_state-ego": 0.07274575499505956, "sim_compute_robot_state-npc0": 0.06745799621287334, "sim_compute_robot_state-npc1": 0.0663107689869762, "sim_compute_robot_state-npc2": 0.0662936304771849, "sim_compute_robot_state-npc3": 0.06562667650214592}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.9487107508352632, "sim_physics": 0.17814332882563272, "survival_time": 14.950000000000076, "driven_lanedir": 0.7985962899089323, "sim_render-ego": 0.06444637854894002, "in-drivable-lane": 1.7000000000000242, "agent_compute-ego": 0.2286875327428182, "deviation-heading": 3.1595272671668613, "set_robot_commands": 0.0971648621559143, "deviation-center-line": 0.7435127051255117, "driven_lanedir_consec": 0.7985962899089323, "sim_compute_sim_state": 0.0403385329246521, "sim_compute_performance-ego": 0.07097038984298706, "sim_compute_robot_state-ego": 0.07930594364802043, "sim_compute_robot_state-npc0": 0.06973622878392538, "sim_compute_robot_state-npc1": 0.07054235935211181, "sim_compute_robot_state-npc2": 0.06982650836308797, "sim_compute_robot_state-npc3": 0.07079195578893026}}
set_robot_commands_max0.1042845959933299
set_robot_commands_mean0.09965485146332022
set_robot_commands_median0.10001238036065048
set_robot_commands_min0.09573908220544904
sim_compute_performance-ego_max0.07253786752808769
sim_compute_performance-ego_mean0.06984860858151223
sim_compute_performance-ego_median0.07003783091846075
sim_compute_performance-ego_min0.06608295747650539
sim_compute_robot_state-ego_max0.07930594364802043
sim_compute_robot_state-ego_mean0.07687476465136582
sim_compute_robot_state-ego_median0.07678272565205892
sim_compute_robot_state-ego_min0.07274575499505956
sim_compute_robot_state-npc0_max0.07287078665236557
sim_compute_robot_state-npc0_mean0.07084698461754567
sim_compute_robot_state-npc0_median0.07179236322079065
sim_compute_robot_state-npc0_min0.06745799621287334
sim_compute_robot_state-npc1_max0.07232784986495971
sim_compute_robot_state-npc1_mean0.07055396958514785
sim_compute_robot_state-npc1_median0.0715619584000156
sim_compute_robot_state-npc1_min0.0663107689869762
sim_compute_robot_state-npc2_max0.07189279397328695
sim_compute_robot_state-npc2_mean0.07012204837463067
sim_compute_robot_state-npc2_median0.07122658504732661
sim_compute_robot_state-npc2_min0.0662936304771849
sim_compute_robot_state-npc3_max0.07298133058368035
sim_compute_robot_state-npc3_mean0.07072899354320754
sim_compute_robot_state-npc3_median0.07193901239692485
sim_compute_robot_state-npc3_min0.06562667650214592
sim_compute_sim_state_max0.04072420597076416
sim_compute_sim_state_mean0.039481163787785975
sim_compute_sim_state_median0.04017598763951716
sim_compute_sim_state_min0.036069920134646695
sim_physics_max0.189065043431408
sim_physics_mean0.18121175028037656
sim_physics_median0.182348615769651
sim_physics_min0.16914276094395717
sim_render-ego_max0.06551824245812758
sim_render-ego_mean0.0635789099917449
sim_render-ego_median0.06409026861190796
sim_render-ego_min0.05982533340290381
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean13.590000000000057
survival_time_min11.65000000000003
207063101awoo424HKU Duckietown Projectaido2-LFV-sim-validationstep1-simulationsuccessno-0:21:16(hidden)
driven_lanedir_consec_median0.4434704365929587
survival_time_median13.300000000000054
deviation-center-line_median0.658003595311064
in-drivable-lane_median5.250000000000065


other stats
agent_compute-ego_max0.1226197067896525
agent_compute-ego_mean0.1210524699927407
agent_compute-ego_median0.1209040127548517
agent_compute-ego_min0.12016444890074028
deviation-center-line_max0.8132426664665893
deviation-center-line_mean0.635026285791169
deviation-center-line_min0.4844235659756217
deviation-heading_max3.352973803126495
deviation-heading_mean2.5068386765895565
deviation-heading_median2.4185519043469443
deviation-heading_min1.1753094982328518
driven_any_max0.93824410752437
driven_any_mean0.8619384153958155
driven_any_median0.8300281175540088
driven_any_min0.788410205146125
driven_lanedir_consec_max0.9001127983452686
driven_lanedir_consec_mean0.5999638228146578
driven_lanedir_consec_min0.37462671691741933
driven_lanedir_max0.9001127983452686
driven_lanedir_mean0.5999638228146578
driven_lanedir_median0.4434704365929587
driven_lanedir_min0.37462671691741933
in-drivable-lane_max6.700000000000062
in-drivable-lane_mean3.700000000000043
in-drivable-lane_min0.10000000000000142
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.8150878195483384, "sim_physics": 0.08998079262962637, "survival_time": 12.900000000000048, "driven_lanedir": 0.37462671691741933, "sim_render-ego": 0.03499367255573125, "in-drivable-lane": 6.700000000000062, "agent_compute-ego": 0.12016444890074028, "deviation-heading": 1.1753094982328518, "set_robot_commands": 0.05520692352176637, "deviation-center-line": 0.658003595311064, "driven_lanedir_consec": 0.37462671691741933, "sim_compute_sim_state": 0.02223327825235766, "sim_compute_performance-ego": 0.039237959440364394, "sim_compute_robot_state-ego": 0.04017306021017621, "sim_compute_robot_state-npc0": 0.0400051941243253, "sim_compute_robot_state-npc1": 0.039133101470710696, "sim_compute_robot_state-npc2": 0.038922836614209554, "sim_compute_robot_state-npc3": 0.03903908415358196}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9379218272062352, "sim_physics": 0.09864480177561442, "survival_time": 14.950000000000076, "driven_lanedir": 0.9001127983452686, "sim_render-ego": 0.034993871847788495, "in-drivable-lane": 0.10000000000000142, "agent_compute-ego": 0.1226197067896525, "deviation-heading": 3.352973803126495, "set_robot_commands": 0.05408549706141154, "deviation-center-line": 0.7204204432371522, "driven_lanedir_consec": 0.9001127983452686, "sim_compute_sim_state": 0.02196122407913208, "sim_compute_performance-ego": 0.03868539651234944, "sim_compute_robot_state-ego": 0.03991535822550456, "sim_compute_robot_state-npc0": 0.039609045187632246, "sim_compute_robot_state-npc1": 0.038498940467834475, "sim_compute_robot_state-npc2": 0.03836216688156128, "sim_compute_robot_state-npc3": 0.03850587606430054}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.8300281175540088, "sim_physics": 0.09088143549467388, "survival_time": 13.300000000000054, "driven_lanedir": 0.4434704365929587, "sim_render-ego": 0.03432492385233255, "in-drivable-lane": 5.600000000000073, "agent_compute-ego": 0.12065617303202925, "deviation-heading": 2.4185519043469443, "set_robot_commands": 0.05443933763002094, "deviation-center-line": 0.4990411579654187, "driven_lanedir_consec": 0.4434704365929587, "sim_compute_sim_state": 0.022063791303706348, "sim_compute_performance-ego": 0.037407209102372475, "sim_compute_robot_state-ego": 0.04007728028118162, "sim_compute_robot_state-npc0": 0.0392975179772628, "sim_compute_robot_state-npc1": 0.040530811575122346, "sim_compute_robot_state-npc2": 0.03820562990088212, "sim_compute_robot_state-npc3": 0.03937955279099314}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.788410205146125, "sim_physics": 0.09023685361824783, "survival_time": 12.750000000000046, "driven_lanedir": 0.43144417737982144, "sim_render-ego": 0.0363799113853305, "in-drivable-lane": 5.250000000000065, "agent_compute-ego": 0.1209040127548517, "deviation-heading": 2.3783435247397895, "set_robot_commands": 0.054087104049383425, "deviation-center-line": 0.4844235659756217, "driven_lanedir_consec": 0.43144417737982144, "sim_compute_sim_state": 0.02211622256858676, "sim_compute_performance-ego": 0.037662111544141584, "sim_compute_robot_state-ego": 0.03955399382348154, "sim_compute_robot_state-npc0": 0.03921079542122635, "sim_compute_robot_state-npc1": 0.038388302746941064, "sim_compute_robot_state-npc2": 0.03835636494206447, "sim_compute_robot_state-npc3": 0.03850990183213178}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.93824410752437, "sim_physics": 0.09453424453735353, "survival_time": 14.950000000000076, "driven_lanedir": 0.8501649848378204, "sim_render-ego": 0.03479205369949341, "in-drivable-lane": 0.8500000000000121, "agent_compute-ego": 0.12091800848642983, "deviation-heading": 3.209014652501702, "set_robot_commands": 0.054774345556894935, "deviation-center-line": 0.8132426664665893, "driven_lanedir_consec": 0.8501649848378204, "sim_compute_sim_state": 0.02203951597213745, "sim_compute_performance-ego": 0.03732408364613851, "sim_compute_robot_state-ego": 0.03967433293660482, "sim_compute_robot_state-npc0": 0.03949718157450358, "sim_compute_robot_state-npc1": 0.03840140581130982, "sim_compute_robot_state-npc2": 0.038635469277699785, "sim_compute_robot_state-npc3": 0.03860084692637126}}
set_robot_commands_max0.05520692352176637
set_robot_commands_mean0.05451864156389544
set_robot_commands_median0.05443933763002094
set_robot_commands_min0.05408549706141154
sim_compute_performance-ego_max0.039237959440364394
sim_compute_performance-ego_mean0.03806335204907328
sim_compute_performance-ego_median0.037662111544141584
sim_compute_performance-ego_min0.03732408364613851
sim_compute_robot_state-ego_max0.04017306021017621
sim_compute_robot_state-ego_mean0.039878805095389755
sim_compute_robot_state-ego_median0.03991535822550456
sim_compute_robot_state-ego_min0.03955399382348154
sim_compute_robot_state-npc0_max0.0400051941243253
sim_compute_robot_state-npc0_mean0.03952394685699005
sim_compute_robot_state-npc0_median0.03949718157450358
sim_compute_robot_state-npc0_min0.03921079542122635
sim_compute_robot_state-npc1_max0.040530811575122346
sim_compute_robot_state-npc1_mean0.03899051241438368
sim_compute_robot_state-npc1_median0.038498940467834475
sim_compute_robot_state-npc1_min0.038388302746941064
sim_compute_robot_state-npc2_max0.038922836614209554
sim_compute_robot_state-npc2_mean0.03849649352328344
sim_compute_robot_state-npc2_median0.03836216688156128
sim_compute_robot_state-npc2_min0.03820562990088212
sim_compute_robot_state-npc3_max0.03937955279099314
sim_compute_robot_state-npc3_mean0.03880705235347573
sim_compute_robot_state-npc3_median0.03860084692637126
sim_compute_robot_state-npc3_min0.03850587606430054
sim_compute_sim_state_max0.02223327825235766
sim_compute_sim_state_mean0.02208280643518406
sim_compute_sim_state_median0.022063791303706348
sim_compute_sim_state_min0.02196122407913208
sim_physics_max0.09864480177561442
sim_physics_mean0.0928556256111032
sim_physics_median0.09088143549467388
sim_physics_min0.08998079262962637
sim_render-ego_max0.0363799113853305
sim_render-ego_mean0.03509688666813524
sim_render-ego_median0.03499367255573125
sim_render-ego_min0.03432492385233255
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean13.770000000000058
survival_time_min12.750000000000046
207053101awoo424HKU Duckietown Projectaido2-LFV-sim-validationstep1-simulationhost-errorno-0:14:48
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20705-946339', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20705-946339', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20705-946339', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
(hidden)