AI Driving Olympics Home Challenges Submissions Jobs

Submission 3368

Submission3368
Competingyes
Challengeaido2-LFV-sim-validation
UserAmaurX
Date submitted
Completecomplete
Resultsuccess
Jobsstep1-simulation: 22370
Next
User labelchallenge-aido_LF-template-random
Admin priority50
User priority50

22370

Click the images to see detailed statistics about the episode.

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
223703368AmaurXchallenge-aido_LF-template-randomaido2-LFV-sim-validationstep1-simulationsuccessyes-0:08:06(hidden)
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.6999999999999984
deviation-center-line_median0.11519544839797546
in-drivable-lane_median0.5999999999999979


other stats
agent_compute-ego_max0.1653811264038086
agent_compute-ego_mean0.14275720167310305
agent_compute-ego_median0.1357676469827
agent_compute-ego_min0.1289552961077009
deviation-center-line_max0.1743913456502906
deviation-center-line_mean0.12651756552531995
deviation-center-line_min0.09047885846140293
deviation-heading_max0.5020407942599273
deviation-heading_mean0.3641979127291371
deviation-heading_median0.34764983431811214
deviation-heading_min0.19401296908406307
driven_any_max1.6169932854589035
driven_any_mean1.0272530755098266
driven_any_median1.0243289631429708
driven_any_min0.4119037580118998
driven_lanedir_consec_max1.2433038249708377
driven_lanedir_consec_mean0.7800316850006626
driven_lanedir_consec_min0.3792793194470208
driven_lanedir_max1.2433038249708377
driven_lanedir_mean0.7800316850006626
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.3792793194470208
in-drivable-lane_max0.9999999999999976
in-drivable-lane_mean0.4899999999999984
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.1516671578089396, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.05554395693319815, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.13391990131802028, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.09441771772172716, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.036886758274502225, "sim_compute_performance-ego": 0.06259040479306821, "sim_compute_robot_state-ego": 0.06976095835367839, "sim_compute_robot_state-npc0": 0.06653802041654233, "sim_compute_robot_state-npc1": 0.05873954737627947, "sim_compute_robot_state-npc2": 0.059659617918509024, "sim_compute_robot_state-npc3": 0.060559873227719906}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4119037580118998, "sim_physics": 0.21529409408569336, "survival_time": 1.2500000000000004, "driven_lanedir": 0.3792793194470208, "sim_render-ego": 0.0652924919128418, "in-drivable-lane": 0, "agent_compute-ego": 0.1653811264038086, "deviation-heading": 0.47860435001913393, "set_robot_commands": 0.10315236091613768, "deviation-center-line": 0.09344288799285194, "driven_lanedir_consec": 0.3792793194470208, "sim_compute_sim_state": 0.041140804290771486, "sim_compute_performance-ego": 0.07150863647460938, "sim_compute_robot_state-ego": 0.08774508476257324, "sim_compute_robot_state-npc0": 0.07220564842224121, "sim_compute_robot_state-npc1": 0.0773888111114502, "sim_compute_robot_state-npc2": 0.07594542503356934, "sim_compute_robot_state-npc3": 0.07374726295471191}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.579862082650716, "sim_physics": 0.17924976976294266, "survival_time": 3.7999999999999945, "driven_lanedir": 1.2433038249708377, "sim_render-ego": 0.060702392929478696, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.1497620375532853, "deviation-heading": 0.5020407942599273, "set_robot_commands": 0.09661884998020372, "deviation-center-line": 0.1743913456502906, "driven_lanedir_consec": 1.2433038249708377, "sim_compute_sim_state": 0.0399524914590936, "sim_compute_performance-ego": 0.07223416629590486, "sim_compute_robot_state-ego": 0.07397449016571045, "sim_compute_robot_state-npc0": 0.07071870565414429, "sim_compute_robot_state-npc1": 0.06944392543090017, "sim_compute_robot_state-npc2": 0.06822553433870014, "sim_compute_robot_state-npc3": 0.0686252148527848}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6169932854589035, "sim_physics": 0.15764643874349474, "survival_time": 3.949999999999994, "driven_lanedir": 1.2126117310470583, "sim_render-ego": 0.05640117729766459, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.1357676469827, "deviation-heading": 0.34764983431811214, "set_robot_commands": 0.0818163050880915, "deviation-center-line": 0.11519544839797546, "driven_lanedir_consec": 1.2126117310470583, "sim_compute_sim_state": 0.03468578978429867, "sim_compute_performance-ego": 0.06297004675563378, "sim_compute_robot_state-ego": 0.06797525248950041, "sim_compute_robot_state-npc0": 0.06250819978834707, "sim_compute_robot_state-npc1": 0.06619101838220524, "sim_compute_robot_state-npc2": 0.05763191814664044, "sim_compute_robot_state-npc3": 0.06263548513002033}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5031772882846431, "sim_physics": 0.1702117919921875, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4926325270527603, "sim_render-ego": 0.056707186358315606, "in-drivable-lane": 0, "agent_compute-ego": 0.1289552961077009, "deviation-heading": 0.29868161596444953, "set_robot_commands": 0.07796416112354823, "deviation-center-line": 0.09047885846140293, "driven_lanedir_consec": 0.4926325270527603, "sim_compute_sim_state": 0.03794162614004953, "sim_compute_performance-ego": 0.056555637291499546, "sim_compute_robot_state-ego": 0.06111586093902588, "sim_compute_robot_state-npc0": 0.05672934225627354, "sim_compute_robot_state-npc1": 0.06067251307623727, "sim_compute_robot_state-npc2": 0.058712473937443325, "sim_compute_robot_state-npc3": 0.07422083616256714}}
set_robot_commands_max0.10315236091613768
set_robot_commands_mean0.09079387896594168
set_robot_commands_median0.09441771772172716
set_robot_commands_min0.07796416112354823
sim_compute_performance-ego_max0.07223416629590486
sim_compute_performance-ego_mean0.06517177832214315
sim_compute_performance-ego_median0.06297004675563378
sim_compute_performance-ego_min0.056555637291499546
sim_compute_robot_state-ego_max0.08774508476257324
sim_compute_robot_state-ego_mean0.07211432934209767
sim_compute_robot_state-ego_median0.06976095835367839
sim_compute_robot_state-ego_min0.06111586093902588
sim_compute_robot_state-npc0_max0.07220564842224121
sim_compute_robot_state-npc0_mean0.06573998330750969
sim_compute_robot_state-npc0_median0.06653802041654233
sim_compute_robot_state-npc0_min0.05672934225627354
sim_compute_robot_state-npc1_max0.0773888111114502
sim_compute_robot_state-npc1_mean0.06648716307541447
sim_compute_robot_state-npc1_median0.06619101838220524
sim_compute_robot_state-npc1_min0.05873954737627947
sim_compute_robot_state-npc2_max0.07594542503356934
sim_compute_robot_state-npc2_mean0.06403499387497245
sim_compute_robot_state-npc2_median0.059659617918509024
sim_compute_robot_state-npc2_min0.05763191814664044
sim_compute_robot_state-npc3_max0.07422083616256714
sim_compute_robot_state-npc3_mean0.06795773446556082
sim_compute_robot_state-npc3_median0.0686252148527848
sim_compute_robot_state-npc3_min0.060559873227719906
sim_compute_sim_state_max0.041140804290771486
sim_compute_sim_state_mean0.038121493989743106
sim_compute_sim_state_median0.03794162614004953
sim_compute_sim_state_min0.03468578978429867
sim_physics_max0.21529409408569336
sim_physics_mean0.17481385047865156
sim_physics_median0.1702117919921875
sim_physics_min0.1516671578089396
sim_render-ego_max0.0652924919128418
sim_render-ego_mean0.05892944108629977
sim_render-ego_median0.056707186358315606
sim_render-ego_min0.05554395693319815
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.6199999999999974
survival_time_min1.2500000000000004
219833368AmaurXchallenge-aido_LF-template-randomaido2-LFV-sim-validationstep1-simulationsuccessno-0:04:22(hidden)
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.6999999999999984
deviation-center-line_median0.11519544839797546
in-drivable-lane_median0.5999999999999979


other stats
agent_compute-ego_max0.07715464543692674
agent_compute-ego_mean0.07640464042491335
agent_compute-ego_median0.07636493130734093
agent_compute-ego_min0.07579853057861329
deviation-center-line_max0.1743913456502906
deviation-center-line_mean0.12651756552531995
deviation-center-line_min0.09047885846140293
deviation-heading_max0.5020407942599273
deviation-heading_mean0.3641979127291371
deviation-heading_median0.34764983431811214
deviation-heading_min0.19401296908406307
driven_any_max1.6169932854589035
driven_any_mean1.0272530755098266
driven_any_median1.0243289631429708
driven_any_min0.4119037580118998
driven_lanedir_consec_max1.2433038249708377
driven_lanedir_consec_mean0.7800316850006626
driven_lanedir_consec_min0.3792793194470208
driven_lanedir_max1.2433038249708377
driven_lanedir_mean0.7800316850006626
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.3792793194470208
in-drivable-lane_max0.9999999999999976
in-drivable-lane_mean0.4899999999999984
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.060980403864825214, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.03179779759159795, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.07671845400774921, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.04927931008515535, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.020150180216188782, "sim_compute_performance-ego": 0.03500182098812527, "sim_compute_robot_state-ego": 0.03764016098446316, "sim_compute_robot_state-npc0": 0.03579678358855071, "sim_compute_robot_state-npc1": 0.03560343936637596, "sim_compute_robot_state-npc2": 0.03580268224080404, "sim_compute_robot_state-npc3": 0.03583289075780798}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4119037580118998, "sim_physics": 0.06833473205566407, "survival_time": 1.2500000000000004, "driven_lanedir": 0.3792793194470208, "sim_render-ego": 0.03190269470214844, "in-drivable-lane": 0, "agent_compute-ego": 0.07579853057861329, "deviation-heading": 0.47860435001913393, "set_robot_commands": 0.049201040267944335, "deviation-center-line": 0.09344288799285194, "driven_lanedir_consec": 0.3792793194470208, "sim_compute_sim_state": 0.01998880386352539, "sim_compute_performance-ego": 0.03470266342163086, "sim_compute_robot_state-ego": 0.03763143539428711, "sim_compute_robot_state-npc0": 0.03583529472351074, "sim_compute_robot_state-npc1": 0.03612822532653809, "sim_compute_robot_state-npc2": 0.03559236526489258, "sim_compute_robot_state-npc3": 0.0355788803100586}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.579862082650716, "sim_physics": 0.06378529573741712, "survival_time": 3.7999999999999945, "driven_lanedir": 1.2433038249708377, "sim_render-ego": 0.03180269818556936, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.07636493130734093, "deviation-heading": 0.5020407942599273, "set_robot_commands": 0.04940810015327052, "deviation-center-line": 0.1743913456502906, "driven_lanedir_consec": 1.2433038249708377, "sim_compute_sim_state": 0.02005777547233983, "sim_compute_performance-ego": 0.03520538618690089, "sim_compute_robot_state-ego": 0.03779346064517373, "sim_compute_robot_state-npc0": 0.035816584762774016, "sim_compute_robot_state-npc1": 0.03583442223699469, "sim_compute_robot_state-npc2": 0.03556185960769653, "sim_compute_robot_state-npc3": 0.03556062045850252}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6169932854589035, "sim_physics": 0.06463404546810102, "survival_time": 3.949999999999994, "driven_lanedir": 1.2126117310470583, "sim_render-ego": 0.03204642971859702, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.07715464543692674, "deviation-heading": 0.34764983431811214, "set_robot_commands": 0.05005577546131762, "deviation-center-line": 0.11519544839797546, "driven_lanedir_consec": 1.2126117310470583, "sim_compute_sim_state": 0.02072787586646744, "sim_compute_performance-ego": 0.03526680982565578, "sim_compute_robot_state-ego": 0.03726985786534563, "sim_compute_robot_state-npc0": 0.036473222925693174, "sim_compute_robot_state-npc1": 0.03599651553962804, "sim_compute_robot_state-npc2": 0.0359797477722168, "sim_compute_robot_state-npc3": 0.03636809844004957}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5031772882846431, "sim_physics": 0.061601485524858744, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4926325270527603, "sim_render-ego": 0.03177774803979056, "in-drivable-lane": 0, "agent_compute-ego": 0.07598664079393659, "deviation-heading": 0.29868161596444953, "set_robot_commands": 0.04920663152422224, "deviation-center-line": 0.09047885846140293, "driven_lanedir_consec": 0.4926325270527603, "sim_compute_sim_state": 0.019973022597176687, "sim_compute_performance-ego": 0.034645301955086846, "sim_compute_robot_state-ego": 0.037401837962014337, "sim_compute_robot_state-npc0": 0.036188406603676934, "sim_compute_robot_state-npc1": 0.035682235445295064, "sim_compute_robot_state-npc2": 0.03539893456867763, "sim_compute_robot_state-npc3": 0.03558628899710519}}
set_robot_commands_max0.05005577546131762
set_robot_commands_mean0.04943017149838201
set_robot_commands_median0.04927931008515535
set_robot_commands_min0.049201040267944335
sim_compute_performance-ego_max0.03526680982565578
sim_compute_performance-ego_mean0.03496439647547993
sim_compute_performance-ego_median0.03500182098812527
sim_compute_performance-ego_min0.034645301955086846
sim_compute_robot_state-ego_max0.03779346064517373
sim_compute_robot_state-ego_mean0.0375473505702568
sim_compute_robot_state-ego_median0.03763143539428711
sim_compute_robot_state-ego_min0.03726985786534563
sim_compute_robot_state-npc0_max0.036473222925693174
sim_compute_robot_state-npc0_mean0.036022058520841116
sim_compute_robot_state-npc0_median0.03583529472351074
sim_compute_robot_state-npc0_min0.03579678358855071
sim_compute_robot_state-npc1_max0.03612822532653809
sim_compute_robot_state-npc1_mean0.035848967582966365
sim_compute_robot_state-npc1_median0.03583442223699469
sim_compute_robot_state-npc1_min0.03560343936637596
sim_compute_robot_state-npc2_max0.0359797477722168
sim_compute_robot_state-npc2_mean0.03566711789085751
sim_compute_robot_state-npc2_median0.03559236526489258
sim_compute_robot_state-npc2_min0.03539893456867763
sim_compute_robot_state-npc3_max0.03636809844004957
sim_compute_robot_state-npc3_mean0.035785355792704775
sim_compute_robot_state-npc3_median0.03558628899710519
sim_compute_robot_state-npc3_min0.03556062045850252
sim_compute_sim_state_max0.02072787586646744
sim_compute_sim_state_mean0.020179531603139626
sim_compute_sim_state_median0.02005777547233983
sim_compute_sim_state_min0.019973022597176687
sim_physics_max0.06833473205566407
sim_physics_mean0.06386719253017323
sim_physics_median0.06378529573741712
sim_physics_min0.060980403864825214
sim_render-ego_max0.03204642971859702
sim_render-ego_mean0.031865473647540665
sim_render-ego_median0.03180269818556936
sim_render-ego_min0.03177774803979056
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.6199999999999974
survival_time_min1.2500000000000004
219773368AmaurXchallenge-aido_LF-template-randomaido2-LFV-sim-validationstep1-simulationhost-errorno-0:00:37
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job21977-581965', 'pull']:

   >  Command '['docker-compose', '-p', 'job21977-581965', 'pull']' returned non-zero exit status 1.

stdout | 

stderr | Some networks were defined but are not used by any service: evaluation
stderr | Pulling evaluator      ...
stderr | Pulling scenario_maker ...
stderr | Pulling simulator      ...
stderr | Pulling solution       ...
stderr | 
stderr | Pulling solution       ... done
stderr | 
stderr | Pulling scenario_maker ... done
stderr | 
stderr | Pulling evaluator      ... done
stderr | 
stderr | ERROR: for simulator  error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/3e/3ecc0062b0ecc477fa7f260c0ab6d4bd752315fbabf8a5895e0aecd90fb7a4a3/data?verify=1557769588-AYZIrW%2B5WrMR2loeW93UjmlO%2BPc%3D: net/http: TLS handshake timeout
stderr | error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/3e/3ecc0062b0ecc477fa7f260c0ab6d4bd752315fbabf8a5895e0aecd90fb7a4a3/data?verify=1557769588-AYZIrW%2B5WrMR2loeW93UjmlO%2BPc%3D: net/http: TLS handshake timeout
stderr | 
(hidden)
219763368AmaurXchallenge-aido_LF-template-randomaido2-LFV-sim-validationstep1-simulationhost-errorno-0:04:43
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 1066, in run_docker
    subprocess.check_call(cmd0, cwd=cwd, stdout=tee_stdout, stderr=tee_stderr)
  File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job21976-156474', 'down']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 535, in get_cr
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1071, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job21976-156474', 'down']:

   >  Command '['docker-compose', '-p', 'job21976-156474', 'down']' returned non-zero exit status 1.

stdout | 

stderr | 
stderr |         Can't find a suitable configuration file in this directory or any
stderr |         parent. Are you in the right directory?
stderr | 
stderr |         Supported filenames: docker-compose.yml, docker-compose.yaml
stderr | 
stderr | 
(hidden)
219483368AmaurXchallenge-aido_LF-template-randomaido2-LFV-sim-validationstep1-simulationtimeoutno-1:05:01
I can see how the jo [...]
I can see how the job 21948 is timeout because passed 3901 seconds and the timeout is 3600.0.
(hidden)