Duckietown Challenges Home Challenges Submissions

Submission 3368

Submission3368
Competingyes
Challengeaido2-LFV-sim-validation
UserAmaury Camus 🇨🇭
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22370
Next
User labelchallenge-aido_LF-template-random
Admin priority50
Blessingn/a
User priority50

22370

Click the images to see detailed statistics about the episode.

ETHZ_autolab_technical_track-0-0

ETHZ_autolab_technical_track-1-0

ETHZ_autolab_technical_track-2-0

ETHZ_autolab_technical_track-3-0

ETHZ_autolab_technical_track-4-0

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22370step1-simulationsuccessyes0:08:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.6999999999999984
deviation-center-line_median0.11519544839797546
in-drivable-lane_median0.5999999999999979


other stats
agent_compute-ego_max0.1653811264038086
agent_compute-ego_mean0.14275720167310305
agent_compute-ego_median0.1357676469827
agent_compute-ego_min0.1289552961077009
deviation-center-line_max0.1743913456502906
deviation-center-line_mean0.12651756552531995
deviation-center-line_min0.09047885846140293
deviation-heading_max0.5020407942599273
deviation-heading_mean0.3641979127291371
deviation-heading_median0.34764983431811214
deviation-heading_min0.19401296908406307
driven_any_max1.6169932854589035
driven_any_mean1.0272530755098266
driven_any_median1.0243289631429708
driven_any_min0.4119037580118998
driven_lanedir_consec_max1.2433038249708377
driven_lanedir_consec_mean0.7800316850006626
driven_lanedir_consec_min0.3792793194470208
driven_lanedir_max1.2433038249708377
driven_lanedir_mean0.7800316850006626
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.3792793194470208
in-drivable-lane_max0.9999999999999976
in-drivable-lane_mean0.4899999999999984
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.1516671578089396, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.05554395693319815, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.13391990131802028, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.09441771772172716, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.036886758274502225, "sim_compute_performance-ego": 0.06259040479306821, "sim_compute_robot_state-ego": 0.06976095835367839, "sim_compute_robot_state-npc0": 0.06653802041654233, "sim_compute_robot_state-npc1": 0.05873954737627947, "sim_compute_robot_state-npc2": 0.059659617918509024, "sim_compute_robot_state-npc3": 0.060559873227719906}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4119037580118998, "sim_physics": 0.21529409408569336, "survival_time": 1.2500000000000004, "driven_lanedir": 0.3792793194470208, "sim_render-ego": 0.0652924919128418, "in-drivable-lane": 0, "agent_compute-ego": 0.1653811264038086, "deviation-heading": 0.47860435001913393, "set_robot_commands": 0.10315236091613768, "deviation-center-line": 0.09344288799285194, "driven_lanedir_consec": 0.3792793194470208, "sim_compute_sim_state": 0.041140804290771486, "sim_compute_performance-ego": 0.07150863647460938, "sim_compute_robot_state-ego": 0.08774508476257324, "sim_compute_robot_state-npc0": 0.07220564842224121, "sim_compute_robot_state-npc1": 0.0773888111114502, "sim_compute_robot_state-npc2": 0.07594542503356934, "sim_compute_robot_state-npc3": 0.07374726295471191}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.579862082650716, "sim_physics": 0.17924976976294266, "survival_time": 3.7999999999999945, "driven_lanedir": 1.2433038249708377, "sim_render-ego": 0.060702392929478696, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.1497620375532853, "deviation-heading": 0.5020407942599273, "set_robot_commands": 0.09661884998020372, "deviation-center-line": 0.1743913456502906, "driven_lanedir_consec": 1.2433038249708377, "sim_compute_sim_state": 0.0399524914590936, "sim_compute_performance-ego": 0.07223416629590486, "sim_compute_robot_state-ego": 0.07397449016571045, "sim_compute_robot_state-npc0": 0.07071870565414429, "sim_compute_robot_state-npc1": 0.06944392543090017, "sim_compute_robot_state-npc2": 0.06822553433870014, "sim_compute_robot_state-npc3": 0.0686252148527848}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6169932854589035, "sim_physics": 0.15764643874349474, "survival_time": 3.949999999999994, "driven_lanedir": 1.2126117310470583, "sim_render-ego": 0.05640117729766459, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.1357676469827, "deviation-heading": 0.34764983431811214, "set_robot_commands": 0.0818163050880915, "deviation-center-line": 0.11519544839797546, "driven_lanedir_consec": 1.2126117310470583, "sim_compute_sim_state": 0.03468578978429867, "sim_compute_performance-ego": 0.06297004675563378, "sim_compute_robot_state-ego": 0.06797525248950041, "sim_compute_robot_state-npc0": 0.06250819978834707, "sim_compute_robot_state-npc1": 0.06619101838220524, "sim_compute_robot_state-npc2": 0.05763191814664044, "sim_compute_robot_state-npc3": 0.06263548513002033}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5031772882846431, "sim_physics": 0.1702117919921875, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4926325270527603, "sim_render-ego": 0.056707186358315606, "in-drivable-lane": 0, "agent_compute-ego": 0.1289552961077009, "deviation-heading": 0.29868161596444953, "set_robot_commands": 0.07796416112354823, "deviation-center-line": 0.09047885846140293, "driven_lanedir_consec": 0.4926325270527603, "sim_compute_sim_state": 0.03794162614004953, "sim_compute_performance-ego": 0.056555637291499546, "sim_compute_robot_state-ego": 0.06111586093902588, "sim_compute_robot_state-npc0": 0.05672934225627354, "sim_compute_robot_state-npc1": 0.06067251307623727, "sim_compute_robot_state-npc2": 0.058712473937443325, "sim_compute_robot_state-npc3": 0.07422083616256714}}
set_robot_commands_max0.10315236091613768
set_robot_commands_mean0.09079387896594168
set_robot_commands_median0.09441771772172716
set_robot_commands_min0.07796416112354823
sim_compute_performance-ego_max0.07223416629590486
sim_compute_performance-ego_mean0.06517177832214315
sim_compute_performance-ego_median0.06297004675563378
sim_compute_performance-ego_min0.056555637291499546
sim_compute_robot_state-ego_max0.08774508476257324
sim_compute_robot_state-ego_mean0.07211432934209767
sim_compute_robot_state-ego_median0.06976095835367839
sim_compute_robot_state-ego_min0.06111586093902588
sim_compute_robot_state-npc0_max0.07220564842224121
sim_compute_robot_state-npc0_mean0.06573998330750969
sim_compute_robot_state-npc0_median0.06653802041654233
sim_compute_robot_state-npc0_min0.05672934225627354
sim_compute_robot_state-npc1_max0.0773888111114502
sim_compute_robot_state-npc1_mean0.06648716307541447
sim_compute_robot_state-npc1_median0.06619101838220524
sim_compute_robot_state-npc1_min0.05873954737627947
sim_compute_robot_state-npc2_max0.07594542503356934
sim_compute_robot_state-npc2_mean0.06403499387497245
sim_compute_robot_state-npc2_median0.059659617918509024
sim_compute_robot_state-npc2_min0.05763191814664044
sim_compute_robot_state-npc3_max0.07422083616256714
sim_compute_robot_state-npc3_mean0.06795773446556082
sim_compute_robot_state-npc3_median0.0686252148527848
sim_compute_robot_state-npc3_min0.060559873227719906
sim_compute_sim_state_max0.041140804290771486
sim_compute_sim_state_mean0.038121493989743106
sim_compute_sim_state_median0.03794162614004953
sim_compute_sim_state_min0.03468578978429867
sim_physics_max0.21529409408569336
sim_physics_mean0.17481385047865156
sim_physics_median0.1702117919921875
sim_physics_min0.1516671578089396
sim_render-ego_max0.0652924919128418
sim_render-ego_mean0.05892944108629977
sim_render-ego_median0.056707186358315606
sim_render-ego_min0.05554395693319815
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.6199999999999974
survival_time_min1.2500000000000004
No reset possible
21983step1-simulationsuccessno0:04:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21977step1-simulationhost-errorno0:00:37
Error while running [...]

stderr | Pulling solution       ... 
stderr | Pulling scenario_maker ... 
stderr | Pulling evaluator      ... done
stderr | 
stderr | ERROR: for simulator  error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/3e/3ecc0062b0ecc477fa7f260c0ab6d4bd752315fbabf8a5895e0aecd90fb7a4a3/data?verify=1557769588-AYZIrW%2B5WrMR2loeW93UjmlO%2BPc%3D: net/http: TLS handshake timeout
stderr | error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/3e/3ecc0062b0ecc477fa7f260c0ab6d4bd752315fbabf8a5895e0aecd90fb7a4a3/data?verify=1557769588-AYZIrW%2B5WrMR2loeW93UjmlO%2BPc%3D: net/http: TLS handshake timeout
stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21976step1-simulationhost-errorno0:04:43
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 1066, in run_docker
    subprocess.check_call(cmd0, cwd=cwd, stdout=tee_stdout, stderr=tee_stderr)
  File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job21976-156474', 'down']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 535, in get_cr
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1071, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job21976-156474', 'down']:

   >  Command '['docker-compose', '-p', 'job21976-156474', 'down']' returned non-zero exit status 1.

stdout | 

stderr | 
stderr |         Can't find a suitable configuration file in this directory or any
stderr |         parent. Are you in the right directory?
stderr | 
stderr |         Supported filenames: docker-compose.yml, docker-compose.yaml
stderr | 
stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21948step1-simulationtimeoutno1:05:01
I can see how the jo [...]
I can see how the job 21948 is timeout because passed 3901 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible