Duckietown Challenges Home Challenges Submissions

Submission 3094

Submission3094
Competingyes
Challengeaido2-LF-sim-testing
UserAngel Woo 🇭🇰
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22624
Next
User labelBaseline-IL-logs-tensorflow
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22624step1-simulationsuccessyes0:43:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6523061495015456
survival_time_median12.650000000000045
deviation-center-line_median0.5946778587674307
in-drivable-lane_median0


other stats
agent_compute-ego_max0.2097931990256676
agent_compute-ego_mean0.20644512680270757
agent_compute-ego_median0.2065522559483846
agent_compute-ego_min0.1996800148928607
deviation-center-line_max1.2178051376751096
deviation-center-line_mean0.6855564791566023
deviation-center-line_min0.283627348787723
deviation-heading_max3.674963758747008
deviation-heading_mean1.7399018713307062
deviation-heading_median1.7044193777552517
deviation-heading_min0.16965746995812037
driven_any_max1.2200010592902049
driven_any_mean0.8829100381869169
driven_any_median0.9955491007854684
driven_any_min0.18224935172653897
driven_lanedir_consec_max1.2173039017028584
driven_lanedir_consec_mean0.7599456838536602
driven_lanedir_consec_min0.17013176657597873
driven_lanedir_max1.2173039017028584
driven_lanedir_mean0.7599456838536602
driven_lanedir_median0.6523061495015456
driven_lanedir_min0.17013176657597873
in-drivable-lane_max4.350000000000062
in-drivable-lane_mean1.1166666666666825
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0207705991977636, "sim_physics": 0.060925042256712914, "survival_time": 12.800000000000049, "driven_lanedir": 0.6636396340244062, "sim_render-ego": 0.06293089222162962, "in-drivable-lane": 3.250000000000046, "agent_compute-ego": 0.20959973987191916, "deviation-heading": 3.674963758747008, "set_robot_commands": 0.07905749324709177, "deviation-center-line": 0.5688470948069573, "driven_lanedir_consec": 0.6636396340244062, "sim_compute_sim_state": 0.04256367404013872, "sim_compute_performance-ego": 0.06595547590404749, "sim_compute_robot_state-ego": 0.06587300263345242}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9940609238534442, "sim_physics": 0.06068859982678271, "survival_time": 12.700000000000044, "driven_lanedir": 0.8072101738551097, "sim_render-ego": 0.06460009972880205, "in-drivable-lane": 1.6500000000000234, "agent_compute-ego": 0.2088010104622428, "deviation-heading": 2.832547427697435, "set_robot_commands": 0.0791890790143351, "deviation-center-line": 0.7868030205809607, "driven_lanedir_consec": 0.8072101738551097, "sim_compute_sim_state": 0.04369183694283793, "sim_compute_performance-ego": 0.06784666241623286, "sim_compute_robot_state-ego": 0.06708754798558754}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.18224935172653897, "sim_physics": 0.06312138945968063, "survival_time": 2.6999999999999984, "driven_lanedir": 0.17013176657597873, "sim_render-ego": 0.06377567185295953, "in-drivable-lane": 0, "agent_compute-ego": 0.1996800148928607, "deviation-heading": 0.9343774825348196, "set_robot_commands": 0.07428594430287679, "deviation-center-line": 0.283627348787723, "driven_lanedir_consec": 0.17013176657597873, "sim_compute_sim_state": 0.04189200754518862, "sim_compute_performance-ego": 0.06702888453448261, "sim_compute_robot_state-ego": 0.0680124009097064}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.503508393358682, "sim_physics": 0.06032766012045054, "survival_time": 6.499999999999985, "driven_lanedir": 0.48552056760986906, "sim_render-ego": 0.06464732793661264, "in-drivable-lane": 0, "agent_compute-ego": 0.2097931990256676, "deviation-heading": 1.7044193777552517, "set_robot_commands": 0.07768769631019005, "deviation-center-line": 0.5044204827108707, "driven_lanedir_consec": 0.48552056760986906, "sim_compute_sim_state": 0.04251192716451792, "sim_compute_performance-ego": 0.06677268285017747, "sim_compute_robot_state-ego": 0.06551454617426945}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.2027774654668777, "sim_physics": 0.058070662021636965, "survival_time": 14.950000000000076, "driven_lanedir": 1.1780269912531711, "sim_render-ego": 0.06402987877527873, "in-drivable-lane": 0, "agent_compute-ego": 0.20880224148432416, "deviation-heading": 1.7990953498767968, "set_robot_commands": 0.07803671280543009, "deviation-center-line": 0.7303747391200331, "driven_lanedir_consec": 1.1780269912531711, "sim_compute_sim_state": 0.04256214459737142, "sim_compute_performance-ego": 0.06556555112202962, "sim_compute_robot_state-ego": 0.06373808622360229}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.4810428279701421, "sim_physics": 0.059297856830415274, "survival_time": 6.299999999999986, "driven_lanedir": 0.4578663053041683, "sim_render-ego": 0.06080844288780576, "in-drivable-lane": 0, "agent_compute-ego": 0.20775327796027773, "deviation-heading": 1.939095142563667, "set_robot_commands": 0.07615298884255546, "deviation-center-line": 0.49332729135590886, "driven_lanedir_consec": 0.4578663053041683, "sim_compute_sim_state": 0.04116091084858728, "sim_compute_performance-ego": 0.06211618771628728, "sim_compute_robot_state-ego": 0.06372166815258208}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.9955491007854684, "sim_physics": 0.0564838435810074, "survival_time": 12.650000000000045, "driven_lanedir": 0.6159985806729336, "sim_render-ego": 0.06093152615392632, "in-drivable-lane": 4.350000000000062, "agent_compute-ego": 0.20571921182715375, "deviation-heading": 1.850254300868432, "set_robot_commands": 0.07690573115593831, "deviation-center-line": 0.5013964133210531, "driven_lanedir_consec": 0.6159985806729336, "sim_compute_sim_state": 0.04183679324364945, "sim_compute_performance-ego": 0.06366274196639833, "sim_compute_robot_state-ego": 0.06351657818428612}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.2889931353427513, "sim_physics": 0.05804152488708496, "survival_time": 3.999999999999994, "driven_lanedir": 0.28121171271941936, "sim_render-ego": 0.06196720898151398, "in-drivable-lane": 0, "agent_compute-ego": 0.20721740424633023, "deviation-heading": 0.9204387374489202, "set_robot_commands": 0.07813375294208527, "deviation-center-line": 0.43020454534709474, "driven_lanedir_consec": 0.28121171271941936, "sim_compute_sim_state": 0.04136331975460053, "sim_compute_performance-ego": 0.06418056488037109, "sim_compute_robot_state-ego": 0.0626752346754074}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.205281109718533, "sim_physics": 0.05669652144114176, "survival_time": 14.950000000000076, "driven_lanedir": 1.2028575181014998, "sim_render-ego": 0.0605112608273824, "in-drivable-lane": 0, "agent_compute-ego": 0.2046729294459025, "deviation-heading": 0.7095458329423281, "set_robot_commands": 0.07750153462092081, "deviation-center-line": 0.9600869493681972, "driven_lanedir_consec": 1.2028575181014998, "sim_compute_sim_state": 0.040570794741312664, "sim_compute_performance-ego": 0.06248031218846639, "sim_compute_robot_state-ego": 0.06337752978006998}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.9976492401276388, "sim_physics": 0.05829899624524364, "survival_time": 12.550000000000043, "driven_lanedir": 0.6480670198840939, "sim_render-ego": 0.06274747563548297, "in-drivable-lane": 3.2000000000000455, "agent_compute-ego": 0.2052507713971385, "deviation-heading": 3.015507728112178, "set_robot_commands": 0.07819761128064645, "deviation-center-line": 0.680559448578591, "driven_lanedir_consec": 0.6480670198840939, "sim_compute_sim_state": 0.04153908581372751, "sim_compute_performance-ego": 0.06423665042892396, "sim_compute_robot_state-ego": 0.06277924514861696}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.2064159917481403, "sim_physics": 0.05864277998606364, "survival_time": 14.950000000000076, "driven_lanedir": 1.1814130224728343, "sim_render-ego": 0.0631562320391337, "in-drivable-lane": 0, "agent_compute-ego": 0.2065522559483846, "deviation-heading": 1.274610115556399, "set_robot_commands": 0.0770912233988444, "deviation-center-line": 1.0532448941856924, "driven_lanedir_consec": 1.1814130224728343, "sim_compute_sim_state": 0.04192339897155762, "sim_compute_performance-ego": 0.06404273509979248, "sim_compute_robot_state-ego": 0.06364928722381592}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.9939432764337164, "sim_physics": 0.0589441801271131, "survival_time": 12.40000000000004, "driven_lanedir": 0.6523061495015456, "sim_render-ego": 0.06263766173393495, "in-drivable-lane": 3.1500000000000448, "agent_compute-ego": 0.20811323773476384, "deviation-heading": 2.9028019703638512, "set_robot_commands": 0.076598277976436, "deviation-center-line": 0.5946778587674307, "driven_lanedir_consec": 0.6523061495015456, "sim_compute_sim_state": 0.04113972187042236, "sim_compute_performance-ego": 0.06308013489169459, "sim_compute_robot_state-ego": 0.06322244578792204}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.7509928569294946, "sim_physics": 0.05663307466005024, "survival_time": 9.5, "driven_lanedir": 0.6373657134532887, "sim_render-ego": 0.06260005298413729, "in-drivable-lane": 1.1500000000000163, "agent_compute-ego": 0.2055214718768471, "deviation-heading": 1.6756527884125696, "set_robot_commands": 0.07768540507868717, "deviation-center-line": 0.4723512987738168, "driven_lanedir_consec": 0.6373657134532887, "sim_compute_sim_state": 0.04237830011468185, "sim_compute_performance-ego": 0.06364208899046245, "sim_compute_robot_state-ego": 0.06366227551510459}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 1.2200010592902049, "sim_physics": 0.05908202966054281, "survival_time": 14.950000000000076, "driven_lanedir": 1.2173039017028584, "sim_render-ego": 0.06492081244786581, "in-drivable-lane": 0, "agent_compute-ego": 0.20646447579065955, "deviation-heading": 0.695560587122821, "set_robot_commands": 0.07788645346959432, "deviation-center-line": 1.0056206639695962, "driven_lanedir_consec": 1.2173039017028584, "sim_compute_sim_state": 0.04315885861714681, "sim_compute_performance-ego": 0.06637797832489013, "sim_compute_robot_state-ego": 0.06490856409072876}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 1.200415240854356, "sim_physics": 0.05722566843032837, "survival_time": 14.950000000000076, "driven_lanedir": 1.2002662006737288, "sim_render-ego": 0.06445024251937866, "in-drivable-lane": 0, "agent_compute-ego": 0.20273566007614136, "deviation-heading": 0.16965746995812037, "set_robot_commands": 0.0768631911277771, "deviation-center-line": 1.2178051376751096, "driven_lanedir_consec": 1.2002662006737288, "sim_compute_sim_state": 0.04255724271138509, "sim_compute_performance-ego": 0.06641220410664876, "sim_compute_robot_state-ego": 0.06387273867925009}}
set_robot_commands_max0.0791890790143351
set_robot_commands_mean0.07741820637156059
set_robot_commands_median0.07768540507868717
set_robot_commands_min0.07428594430287679
sim_compute_performance-ego_max0.06784666241623286
sim_compute_performance-ego_mean0.06489339036139369
sim_compute_performance-ego_median0.06423665042892396
sim_compute_performance-ego_min0.06211618771628728
sim_compute_robot_state-ego_max0.0680124009097064
sim_compute_robot_state-ego_mean0.06437407674429348
sim_compute_robot_state-ego_median0.06372166815258208
sim_compute_robot_state-ego_min0.0626752346754074
sim_compute_sim_state_max0.04369183694283793
sim_compute_sim_state_mean0.04205666779847504
sim_compute_sim_state_median0.04192339897155762
sim_compute_sim_state_min0.040570794741312664
sim_physics_max0.06312138945968063
sim_physics_mean0.058831988635617
sim_physics_median0.05864277998606364
sim_physics_min0.0564838435810074
sim_render-ego_max0.06492081244786581
sim_render-ego_mean0.06298098578172297
sim_render-ego_median0.06293089222162962
sim_render-ego_min0.0605112608273824
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean11.12333333333337
survival_time_min2.6999999999999984
No reset possible
20863step1-simulationsuccessno0:57:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20828step1-simulationhost-errorno0:35:44
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-testing/submission3094/step1-simulation-ip-172-31-42-167-7357-job20828/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20796step1-simulationhost-errorno0:39:04
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-testing/submission3094/step1-simulation-ip-172-31-38-104-5197-job20796/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20738step1-simulationtimeoutno1:05:25
I can see how the jo [...]
I can see how the job 20738 is timeout because passed 3925 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20720step1-simulationhost-errorno0:03:50
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 996, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20720-484599', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 579, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 905, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 591, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1004, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20720-484599', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20720-484599', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20697step1-simulationsuccessno0:11:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20694step1-simulationhost-errorno0:08:13
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20694-761615', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20694-761615', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20694-761615', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible