Duckietown Challenges Home Challenges Submissions

Submission 2637

Submission2637
Competingyes
Challengeaido2-LFVI-sim-validation
UserAndrea Censi 🇨🇭
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22844
Next
User labelchallenge-aido_LF-template-random
Admin priority50
Blessingn/a
User priority50

22844

Click the images to see detailed statistics about the episode.

udem1-0-0

udem1-1-0

udem1-2-0

udem1-3-0

udem1-4-0

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22844step1-simulationsuccessyes0:10:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.493517638549009
survival_time_median2.3999999999999995
deviation-center-line_median0.09791133281082
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.17673830758957637
agent_compute-ego_mean0.16762479697077787
agent_compute-ego_median0.1648905790304836
agent_compute-ego_min0.16171585662024365
deviation-center-line_max0.2348196785686717
deviation-center-line_mean0.13547481992266058
deviation-center-line_min0.06216029860587694
deviation-heading_max0.8115272458314897
deviation-heading_mean0.49959867397150787
deviation-heading_median0.46504862964833654
deviation-heading_min0.30044835691402494
driven_any_max2.1864003300000694
driven_any_mean1.1068208197848532
driven_any_median0.9410513078477724
driven_any_min0.34749292376309954
driven_lanedir_consec_max1.826231945363564
driven_lanedir_consec_mean0.804744013449103
driven_lanedir_consec_min0.3325249708009643
driven_lanedir_max1.826231945363564
driven_lanedir_mean0.804828147320482
driven_lanedir_median0.493517638549009
driven_lanedir_min0.3325249708009643
in-drivable-lane_max1.3499999999999952
in-drivable-lane_mean0.5599999999999984
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.2224725439578672, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.06505466714690003, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.1648905790304836, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.10464615761479244, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.04293983193892467, "sim_compute_performance-ego": 0.07435587388050707, "sim_compute_robot_state-ego": 0.07775033576579034, "sim_compute_robot_state-npc0": 0.07306730596325066, "sim_compute_robot_state-npc1": 0.07342689550375636, "sim_compute_robot_state-npc2": 0.07193746747849863, "sim_compute_robot_state-npc3": 0.07483788381648969}, "udem1-1-0": {"driven_any": 0.47798123795353353, "sim_physics": 0.23810440301895144, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.06949561834335327, "in-drivable-lane": 0, "agent_compute-ego": 0.16171585662024365, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.10622424738747734, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.04189591748373849, "sim_compute_performance-ego": 0.07581346375601632, "sim_compute_robot_state-ego": 0.08009161267961774, "sim_compute_robot_state-npc0": 0.07443892955780029, "sim_compute_robot_state-npc1": 0.07795044354030065, "sim_compute_robot_state-npc2": 0.07600026471274239, "sim_compute_robot_state-npc3": 0.07640754325049263}, "udem1-2-0": {"driven_any": 2.1864003300000694, "sim_physics": 0.2146683086469335, "survival_time": 5.14999999999999, "driven_lanedir": 1.826231945363564, "sim_render-ego": 0.06503073914537152, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.1622748861035097, "deviation-heading": 0.8115272458314897, "set_robot_commands": 0.10349648670085423, "deviation-center-line": 0.1927316041609401, "driven_lanedir_consec": 1.826231945363564, "sim_compute_sim_state": 0.04168602563802478, "sim_compute_performance-ego": 0.07352561163670808, "sim_compute_robot_state-ego": 0.07762907083752085, "sim_compute_robot_state-npc0": 0.07268203577949005, "sim_compute_robot_state-npc1": 0.0737385842406634, "sim_compute_robot_state-npc2": 0.07401721917309807, "sim_compute_robot_state-npc3": 0.07278293776280671}, "udem1-3-0": {"driven_any": 0.9410513078477724, "sim_physics": 0.24045948684215543, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.06785645584265391, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.1725043555100759, "deviation-heading": 0.35154027506548424, "set_robot_commands": 0.10584088166554768, "deviation-center-line": 0.08975118546699407, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.042349850138028465, "sim_compute_performance-ego": 0.07496668895085652, "sim_compute_robot_state-ego": 0.0836743712425232, "sim_compute_robot_state-npc0": 0.07504473626613617, "sim_compute_robot_state-npc1": 0.07639263570308685, "sim_compute_robot_state-npc2": 0.07685925563176473, "sim_compute_robot_state-npc3": 0.0752473771572113}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.249216670081729, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.06826709565662202, "in-drivable-lane": 0, "agent_compute-ego": 0.17673830758957637, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.10612329982575916, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.043867088499523345, "sim_compute_performance-ego": 0.07355475425720215, "sim_compute_robot_state-ego": 0.08150423140752883, "sim_compute_robot_state-npc0": 0.07465748559860956, "sim_compute_robot_state-npc1": 0.07808980487641834, "sim_compute_robot_state-npc2": 0.08144522848583403, "sim_compute_robot_state-npc3": 0.0784686633518764}}
set_robot_commands_max0.10622424738747734
set_robot_commands_mean0.10526621463888616
set_robot_commands_median0.10584088166554768
set_robot_commands_min0.10349648670085423
sim_compute_performance-ego_max0.07581346375601632
sim_compute_performance-ego_mean0.07444327849625802
sim_compute_performance-ego_median0.07435587388050707
sim_compute_performance-ego_min0.07352561163670808
sim_compute_robot_state-ego_max0.0836743712425232
sim_compute_robot_state-ego_mean0.08012992438659619
sim_compute_robot_state-ego_median0.08009161267961774
sim_compute_robot_state-ego_min0.07762907083752085
sim_compute_robot_state-npc0_max0.07504473626613617
sim_compute_robot_state-npc0_mean0.07397809863305735
sim_compute_robot_state-npc0_median0.07443892955780029
sim_compute_robot_state-npc0_min0.07268203577949005
sim_compute_robot_state-npc1_max0.07808980487641834
sim_compute_robot_state-npc1_mean0.07591967277284513
sim_compute_robot_state-npc1_median0.07639263570308685
sim_compute_robot_state-npc1_min0.07342689550375636
sim_compute_robot_state-npc2_max0.08144522848583403
sim_compute_robot_state-npc2_mean0.07605188709638756
sim_compute_robot_state-npc2_median0.07600026471274239
sim_compute_robot_state-npc2_min0.07193746747849863
sim_compute_robot_state-npc3_max0.0784686633518764
sim_compute_robot_state-npc3_mean0.07554888106777535
sim_compute_robot_state-npc3_median0.0752473771572113
sim_compute_robot_state-npc3_min0.07278293776280671
sim_compute_sim_state_max0.043867088499523345
sim_compute_sim_state_mean0.042547742739647954
sim_compute_sim_state_median0.042349850138028465
sim_compute_sim_state_min0.04168602563802478
sim_physics_max0.249216670081729
sim_physics_mean0.23298428250952732
sim_physics_median0.23810440301895144
sim_physics_min0.2146683086469335
sim_render-ego_max0.06949561834335327
sim_render-ego_mean0.06714091522698015
sim_render-ego_median0.06785645584265391
sim_render-ego_min0.06503073914537152
simulation-passed1
survival_time_max5.14999999999999
survival_time_mean2.789999999999997
survival_time_min1.0500000000000005
No reset possible
21122step1-simulationsuccessno0:10:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20428step1-simulationsuccessno0:07:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20416step1-simulationhost-errorno0:08:48
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20416-345948', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20416-345948', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20416-345948', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19452step1-simulationsuccessno0:08:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18727step1-simulationsuccessno0:16:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18713step1-simulationhost-errorno0:17:00
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job18713-698266', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job18713-698266', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job18713-698266', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18370step1-simulationsuccessno0:11:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18041step1-simulationsuccessno0:06:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible