Duckietown Challenges Home Challenges Submissions

Submission 2712

Submission2712
Competingyes
Challengeaido2-LFVI-sim-validation
UserRami Al-Naim 🇷🇺
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22815
Next
User labelchallenge-aido_LF-baseline-duckietown
Admin priority50
Blessingn/a
User priority50

22815

Click the images to see detailed statistics about the episode.

udem1-0-0

udem1-1-0

udem1-2-0

udem1-3-0

udem1-4-0

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22815step1-simulationsuccessyes0:22:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.26543479260232994
survival_time_median3.5499999999999954
deviation-center-line_median0.23910407569423
in-drivable-lane_median0.30000000000000426


other stats
agent_compute-ego_max0.10860885514153376
agent_compute-ego_mean0.09789067797329024
agent_compute-ego_median0.0961849578221639
agent_compute-ego_min0.0844975864286903
deviation-center-line_max1.1535568089483947
deviation-center-line_mean0.4925611681048516
deviation-center-line_min0.14171773416189698
deviation-heading_max6.730851577205935
deviation-heading_mean3.1136331686511345
deviation-heading_median1.6025051766500047
deviation-heading_min0.5439981627131697
driven_any_max2.348858998049182
driven_any_mean1.116061327104622
driven_any_median0.499959472880548
driven_any_min0.23562442318253665
driven_lanedir_consec_max1.563231605767316
driven_lanedir_consec_mean0.6334436558637075
driven_lanedir_consec_min0.1838990275548249
driven_lanedir_max2.037126293343852
driven_lanedir_mean0.7807369369178254
driven_lanedir_median0.26543479260232994
driven_lanedir_min0.2109222805807189
in-drivable-lane_max3.6000000000000343
in-drivable-lane_mean0.9600000000000072
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 0.499959472880548, "sim_physics": 0.23930889452007456, "survival_time": 3.5499999999999954, "driven_lanedir": 0.23148868050436144, "sim_render-ego": 0.06909680030715297, "in-drivable-lane": 0.8999999999999977, "agent_compute-ego": 0.09267558178431552, "deviation-heading": 1.6025051766500047, "set_robot_commands": 0.11561803414788044, "deviation-center-line": 0.23910407569423, "driven_lanedir_consec": 0.1838990275548249, "sim_compute_sim_state": 0.04144963748018506, "sim_compute_performance-ego": 0.07635457750777123, "sim_compute_robot_state-ego": 0.08474665964153451, "sim_compute_robot_state-npc0": 0.07816729747073752, "sim_compute_robot_state-npc1": 0.07566748202686578, "sim_compute_robot_state-npc2": 0.07496113844320808, "sim_compute_robot_state-npc3": 0.07688702663905184}, "udem1-1-0": {"driven_any": 0.3159741146440911, "sim_physics": 0.2373695943666541, "survival_time": 2.3, "driven_lanedir": 0.26543479260232994, "sim_render-ego": 0.07099132434181545, "in-drivable-lane": 0, "agent_compute-ego": 0.10748640868974768, "deviation-heading": 1.1083072802579417, "set_robot_commands": 0.11381580518639606, "deviation-center-line": 0.14171773416189698, "driven_lanedir_consec": 0.26543479260232994, "sim_compute_sim_state": 0.04328583116116731, "sim_compute_performance-ego": 0.07575909469438635, "sim_compute_robot_state-ego": 0.08443653065225352, "sim_compute_robot_state-npc0": 0.07419802313265593, "sim_compute_robot_state-npc1": 0.0754725984905077, "sim_compute_robot_state-npc2": 0.07704048571379288, "sim_compute_robot_state-npc3": 0.07397981311963953}, "udem1-2-0": {"driven_any": 2.1798896267667525, "sim_physics": 0.2087190622906033, "survival_time": 13.900000000000064, "driven_lanedir": 1.1587126375578642, "sim_render-ego": 0.06320085199616796, "in-drivable-lane": 3.6000000000000343, "agent_compute-ego": 0.0844975864286903, "deviation-heading": 6.730851577205935, "set_robot_commands": 0.10065491079426496, "deviation-center-line": 0.7841236293430482, "driven_lanedir_consec": 0.9437305728133478, "sim_compute_sim_state": 0.03918761486629787, "sim_compute_performance-ego": 0.06957814590536433, "sim_compute_robot_state-ego": 0.07666157904288752, "sim_compute_robot_state-npc0": 0.07632664896601396, "sim_compute_robot_state-npc1": 0.06828542064419753, "sim_compute_robot_state-npc2": 0.07350385960915105, "sim_compute_robot_state-npc3": 0.06963009199650168}, "udem1-3-0": {"driven_any": 0.23562442318253665, "sim_physics": 0.26782382196850246, "survival_time": 1.800000000000001, "driven_lanedir": 0.2109222805807189, "sim_render-ego": 0.07236828406651814, "in-drivable-lane": 0, "agent_compute-ego": 0.10860885514153376, "deviation-heading": 0.5439981627131697, "set_robot_commands": 0.1275033023622301, "deviation-center-line": 0.14430359237668816, "driven_lanedir_consec": 0.2109222805807189, "sim_compute_sim_state": 0.046732677353752985, "sim_compute_performance-ego": 0.08079432778888279, "sim_compute_robot_state-ego": 0.09050678544574314, "sim_compute_robot_state-npc0": 0.08146519793404473, "sim_compute_robot_state-npc1": 0.08052429225709704, "sim_compute_robot_state-npc2": 0.0833574997054206, "sim_compute_robot_state-npc3": 0.08062035507626003}, "udem1-4-0": {"driven_any": 2.348858998049182, "sim_physics": 0.2487187647819519, "survival_time": 14.950000000000076, "driven_lanedir": 2.037126293343852, "sim_render-ego": 0.07188812096913656, "in-drivable-lane": 0.30000000000000426, "agent_compute-ego": 0.0961849578221639, "deviation-heading": 5.582503646428622, "set_robot_commands": 0.11770303328831992, "deviation-center-line": 1.1535568089483947, "driven_lanedir_consec": 1.563231605767316, "sim_compute_sim_state": 0.044566972255706785, "sim_compute_performance-ego": 0.080744415918986, "sim_compute_robot_state-ego": 0.08648629188537597, "sim_compute_robot_state-npc0": 0.08032128890355428, "sim_compute_robot_state-npc1": 0.08067387421925863, "sim_compute_robot_state-npc2": 0.07905415217081706, "sim_compute_robot_state-npc3": 0.07937532345453899}}
set_robot_commands_max0.1275033023622301
set_robot_commands_mean0.1150590171558183
set_robot_commands_median0.11561803414788044
set_robot_commands_min0.10065491079426496
sim_compute_performance-ego_max0.08079432778888279
sim_compute_performance-ego_mean0.07664611236307814
sim_compute_performance-ego_median0.07635457750777123
sim_compute_performance-ego_min0.06957814590536433
sim_compute_robot_state-ego_max0.09050678544574314
sim_compute_robot_state-ego_mean0.08456756933355894
sim_compute_robot_state-ego_median0.08474665964153451
sim_compute_robot_state-ego_min0.07666157904288752
sim_compute_robot_state-npc0_max0.08146519793404473
sim_compute_robot_state-npc0_mean0.07809569128140129
sim_compute_robot_state-npc0_median0.07816729747073752
sim_compute_robot_state-npc0_min0.07419802313265593
sim_compute_robot_state-npc1_max0.08067387421925863
sim_compute_robot_state-npc1_mean0.07612473352758534
sim_compute_robot_state-npc1_median0.07566748202686578
sim_compute_robot_state-npc1_min0.06828542064419753
sim_compute_robot_state-npc2_max0.0833574997054206
sim_compute_robot_state-npc2_mean0.07758342712847795
sim_compute_robot_state-npc2_median0.07704048571379288
sim_compute_robot_state-npc2_min0.07350385960915105
sim_compute_robot_state-npc3_max0.08062035507626003
sim_compute_robot_state-npc3_mean0.0760985220571984
sim_compute_robot_state-npc3_median0.07688702663905184
sim_compute_robot_state-npc3_min0.06963009199650168
sim_compute_sim_state_max0.046732677353752985
sim_compute_sim_state_mean0.043044546623422
sim_compute_sim_state_median0.04328583116116731
sim_compute_sim_state_min0.03918761486629787
sim_physics_max0.26782382196850246
sim_physics_mean0.2403880275855573
sim_physics_median0.23930889452007456
sim_physics_min0.2087190622906033
sim_render-ego_max0.07236828406651814
sim_render-ego_mean0.06950907633615822
sim_render-ego_median0.07099132434181545
sim_render-ego_min0.06320085199616796
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean7.300000000000027
survival_time_min1.800000000000001
No reset possible
21079step1-simulationsuccessno0:16:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21070step1-simulationhost-errorno0:11:41
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-validation/submission2712/step1-simulation-ip-172-31-42-167-7227-job21070/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20390step1-simulationsuccessno0:23:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20372step1-simulationhost-errorno0:04:43
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20372-621167', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20372-621167', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20372-621167', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19415step1-simulationsuccessno0:09:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19406step1-simulationsuccessno0:19:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18670step1-simulationerrorno0:21:43
Waited 1203 seconds [...]
Waited 1203 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18328step1-simulationsuccessno0:17:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18093step1-simulationsuccessno0:08:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18092step1-simulationsuccessno0:10:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible