Duckietown Challenges Home Challenges Submissions

Submission 2451

Submission2451
Competingyes
Challengeaido2-LFVI-sim-validation
UserNicky Eichmann
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22965
Next
User labelBaseline solution using reinforcement learning
Admin priority50
Blessingn/a
User priority50

22965

Click the images to see detailed statistics about the episode.

udem1-0-0

udem1-1-0

udem1-2-0

udem1-3-0

udem1-4-0

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22965step1-simulationsuccessyes0:35:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median-1.339803295303147
survival_time_median14.950000000000076
deviation-center-line_median1.0021347592871723
in-drivable-lane_median4.050000000000024


other stats
agent_compute-ego_max0.16164000272750856
agent_compute-ego_mean0.15610354566574097
agent_compute-ego_median0.1540182638168335
agent_compute-ego_min0.1513586123784383
deviation-center-line_max1.5296442281508644
deviation-center-line_mean0.9253718574185378
deviation-center-line_min0.2826196070981827
deviation-heading_max8.558153948461719
deviation-heading_mean7.098260242116498
deviation-heading_median7.090641611385008
deviation-heading_min5.695926383403844
driven_any_max4.3785064440528245
driven_any_mean4.378506444052816
driven_any_median4.378506444052819
driven_any_min4.378506444052804
driven_lanedir_consec_max-1.272834629261013
driven_lanedir_consec_mean-1.3313573411979192
driven_lanedir_consec_min-1.3670060414727203
driven_lanedir_max-1.366266842110191
driven_lanedir_mean-2.078204689886424
driven_lanedir_median-2.260643253811775
driven_lanedir_min-2.710754983563031
in-drivable-lane_max7.400000000000039
in-drivable-lane_mean4.620000000000026
in-drivable-lane_min2.150000000000011
per-episodes
details{"udem1-0-0": {"driven_any": 4.378506444052811, "sim_physics": 0.16945639848709107, "survival_time": 14.950000000000076, "driven_lanedir": -2.710754983563031, "sim_render-ego": 0.05819206953048706, "in-drivable-lane": 2.150000000000011, "agent_compute-ego": 0.1513586123784383, "deviation-heading": 8.372894374860635, "set_robot_commands": 0.08639689366022746, "deviation-center-line": 1.5195321584534802, "driven_lanedir_consec": -1.3108758978425243, "sim_compute_sim_state": 0.03571542819341024, "sim_compute_performance-ego": 0.06123533646265666, "sim_compute_robot_state-ego": 0.06555419365564982, "sim_compute_robot_state-npc0": 0.062415278752644854, "sim_compute_robot_state-npc1": 0.06303441445032755, "sim_compute_robot_state-npc2": 0.06254379431406656, "sim_compute_robot_state-npc3": 0.061077833970387775}, "udem1-1-0": {"driven_any": 4.3785064440528245, "sim_physics": 0.1963050842285156, "survival_time": 14.950000000000076, "driven_lanedir": -2.686352328474403, "sim_render-ego": 0.06244195063908895, "in-drivable-lane": 2.150000000000011, "agent_compute-ego": 0.16164000272750856, "deviation-heading": 8.558153948461719, "set_robot_commands": 0.09260231733322144, "deviation-center-line": 1.5296442281508644, "driven_lanedir_consec": -1.272834629261013, "sim_compute_sim_state": 0.03846120834350586, "sim_compute_performance-ego": 0.0677091670036316, "sim_compute_robot_state-ego": 0.07323710282643636, "sim_compute_robot_state-npc0": 0.06724067529042561, "sim_compute_robot_state-npc1": 0.06775244156519572, "sim_compute_robot_state-npc2": 0.06679384152094524, "sim_compute_robot_state-npc3": 0.06824360211690267}, "udem1-2-0": {"driven_any": 4.378506444052804, "sim_physics": 0.18989391009012857, "survival_time": 14.950000000000076, "driven_lanedir": -1.366266842110191, "sim_render-ego": 0.0632138458887736, "in-drivable-lane": 7.35000000000004, "agent_compute-ego": 0.16144652446111044, "deviation-heading": 5.773684892471283, "set_robot_commands": 0.08892913262049357, "deviation-center-line": 0.29292853410298886, "driven_lanedir_consec": -1.366266842110191, "sim_compute_sim_state": 0.03933207432428996, "sim_compute_performance-ego": 0.06654704729715984, "sim_compute_robot_state-ego": 0.0708387041091919, "sim_compute_robot_state-npc0": 0.06876530408859254, "sim_compute_robot_state-npc1": 0.06705504814783732, "sim_compute_robot_state-npc2": 0.06762919743855794, "sim_compute_robot_state-npc3": 0.06814869006474812}, "udem1-3-0": {"driven_any": 4.3785064440528245, "sim_physics": 0.18457745790481567, "survival_time": 14.950000000000076, "driven_lanedir": -1.3670060414727203, "sim_render-ego": 0.05885006825129191, "in-drivable-lane": 7.400000000000039, "agent_compute-ego": 0.1540182638168335, "deviation-heading": 5.695926383403844, "set_robot_commands": 0.0852444338798523, "deviation-center-line": 0.2826196070981827, "driven_lanedir_consec": -1.3670060414727203, "sim_compute_sim_state": 0.03720717748006185, "sim_compute_performance-ego": 0.06571757634480795, "sim_compute_robot_state-ego": 0.0673143990834554, "sim_compute_robot_state-npc0": 0.064660009543101, "sim_compute_robot_state-npc1": 0.06520589749018352, "sim_compute_robot_state-npc2": 0.06374581098556518, "sim_compute_robot_state-npc3": 0.06520259459813436}, "udem1-4-0": {"driven_any": 4.378506444052819, "sim_physics": 0.1780240519841512, "survival_time": 14.950000000000076, "driven_lanedir": -2.260643253811775, "sim_render-ego": 0.05824246724446615, "in-drivable-lane": 4.050000000000024, "agent_compute-ego": 0.15205432494481405, "deviation-heading": 7.090641611385008, "set_robot_commands": 0.08677097241083781, "deviation-center-line": 1.0021347592871723, "driven_lanedir_consec": -1.339803295303147, "sim_compute_sim_state": 0.03556403954823812, "sim_compute_performance-ego": 0.062419451872507734, "sim_compute_robot_state-ego": 0.06781713008880615, "sim_compute_robot_state-npc0": 0.06343355655670166, "sim_compute_robot_state-npc1": 0.06446100870768229, "sim_compute_robot_state-npc2": 0.0638701581954956, "sim_compute_robot_state-npc3": 0.06762945731480917}}
set_robot_commands_max0.09260231733322144
set_robot_commands_mean0.08798874998092651
set_robot_commands_median0.08677097241083781
set_robot_commands_min0.0852444338798523
sim_compute_performance-ego_max0.0677091670036316
sim_compute_performance-ego_mean0.06472571579615276
sim_compute_performance-ego_median0.06571757634480795
sim_compute_performance-ego_min0.06123533646265666
sim_compute_robot_state-ego_max0.07323710282643636
sim_compute_robot_state-ego_mean0.06895230595270793
sim_compute_robot_state-ego_median0.06781713008880615
sim_compute_robot_state-ego_min0.06555419365564982
sim_compute_robot_state-npc0_max0.06876530408859254
sim_compute_robot_state-npc0_mean0.06530296484629314
sim_compute_robot_state-npc0_median0.064660009543101
sim_compute_robot_state-npc0_min0.062415278752644854
sim_compute_robot_state-npc1_max0.06775244156519572
sim_compute_robot_state-npc1_mean0.06550176207224527
sim_compute_robot_state-npc1_median0.06520589749018352
sim_compute_robot_state-npc1_min0.06303441445032755
sim_compute_robot_state-npc2_max0.06762919743855794
sim_compute_robot_state-npc2_mean0.06491656049092612
sim_compute_robot_state-npc2_median0.0638701581954956
sim_compute_robot_state-npc2_min0.06254379431406656
sim_compute_robot_state-npc3_max0.06824360211690267
sim_compute_robot_state-npc3_mean0.06606043561299643
sim_compute_robot_state-npc3_median0.06762945731480917
sim_compute_robot_state-npc3_min0.061077833970387775
sim_compute_sim_state_max0.03933207432428996
sim_compute_sim_state_mean0.0372559855779012
sim_compute_sim_state_median0.03720717748006185
sim_compute_sim_state_min0.03556403954823812
sim_physics_max0.1963050842285156
sim_physics_mean0.1836513805389404
sim_physics_median0.18457745790481567
sim_physics_min0.16945639848709107
sim_render-ego_max0.0632138458887736
sim_render-ego_mean0.06018808031082153
sim_render-ego_median0.05885006825129191
sim_render-ego_min0.05819206953048706
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
21277step1-simulationsuccessno0:38:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21261step1-simulationhost-errorno0:30:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-validation/submission2451/step1-simulation-ip-172-31-42-167-7227-job21261/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20553step1-simulationsuccessno0:37:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19625step1-simulationsuccessno0:30:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19590step1-simulationhost-errorno0:10:21
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job19590-827658', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job19590-827658', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job19590-827658', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18846step1-simulationerrorno0:21:33
Waited 1203 seconds [...]
Waited 1203 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18489step1-simulationerrorno0:21:34
Waited 1203 seconds [...]
Waited 1203 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17934step1-simulationerrorno0:20:43
Waited 1200 seconds [...]
Waited 1200 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17851step1-simulationerrorno0:20:46
Waited 1200 seconds [...]
Waited 1200 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17850step1-simulationerrorno0:20:59
Waited 1203 seconds [...]
Waited 1203 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17822step1-simulationtimeoutno0:25:36
I can see how the jo [...]
I can see how the job 17822 is timeout because passed 1536 seconds and the timeout is 1200.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17821step1-simulationerrorno0:21:23
Waited 1203 seconds [...]
Waited 1203 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17629step1-simulationerrorno0:21:32
Waited 1203 seconds [...]
Waited 1203 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible