Duckietown Challenges Home Challenges Submissions

Submission 2991

Submission2991
Competingyes
Challengeaido2-LF-sim-testing
UserArtem Ioselevskii
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22667
Next
User labelchallenge-aido_LF-template-tensorflow
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22667step1-simulationsuccessyes0:14:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.3996987536249481
survival_time_median1.7500000000000009
deviation-center-line_median0.11599543902053434
in-drivable-lane_median0


other stats
agent_compute-ego_max0.1583335592940047
agent_compute-ego_mean0.15158669671464872
agent_compute-ego_median0.15009116191490024
agent_compute-ego_min0.1474708012172154
deviation-center-line_max0.34163211925298725
deviation-center-line_mean0.14972790754800175
deviation-center-line_min0.06569038487387166
deviation-heading_max2.1780968084526124
deviation-heading_mean0.8610899603099935
deviation-heading_median0.49989292727583834
deviation-heading_min0.16539914836525055
driven_any_max1.708519733018482
driven_any_mean0.7091010866765619
driven_any_median0.48229121935621866
driven_any_min0.21081382766787707
driven_lanedir_consec_max1.0045160964113204
driven_lanedir_consec_mean0.49353539801451246
driven_lanedir_consec_min0.17608356754231158
driven_lanedir_max1.0045160964113204
driven_lanedir_mean0.49353539801451246
driven_lanedir_median0.3996987536249481
driven_lanedir_min0.17608356754231158
in-drivable-lane_max2.6499999999999906
in-drivable-lane_mean0.5033333333333316
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.47747813305008135, "sim_physics": 0.06293808260271626, "survival_time": 1.5500000000000007, "driven_lanedir": 0.38987975590456214, "sim_render-ego": 0.06312301851088001, "in-drivable-lane": 0.10000000000000007, "agent_compute-ego": 0.1481223875476468, "deviation-heading": 0.5040515789759067, "set_robot_commands": 0.07906716869723412, "deviation-center-line": 0.08936635068807702, "driven_lanedir_consec": 0.38987975590456214, "sim_compute_sim_state": 0.040909044204219695, "sim_compute_performance-ego": 0.06352641505579795, "sim_compute_robot_state-ego": 0.06504906377484722}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.29150115377437114, "sim_physics": 0.061201519436306424, "survival_time": 1.3500000000000003, "driven_lanedir": 0.24487697402917996, "sim_render-ego": 0.06091053397567184, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.15752485946372705, "deviation-heading": 0.49989292727583834, "set_robot_commands": 0.07941347581368906, "deviation-center-line": 0.09022645876344353, "driven_lanedir_consec": 0.24487697402917996, "sim_compute_sim_state": 0.04368906550937229, "sim_compute_performance-ego": 0.06615402080394604, "sim_compute_robot_state-ego": 0.0690145492553711}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.6235652606440992, "sim_physics": 0.06098707302196606, "survival_time": 1.850000000000001, "driven_lanedir": 0.6015460617319315, "sim_render-ego": 0.0647612713478707, "in-drivable-lane": 0, "agent_compute-ego": 0.1583335592940047, "deviation-heading": 0.3784645829606996, "set_robot_commands": 0.08129418862832559, "deviation-center-line": 0.11599543902053434, "driven_lanedir_consec": 0.6015460617319315, "sim_compute_sim_state": 0.04240068229469093, "sim_compute_performance-ego": 0.06800228840595968, "sim_compute_robot_state-ego": 0.06829544660207387}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.708519733018482, "sim_physics": 0.05641943613688151, "survival_time": 5.249999999999989, "driven_lanedir": 0.6522503952882599, "sim_render-ego": 0.06399176234290713, "in-drivable-lane": 2.5999999999999908, "agent_compute-ego": 0.15330376852126348, "deviation-heading": 1.0972133992819515, "set_robot_commands": 0.07720356668744768, "deviation-center-line": 0.15276634820527762, "driven_lanedir_consec": 0.6522503952882599, "sim_compute_sim_state": 0.04148981003534226, "sim_compute_performance-ego": 0.06661119688124884, "sim_compute_robot_state-ego": 0.0645515623546782}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.4528422872267424, "sim_physics": 0.05929909433637347, "survival_time": 6.299999999999986, "driven_lanedir": 1.0045160964113204, "sim_render-ego": 0.06728562097700816, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.148918626800416, "deviation-heading": 2.1780968084526124, "set_robot_commands": 0.07983912740434919, "deviation-center-line": 0.34163211925298725, "driven_lanedir_consec": 1.0045160964113204, "sim_compute_sim_state": 0.04251088225652301, "sim_compute_performance-ego": 0.06928694815862746, "sim_compute_robot_state-ego": 0.06626319128369527}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.137543982553609, "sim_physics": 0.05983947096644221, "survival_time": 3.699999999999995, "driven_lanedir": 0.7133839920444605, "sim_render-ego": 0.06749529774124557, "in-drivable-lane": 0.7499999999999973, "agent_compute-ego": 0.15537011945569837, "deviation-heading": 1.6634647411257313, "set_robot_commands": 0.07920950490075189, "deviation-center-line": 0.20431653422172105, "driven_lanedir_consec": 0.7133839920444605, "sim_compute_sim_state": 0.04333292793583225, "sim_compute_performance-ego": 0.07001266286179826, "sim_compute_robot_state-ego": 0.0674569896749548}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.3901135008676592, "sim_physics": 0.061312501247112565, "survival_time": 1.3000000000000005, "driven_lanedir": 0.37864260555184304, "sim_render-ego": 0.06692833166856033, "in-drivable-lane": 0, "agent_compute-ego": 0.14966357671297514, "deviation-heading": 0.2872649878496704, "set_robot_commands": 0.07919949751633865, "deviation-center-line": 0.1012287181281264, "driven_lanedir_consec": 0.37864260555184304, "sim_compute_sim_state": 0.04267523838923527, "sim_compute_performance-ego": 0.06734845271477333, "sim_compute_robot_state-ego": 0.062368576343242936}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.4848078861597781, "sim_physics": 0.06056657518659319, "survival_time": 1.7500000000000009, "driven_lanedir": 0.459316772785724, "sim_render-ego": 0.06663785661969866, "in-drivable-lane": 0, "agent_compute-ego": 0.1474708012172154, "deviation-heading": 0.4438992873553276, "set_robot_commands": 0.07814365114484514, "deviation-center-line": 0.11907587348619962, "driven_lanedir_consec": 0.459316772785724, "sim_compute_sim_state": 0.04353241920471192, "sim_compute_performance-ego": 0.07168259620666503, "sim_compute_robot_state-ego": 0.06823588098798479}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.21081382766787707, "sim_physics": 0.05902634967457165, "survival_time": 1.1000000000000003, "driven_lanedir": 0.17608356754231158, "sim_render-ego": 0.06340968608856201, "in-drivable-lane": 0, "agent_compute-ego": 0.1522698619148948, "deviation-heading": 0.4946604144452886, "set_robot_commands": 0.08133679086511786, "deviation-center-line": 0.06569038487387166, "driven_lanedir_consec": 0.17608356754231158, "sim_compute_sim_state": 0.03962473435835405, "sim_compute_performance-ego": 0.06704367290843617, "sim_compute_robot_state-ego": 0.06302773952484131}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.48229121935621866, "sim_physics": 0.06477491060892741, "survival_time": 2.549999999999999, "driven_lanedir": 0.3996987536249481, "sim_render-ego": 0.06825748611898984, "in-drivable-lane": 0, "agent_compute-ego": 0.15009116191490024, "deviation-heading": 1.1428770240195931, "set_robot_commands": 0.07833979176540001, "deviation-center-line": 0.17472142261812815, "driven_lanedir_consec": 0.3996987536249481, "sim_compute_sim_state": 0.045714672874001896, "sim_compute_performance-ego": 0.06882904090133368, "sim_compute_robot_state-ego": 0.06779836673362583}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.0650121379144897, "sim_physics": 0.06161525652006075, "survival_time": 3.8499999999999943, "driven_lanedir": 0.9682337966925624, "sim_render-ego": 0.06832059018023602, "in-drivable-lane": 0, "agent_compute-ego": 0.14831820710912927, "deviation-heading": 1.402130058920228, "set_robot_commands": 0.07734804029588575, "deviation-center-line": 0.3026009605701227, "driven_lanedir_consec": 0.9682337966925624, "sim_compute_sim_state": 0.04347377938109559, "sim_compute_performance-ego": 0.06780241681383802, "sim_compute_robot_state-ego": 0.06594540546466778}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.3315443846328699, "sim_physics": 0.0637696142549868, "survival_time": 1.3500000000000003, "driven_lanedir": 0.3126142815955265, "sim_render-ego": 0.06654235168739601, "in-drivable-lane": 0, "agent_compute-ego": 0.15314415649131494, "deviation-heading": 0.4365675920987284, "set_robot_commands": 0.08093329712196633, "deviation-center-line": 0.11451107030580188, "driven_lanedir_consec": 0.3126142815955265, "sim_compute_sim_state": 0.043733587971440065, "sim_compute_performance-ego": 0.0670782106894034, "sim_compute_robot_state-ego": 0.0674635304345025}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.4862415998759038, "sim_physics": 0.058464084641408114, "survival_time": 5.899999999999987, "driven_lanedir": 0.6623962362933695, "sim_render-ego": 0.06698221675420211, "in-drivable-lane": 2.6499999999999906, "agent_compute-ego": 0.15388390573404603, "deviation-heading": 1.764492178355693, "set_robot_commands": 0.07989994146056094, "deviation-center-line": 0.2121398895801658, "driven_lanedir_consec": 0.6623962362933695, "sim_compute_sim_state": 0.043446815620034426, "sim_compute_performance-ego": 0.06725102157916053, "sim_compute_robot_state-ego": 0.06552109273813539}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.2658277338657173, "sim_physics": 0.05788068149400794, "survival_time": 1.1500000000000004, "driven_lanedir": 0.2185194334768883, "sim_render-ego": 0.06571978071461552, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.1482939512833305, "deviation-heading": 0.4578746751673839, "set_robot_commands": 0.07824586785357932, "deviation-center-line": 0.069141279951549, "driven_lanedir_consec": 0.2185194334768883, "sim_compute_sim_state": 0.03938436508178711, "sim_compute_performance-ego": 0.06922211854354195, "sim_compute_robot_state-ego": 0.06524894548499066}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.22841345954053124, "sim_physics": 0.060586665806017424, "survival_time": 0.9500000000000004, "driven_lanedir": 0.2210722472447979, "sim_render-ego": 0.06543655144540887, "in-drivable-lane": 0, "agent_compute-ego": 0.14909150725916812, "deviation-heading": 0.16539914836525055, "set_robot_commands": 0.07892249759874846, "deviation-center-line": 0.09250576355401992, "driven_lanedir_consec": 0.2210722472447979, "sim_compute_sim_state": 0.04181579539650365, "sim_compute_performance-ego": 0.07039864439713328, "sim_compute_robot_state-ego": 0.06706350728085167}}
set_robot_commands_max0.08133679086511786
set_robot_commands_mean0.079226427183616
set_robot_commands_median0.07919949751633865
set_robot_commands_min0.07720356668744768
sim_compute_performance-ego_max0.07168259620666503
sim_compute_performance-ego_mean0.06801664712811091
sim_compute_performance-ego_median0.06780241681383802
sim_compute_performance-ego_min0.06352641505579795
sim_compute_robot_state-ego_max0.0690145492553711
sim_compute_robot_state-ego_mean0.06622025652923089
sim_compute_robot_state-ego_median0.06626319128369527
sim_compute_robot_state-ego_min0.062368576343242936
sim_compute_sim_state_max0.045714672874001896
sim_compute_sim_state_mean0.04251558803420963
sim_compute_sim_state_median0.04267523838923527
sim_compute_sim_state_min0.03938436508178711
sim_physics_max0.06477491060892741
sim_physics_mean0.06057875439562478
sim_physics_median0.060586665806017424
sim_physics_min0.05641943613688151
sim_render-ego_max0.06832059018023602
sim_render-ego_mean0.06572015707821685
sim_render-ego_median0.06654235168739601
sim_render-ego_min0.06091053397567184
simulation-passed1
survival_time_max6.299999999999986
survival_time_mean2.659999999999997
survival_time_min0.9500000000000004
No reset possible
20889step1-simulationsuccessno0:19:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20801step1-simulationhost-errorno0:14:58
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-testing/submission2991/step1-simulation-ip-172-31-25-98-2348-job20801/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20749step1-simulationtimeoutno1:05:11
I can see how the jo [...]
I can see how the job 20749 is timeout because passed 3911 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20712step1-simulationhost-errorno0:03:42
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 996, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20712-257984', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 579, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 905, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 591, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1004, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20712-257984', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20712-257984', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20213step1-simulationsuccessno0:04:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20134step1-simulationsuccessno0:07:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19686step1-simulationsuccessno0:04:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible