Duckietown Challenges Home Challenges Submissions

Submission 2608

Submission2608
Competingyes
Challengeaido2-LFV-sim-testing
UserKonstantin Chaika
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22866
Next
User labelchallenge-aido_LF-template-ros - Template solution using ROS
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22866step1-simulationsuccessyes0:26:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.30314311735235044
survival_time_median2.6499999999999986
deviation-center-line_median0.1192462800776758
in-drivable-lane_median0.9499999999999966


other stats
agent_compute-ego_max0.07598071373425998
agent_compute-ego_mean0.07157388914380029
agent_compute-ego_median0.07181255022684734
agent_compute-ego_min0.06586734921324487
deviation-center-line_max0.2294363663374862
deviation-center-line_mean0.12653708755488874
deviation-center-line_min0.06426934298501713
deviation-heading_max0.6986268167330368
deviation-heading_mean0.44799520296706863
deviation-heading_median0.41847631252347706
deviation-heading_min0.2596285487829774
driven_any_max1.599733354117788
driven_any_mean0.8095342605803044
driven_any_median0.7336463183586153
driven_any_min0.26903550091980144
driven_lanedir_consec_max1.021002447119848
driven_lanedir_consec_mean0.448904809024379
driven_lanedir_consec_min0.2407721645627201
driven_lanedir_max1.021002447119848
driven_lanedir_mean0.448904809024379
driven_lanedir_median0.30314311735235044
driven_lanedir_min0.2407721645627201
in-drivable-lane_max4.799999999999986
in-drivable-lane_mean1.1466666666666634
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.922682252912863, "sim_physics": 0.17790579460036587, "survival_time": 3.5499999999999954, "driven_lanedir": 0.36592094708219225, "sim_render-ego": 0.06599681813951949, "in-drivable-lane": 1.8999999999999944, "agent_compute-ego": 0.07527728147909675, "deviation-heading": 0.2596285487829774, "set_robot_commands": 0.10218494039186288, "deviation-center-line": 0.16987818035809085, "driven_lanedir_consec": 0.36592094708219225, "sim_compute_sim_state": 0.041508983558332414, "sim_compute_performance-ego": 0.07322247263411401, "sim_compute_robot_state-ego": 0.08121262469761809, "sim_compute_robot_state-npc0": 0.07128298114722884, "sim_compute_robot_state-npc1": 0.07164493413038657, "sim_compute_robot_state-npc2": 0.07118532019601741, "sim_compute_robot_state-npc3": 0.0698850154876709}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.3642886245953926, "sim_physics": 0.1969302743673325, "survival_time": 1.6000000000000008, "driven_lanedir": 0.3026144243018116, "sim_render-ego": 0.06070700287818909, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.069203220307827, "deviation-heading": 0.6986268167330368, "set_robot_commands": 0.09782613813877106, "deviation-center-line": 0.10463456752710916, "driven_lanedir_consec": 0.3026144243018116, "sim_compute_sim_state": 0.04052477329969406, "sim_compute_performance-ego": 0.06818988919258118, "sim_compute_robot_state-ego": 0.07434327900409698, "sim_compute_robot_state-npc0": 0.06914889067411423, "sim_compute_robot_state-npc1": 0.06941241770982742, "sim_compute_robot_state-npc2": 0.06752005964517593, "sim_compute_robot_state-npc3": 0.07101272791624069}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.7336463183586153, "sim_physics": 0.20366047437374407, "survival_time": 2.5999999999999988, "driven_lanedir": 0.727268775640014, "sim_render-ego": 0.06709105234879714, "in-drivable-lane": 0, "agent_compute-ego": 0.07598071373425998, "deviation-heading": 0.2723076711805525, "set_robot_commands": 0.10007033898280215, "deviation-center-line": 0.2294363663374862, "driven_lanedir_consec": 0.727268775640014, "sim_compute_sim_state": 0.04040709825662466, "sim_compute_performance-ego": 0.0750968731366671, "sim_compute_robot_state-ego": 0.08269796921656682, "sim_compute_robot_state-npc0": 0.07210063017331637, "sim_compute_robot_state-npc1": 0.07029875425192025, "sim_compute_robot_state-npc2": 0.07320343072597797, "sim_compute_robot_state-npc3": 0.07333630781907302}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.2972979671834337, "sim_physics": 0.19285789869164907, "survival_time": 4.6499999999999915, "driven_lanedir": 0.97716132702229, "sim_render-ego": 0.06403279048140331, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.07181255022684734, "deviation-heading": 0.4911127339226078, "set_robot_commands": 0.09942010141188098, "deviation-center-line": 0.18038572117738352, "driven_lanedir_consec": 0.97716132702229, "sim_compute_sim_state": 0.04061746084561912, "sim_compute_performance-ego": 0.07199180767100344, "sim_compute_robot_state-ego": 0.0763579773646529, "sim_compute_robot_state-npc0": 0.07097344757408224, "sim_compute_robot_state-npc1": 0.07114245045569635, "sim_compute_robot_state-npc2": 0.07030116870839109, "sim_compute_robot_state-npc3": 0.0708130072521907}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.26903550091980144, "sim_physics": 0.1621681395031157, "survival_time": 1.0500000000000005, "driven_lanedir": 0.2407721645627201, "sim_render-ego": 0.06505999110993885, "in-drivable-lane": 0, "agent_compute-ego": 0.06932466370718819, "deviation-heading": 0.41847631252347706, "set_robot_commands": 0.09251716023399716, "deviation-center-line": 0.06426934298501713, "driven_lanedir_consec": 0.2407721645627201, "sim_compute_sim_state": 0.03537971632821219, "sim_compute_performance-ego": 0.067108949025472, "sim_compute_robot_state-ego": 0.06542071842011951, "sim_compute_robot_state-npc0": 0.06712694395156134, "sim_compute_robot_state-npc1": 0.06346086093357631, "sim_compute_robot_state-npc2": 0.0670271600995745, "sim_compute_robot_state-npc3": 0.0635772773197719}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.599733354117788, "sim_physics": 0.1563308948327687, "survival_time": 6.0499999999999865, "driven_lanedir": 0.26415979567759074, "sim_render-ego": 0.06505114973084, "in-drivable-lane": 4.799999999999986, "agent_compute-ego": 0.06767090490041686, "deviation-heading": 0.39392468133084674, "set_robot_commands": 0.0858512456752052, "deviation-center-line": 0.1277842828221673, "driven_lanedir_consec": 0.26415979567759074, "sim_compute_sim_state": 0.035458489883044536, "sim_compute_performance-ego": 0.061369141271291686, "sim_compute_robot_state-ego": 0.06663859974254262, "sim_compute_robot_state-npc0": 0.06273405611022445, "sim_compute_robot_state-npc1": 0.06289172172546387, "sim_compute_robot_state-npc2": 0.06169039749902142, "sim_compute_robot_state-npc3": 0.0611401687968861}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0951392646085512, "sim_physics": 0.16973566157477243, "survival_time": 4.199999999999993, "driven_lanedir": 0.5659790827231905, "sim_render-ego": 0.06095896732239496, "in-drivable-lane": 1.7499999999999938, "agent_compute-ego": 0.07458756083533877, "deviation-heading": 0.5971215168818823, "set_robot_commands": 0.1009211568605332, "deviation-center-line": 0.11288019725884578, "driven_lanedir_consec": 0.5659790827231905, "sim_compute_sim_state": 0.03830744822820028, "sim_compute_performance-ego": 0.06979384876432873, "sim_compute_robot_state-ego": 0.07384598538989112, "sim_compute_robot_state-npc0": 0.0677955633118039, "sim_compute_robot_state-npc1": 0.06839470636276972, "sim_compute_robot_state-npc2": 0.06855923788888114, "sim_compute_robot_state-npc3": 0.07066713344483148}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 1.0452441176451972, "sim_physics": 0.1913647426141275, "survival_time": 3.699999999999995, "driven_lanedir": 1.021002447119848, "sim_render-ego": 0.06608001605884449, "in-drivable-lane": 0, "agent_compute-ego": 0.07544406362482019, "deviation-heading": 0.6206310196168163, "set_robot_commands": 0.10026367934974464, "deviation-center-line": 0.14363020423695483, "driven_lanedir_consec": 1.021002447119848, "sim_compute_sim_state": 0.04097475554492023, "sim_compute_performance-ego": 0.06947028314745105, "sim_compute_robot_state-ego": 0.07662349133878141, "sim_compute_robot_state-npc0": 0.07509284083907669, "sim_compute_robot_state-npc1": 0.07568524012694487, "sim_compute_robot_state-npc2": 0.0721378551947104, "sim_compute_robot_state-npc3": 0.07242814270225731}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.0535850482576197, "sim_physics": 0.1726762851079305, "survival_time": 3.599999999999995, "driven_lanedir": 0.30314311735235044, "sim_render-ego": 0.06385178367296855, "in-drivable-lane": 2.2999999999999945, "agent_compute-ego": 0.06971031096246508, "deviation-heading": 0.40636604380954594, "set_robot_commands": 0.09917943345175848, "deviation-center-line": 0.10907487511555897, "driven_lanedir_consec": 0.30314311735235044, "sim_compute_sim_state": 0.03764798574977451, "sim_compute_performance-ego": 0.06784274843004015, "sim_compute_robot_state-ego": 0.07644206947750515, "sim_compute_robot_state-npc0": 0.07157939672470093, "sim_compute_robot_state-npc1": 0.07018242941962348, "sim_compute_robot_state-npc2": 0.07035383250978258, "sim_compute_robot_state-npc3": 0.06835058662626478}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.4799799001882503, "sim_physics": 0.22602363835985417, "survival_time": 5.349999999999989, "driven_lanedir": 0.5481518397312493, "sim_render-ego": 0.06446901214457004, "in-drivable-lane": 2.9499999999999895, "agent_compute-ego": 0.07424623052650522, "deviation-heading": 0.5514444521435958, "set_robot_commands": 0.10175731694586924, "deviation-center-line": 0.10072735470855042, "driven_lanedir_consec": 0.5481518397312493, "sim_compute_sim_state": 0.03882212282341217, "sim_compute_performance-ego": 0.07343430385411343, "sim_compute_robot_state-ego": 0.07976296237695997, "sim_compute_robot_state-npc0": 0.07008897478335371, "sim_compute_robot_state-npc1": 0.07167925567270439, "sim_compute_robot_state-npc2": 0.0713164383005873, "sim_compute_robot_state-npc3": 0.0715497453636098}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.29610946631760215, "sim_physics": 0.20917342327259209, "survival_time": 1.3500000000000003, "driven_lanedir": 0.27060744077321397, "sim_render-ego": 0.061451964908176, "in-drivable-lane": 0, "agent_compute-ego": 0.0708331443645336, "deviation-heading": 0.5373504765871612, "set_robot_commands": 0.09875242798416704, "deviation-center-line": 0.08269292286428997, "driven_lanedir_consec": 0.27060744077321397, "sim_compute_sim_state": 0.0371595753563775, "sim_compute_performance-ego": 0.06567626529269749, "sim_compute_robot_state-ego": 0.071870900966503, "sim_compute_robot_state-npc0": 0.07151754697163899, "sim_compute_robot_state-npc1": 0.07321791295652036, "sim_compute_robot_state-npc2": 0.07210534590261954, "sim_compute_robot_state-npc3": 0.06945041373923973}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.27791471406425977, "sim_physics": 0.1770508861541748, "survival_time": 1.2500000000000004, "driven_lanedir": 0.2638710416011838, "sim_render-ego": 0.0661042308807373, "in-drivable-lane": 0, "agent_compute-ego": 0.07245863914489746, "deviation-heading": 0.3105689945980953, "set_robot_commands": 0.1010330295562744, "deviation-center-line": 0.1192462800776758, "driven_lanedir_consec": 0.2638710416011838, "sim_compute_sim_state": 0.038801956176757815, "sim_compute_performance-ego": 0.06499267578125, "sim_compute_robot_state-ego": 0.07016545295715332, "sim_compute_robot_state-npc0": 0.07524209976196289, "sim_compute_robot_state-npc1": 0.07222199440002441, "sim_compute_robot_state-npc2": 0.07155335426330567, "sim_compute_robot_state-npc3": 0.07196145057678223}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.7174039891727135, "sim_physics": 0.17776868028460807, "survival_time": 2.6499999999999986, "driven_lanedir": 0.29766819613667395, "sim_render-ego": 0.062206713658458784, "in-drivable-lane": 1.249999999999998, "agent_compute-ego": 0.0737289752600328, "deviation-heading": 0.3122593881578996, "set_robot_commands": 0.1030506412937956, "deviation-center-line": 0.14209620948179827, "driven_lanedir_consec": 0.29766819613667395, "sim_compute_sim_state": 0.0390731118760019, "sim_compute_performance-ego": 0.06925577037739304, "sim_compute_robot_state-ego": 0.07704734352399718, "sim_compute_robot_state-npc0": 0.0664913609342755, "sim_compute_robot_state-npc1": 0.06528207491028984, "sim_compute_robot_state-npc2": 0.06859666896316241, "sim_compute_robot_state-npc3": 0.06786111165892403}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.3170173854101942, "sim_physics": 0.20063021106104695, "survival_time": 1.5500000000000007, "driven_lanedir": 0.30712670008511367, "sim_render-ego": 0.0634944362025107, "in-drivable-lane": 0, "agent_compute-ego": 0.06746272886953046, "deviation-heading": 0.2996283459319333, "set_robot_commands": 0.09710262667748236, "deviation-center-line": 0.13363659833059607, "driven_lanedir_consec": 0.30712670008511367, "sim_compute_sim_state": 0.04135862473518618, "sim_compute_performance-ego": 0.07061746812635852, "sim_compute_robot_state-ego": 0.07818825014175908, "sim_compute_robot_state-npc0": 0.06498882078355359, "sim_compute_robot_state-npc1": 0.06445282505404565, "sim_compute_robot_state-npc2": 0.06637400196444604, "sim_compute_robot_state-npc3": 0.0732005565397201}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.6739360049522848, "sim_physics": 0.19378946809207692, "survival_time": 2.549999999999999, "driven_lanedir": 0.27812483555624423, "sim_render-ego": 0.05650463758730421, "in-drivable-lane": 1.2499999999999984, "agent_compute-ego": 0.06586734921324487, "deviation-heading": 0.550481042305602, "set_robot_commands": 0.11454082470314177, "deviation-center-line": 0.07768321004180687, "driven_lanedir_consec": 0.27812483555624423, "sim_compute_sim_state": 0.03850349257974064, "sim_compute_performance-ego": 0.06301177249235265, "sim_compute_robot_state-ego": 0.06696006363513422, "sim_compute_robot_state-npc0": 0.06522477374357336, "sim_compute_robot_state-npc1": 0.09332939222747204, "sim_compute_robot_state-npc2": 0.06262338395212211, "sim_compute_robot_state-npc3": 0.06244872598087087}}
set_robot_commands_max0.11454082470314177
set_robot_commands_mean0.09963140411048577
set_robot_commands_median0.10007033898280215
set_robot_commands_min0.0858512456752052
sim_compute_performance-ego_max0.0750968731366671
sim_compute_performance-ego_mean0.06873828461314097
sim_compute_performance-ego_median0.06925577037739304
sim_compute_performance-ego_min0.061369141271291686
sim_compute_robot_state-ego_max0.08269796921656682
sim_compute_robot_state-ego_mean0.07450517921688543
sim_compute_robot_state-ego_median0.0763579773646529
sim_compute_robot_state-ego_min0.06542071842011951
sim_compute_robot_state-npc0_max0.07524209976196289
sim_compute_robot_state-npc0_mean0.06942588849896447
sim_compute_robot_state-npc0_median0.07008897478335371
sim_compute_robot_state-npc0_min0.06273405611022445
sim_compute_robot_state-npc1_max0.09332939222747204
sim_compute_robot_state-npc1_mean0.07088646468915104
sim_compute_robot_state-npc1_median0.07029875425192025
sim_compute_robot_state-npc1_min0.06289172172546387
sim_compute_robot_state-npc2_max0.07320343072597797
sim_compute_robot_state-npc2_mean0.06896984372091837
sim_compute_robot_state-npc2_median0.07030116870839109
sim_compute_robot_state-npc2_min0.06169039749902142
sim_compute_robot_state-npc3_max0.07333630781907302
sim_compute_robot_state-npc3_mean0.06917882474828892
sim_compute_robot_state-npc3_median0.07066713344483148
sim_compute_robot_state-npc3_min0.0611401687968861
sim_compute_sim_state_max0.041508983558332414
sim_compute_sim_state_mean0.03896970634945987
sim_compute_sim_state_median0.03882212282341217
sim_compute_sim_state_min0.03537971632821219
sim_physics_max0.22602363835985417
sim_physics_mean0.18720443152601063
sim_physics_median0.1913647426141275
sim_physics_min0.1563308948327687
sim_render-ego_max0.06709105234879714
sim_render-ego_mean0.06353737114164353
sim_render-ego_median0.06403279048140331
sim_render-ego_min0.05650463758730421
simulation-passed1
survival_time_max6.0499999999999865
survival_time_mean3.046666666666663
survival_time_min1.0500000000000005
No reset possible
21141step1-simulationsuccessno0:19:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20449step1-simulationsuccessno0:06:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19485step1-simulationsuccessno0:07:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18733step1-simulationsuccessno0:09:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18398step1-simulationsuccessno0:09:33
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18385step1-simulationhost-errorno0:08:25
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job18385-249543', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job18385-249543', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job18385-249543', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18025step1-simulationsuccessno0:05:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible