Duckietown Challenges Home Challenges Submissions

Submission 2644

Submission2644
Competingyes
Challengeaido2-LFV-sim-testing
UserEgor Zamotaev
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22850
Next
User labelchallenge-aido_LF-template-ros - Template solution using ROS
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22850step1-simulationsuccessyes0:27:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.30314311735235044
survival_time_median2.6499999999999986
deviation-center-line_median0.1192462800776758
in-drivable-lane_median0.9499999999999966


other stats
agent_compute-ego_max0.07913545199802943
agent_compute-ego_mean0.07449237095353241
agent_compute-ego_median0.07496196031570435
agent_compute-ego_min0.06973845572084994
deviation-center-line_max0.2294363663374862
deviation-center-line_mean0.12653708755488874
deviation-center-line_min0.06426934298501713
deviation-heading_max0.6986268167330368
deviation-heading_mean0.44799520296706863
deviation-heading_median0.41847631252347706
deviation-heading_min0.2596285487829774
driven_any_max1.599733354117788
driven_any_mean0.8095342605803044
driven_any_median0.7336463183586153
driven_any_min0.26903550091980144
driven_lanedir_consec_max1.021002447119848
driven_lanedir_consec_mean0.448904809024379
driven_lanedir_consec_min0.2407721645627201
driven_lanedir_max1.021002447119848
driven_lanedir_mean0.448904809024379
driven_lanedir_median0.30314311735235044
driven_lanedir_min0.2407721645627201
in-drivable-lane_max4.799999999999986
in-drivable-lane_mean1.1466666666666634
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.922682252912863, "sim_physics": 0.18471095931362097, "survival_time": 3.5499999999999954, "driven_lanedir": 0.36592094708219225, "sim_render-ego": 0.0681402884738546, "in-drivable-lane": 1.8999999999999944, "agent_compute-ego": 0.07674082903794839, "deviation-heading": 0.2596285487829774, "set_robot_commands": 0.10487196479045167, "deviation-center-line": 0.16987818035809085, "driven_lanedir_consec": 0.36592094708219225, "sim_compute_sim_state": 0.04142232679984939, "sim_compute_performance-ego": 0.07401034865580813, "sim_compute_robot_state-ego": 0.07916034107476892, "sim_compute_robot_state-npc0": 0.07417391387509628, "sim_compute_robot_state-npc1": 0.0740076891133483, "sim_compute_robot_state-npc2": 0.07304646935261472, "sim_compute_robot_state-npc3": 0.07369482013541208}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.3642886245953926, "sim_physics": 0.21684011071920395, "survival_time": 1.6000000000000008, "driven_lanedir": 0.3026144243018116, "sim_render-ego": 0.06651924550533295, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.07500080019235611, "deviation-heading": 0.6986268167330368, "set_robot_commands": 0.1003878191113472, "deviation-center-line": 0.10463456752710916, "driven_lanedir_consec": 0.3026144243018116, "sim_compute_sim_state": 0.04406455159187317, "sim_compute_performance-ego": 0.07486630976200104, "sim_compute_robot_state-ego": 0.08481421321630478, "sim_compute_robot_state-npc0": 0.07373550534248352, "sim_compute_robot_state-npc1": 0.07279504090547562, "sim_compute_robot_state-npc2": 0.07589297741651535, "sim_compute_robot_state-npc3": 0.07735739648342133}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.7336463183586153, "sim_physics": 0.2091369995704064, "survival_time": 2.5999999999999988, "driven_lanedir": 0.727268775640014, "sim_render-ego": 0.0667339563369751, "in-drivable-lane": 0, "agent_compute-ego": 0.07496196031570435, "deviation-heading": 0.2723076711805525, "set_robot_commands": 0.1038284393457266, "deviation-center-line": 0.2294363663374862, "driven_lanedir_consec": 0.727268775640014, "sim_compute_sim_state": 0.04033804398316603, "sim_compute_performance-ego": 0.07680705418953529, "sim_compute_robot_state-ego": 0.0942600598702064, "sim_compute_robot_state-npc0": 0.07307683504544772, "sim_compute_robot_state-npc1": 0.0745918475664579, "sim_compute_robot_state-npc2": 0.07353613926814152, "sim_compute_robot_state-npc3": 0.07172696865521945}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.2972979671834337, "sim_physics": 0.21459559727740543, "survival_time": 4.6499999999999915, "driven_lanedir": 0.97716132702229, "sim_render-ego": 0.06811651619531775, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.0763096809387207, "deviation-heading": 0.4911127339226078, "set_robot_commands": 0.10620016436423024, "deviation-center-line": 0.18038572117738352, "driven_lanedir_consec": 0.97716132702229, "sim_compute_sim_state": 0.04581197871956774, "sim_compute_performance-ego": 0.07600889154659804, "sim_compute_robot_state-ego": 0.08712012024335963, "sim_compute_robot_state-npc0": 0.07462663291603007, "sim_compute_robot_state-npc1": 0.0729456409331291, "sim_compute_robot_state-npc2": 0.0736506215987667, "sim_compute_robot_state-npc3": 0.07684189529829127}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.26903550091980144, "sim_physics": 0.22047461782182967, "survival_time": 1.0500000000000005, "driven_lanedir": 0.2407721645627201, "sim_render-ego": 0.06980596269880023, "in-drivable-lane": 0, "agent_compute-ego": 0.07913545199802943, "deviation-heading": 0.41847631252347706, "set_robot_commands": 0.110781953448341, "deviation-center-line": 0.06426934298501713, "driven_lanedir_consec": 0.2407721645627201, "sim_compute_sim_state": 0.04277202061244419, "sim_compute_performance-ego": 0.07605912571861631, "sim_compute_robot_state-ego": 0.09032387960524786, "sim_compute_robot_state-npc0": 0.07845115661621094, "sim_compute_robot_state-npc1": 0.07843859990437825, "sim_compute_robot_state-npc2": 0.07839478765215192, "sim_compute_robot_state-npc3": 0.07631297338576544}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.599733354117788, "sim_physics": 0.18660945734701867, "survival_time": 6.0499999999999865, "driven_lanedir": 0.26415979567759074, "sim_render-ego": 0.06313815589778679, "in-drivable-lane": 4.799999999999986, "agent_compute-ego": 0.07163423743129761, "deviation-heading": 0.39392468133084674, "set_robot_commands": 0.09518232227356964, "deviation-center-line": 0.1277842828221673, "driven_lanedir_consec": 0.26415979567759074, "sim_compute_sim_state": 0.039934398714175895, "sim_compute_performance-ego": 0.06922459996436253, "sim_compute_robot_state-ego": 0.07652435617998612, "sim_compute_robot_state-npc0": 0.07374517779704953, "sim_compute_robot_state-npc1": 0.06864189116422795, "sim_compute_robot_state-npc2": 0.06964894562713371, "sim_compute_robot_state-npc3": 0.07039062444828759}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0951392646085512, "sim_physics": 0.18385265838532225, "survival_time": 4.199999999999993, "driven_lanedir": 0.5659790827231905, "sim_render-ego": 0.06321603059768677, "in-drivable-lane": 1.7499999999999938, "agent_compute-ego": 0.0700681209564209, "deviation-heading": 0.5971215168818823, "set_robot_commands": 0.10028538533619472, "deviation-center-line": 0.11288019725884578, "driven_lanedir_consec": 0.5659790827231905, "sim_compute_sim_state": 0.041574018342154365, "sim_compute_performance-ego": 0.07196937572388422, "sim_compute_robot_state-ego": 0.07807062637238275, "sim_compute_robot_state-npc0": 0.07139181806927636, "sim_compute_robot_state-npc1": 0.07443924177260626, "sim_compute_robot_state-npc2": 0.0732546675772894, "sim_compute_robot_state-npc3": 0.07280591556004115}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 1.0452441176451972, "sim_physics": 0.2022600625012372, "survival_time": 3.699999999999995, "driven_lanedir": 1.021002447119848, "sim_render-ego": 0.06366954623041926, "in-drivable-lane": 0, "agent_compute-ego": 0.06973845572084994, "deviation-heading": 0.6206310196168163, "set_robot_commands": 0.09734498487936484, "deviation-center-line": 0.14363020423695483, "driven_lanedir_consec": 1.021002447119848, "sim_compute_sim_state": 0.04205184691661113, "sim_compute_performance-ego": 0.0730354270419559, "sim_compute_robot_state-ego": 0.07726746314280741, "sim_compute_robot_state-npc0": 0.07229541121302424, "sim_compute_robot_state-npc1": 0.07130748838991732, "sim_compute_robot_state-npc2": 0.07191603892558329, "sim_compute_robot_state-npc3": 0.07142829250644993}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.0535850482576197, "sim_physics": 0.1801835364765591, "survival_time": 3.599999999999995, "driven_lanedir": 0.30314311735235044, "sim_render-ego": 0.06625636418660481, "in-drivable-lane": 2.2999999999999945, "agent_compute-ego": 0.07277243336041768, "deviation-heading": 0.40636604380954594, "set_robot_commands": 0.1004791027969784, "deviation-center-line": 0.10907487511555897, "driven_lanedir_consec": 0.30314311735235044, "sim_compute_sim_state": 0.04042817155520121, "sim_compute_performance-ego": 0.0732245213455624, "sim_compute_robot_state-ego": 0.0778932770093282, "sim_compute_robot_state-npc0": 0.07505567868550618, "sim_compute_robot_state-npc1": 0.07363707489437527, "sim_compute_robot_state-npc2": 0.07354340288374159, "sim_compute_robot_state-npc3": 0.07406127452850342}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.4799799001882503, "sim_physics": 0.22600426406503837, "survival_time": 5.349999999999989, "driven_lanedir": 0.5481518397312493, "sim_render-ego": 0.06761682590591574, "in-drivable-lane": 2.9499999999999895, "agent_compute-ego": 0.07438876472900961, "deviation-heading": 0.5514444521435958, "set_robot_commands": 0.10169498719901682, "deviation-center-line": 0.10072735470855042, "driven_lanedir_consec": 0.5481518397312493, "sim_compute_sim_state": 0.041966685624880214, "sim_compute_performance-ego": 0.07252001539569035, "sim_compute_robot_state-ego": 0.08037916967801959, "sim_compute_robot_state-npc0": 0.07218689562004303, "sim_compute_robot_state-npc1": 0.07675089568735283, "sim_compute_robot_state-npc2": 0.07254287015611881, "sim_compute_robot_state-npc3": 0.07131145156432535}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.29610946631760215, "sim_physics": 0.23385113256948967, "survival_time": 1.3500000000000003, "driven_lanedir": 0.27060744077321397, "sim_render-ego": 0.07031467225816515, "in-drivable-lane": 0, "agent_compute-ego": 0.07391037764372649, "deviation-heading": 0.5373504765871612, "set_robot_commands": 0.108041869269477, "deviation-center-line": 0.08269292286428997, "driven_lanedir_consec": 0.27060744077321397, "sim_compute_sim_state": 0.0427421905376293, "sim_compute_performance-ego": 0.07570041550530328, "sim_compute_robot_state-ego": 0.09127739623740869, "sim_compute_robot_state-npc0": 0.07642535810117368, "sim_compute_robot_state-npc1": 0.07611306508382161, "sim_compute_robot_state-npc2": 0.07740615915369105, "sim_compute_robot_state-npc3": 0.07607811468618887}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.27791471406425977, "sim_physics": 0.1971131706237793, "survival_time": 1.2500000000000004, "driven_lanedir": 0.2638710416011838, "sim_render-ego": 0.06515002250671387, "in-drivable-lane": 0, "agent_compute-ego": 0.07639513015747071, "deviation-heading": 0.3105689945980953, "set_robot_commands": 0.10110649108886718, "deviation-center-line": 0.1192462800776758, "driven_lanedir_consec": 0.2638710416011838, "sim_compute_sim_state": 0.04204245567321777, "sim_compute_performance-ego": 0.07428007125854492, "sim_compute_robot_state-ego": 0.07887042999267578, "sim_compute_robot_state-npc0": 0.07755794525146484, "sim_compute_robot_state-npc1": 0.07687727928161621, "sim_compute_robot_state-npc2": 0.07934060096740722, "sim_compute_robot_state-npc3": 0.07701337814331055}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.7174039891727135, "sim_physics": 0.1835240508025547, "survival_time": 2.6499999999999986, "driven_lanedir": 0.29766819613667395, "sim_render-ego": 0.06478748681410304, "in-drivable-lane": 1.249999999999998, "agent_compute-ego": 0.07591422098987508, "deviation-heading": 0.3122593881578996, "set_robot_commands": 0.10481437647117758, "deviation-center-line": 0.14209620948179827, "driven_lanedir_consec": 0.29766819613667395, "sim_compute_sim_state": 0.042504504041851694, "sim_compute_performance-ego": 0.0747360553381578, "sim_compute_robot_state-ego": 0.08460343558833285, "sim_compute_robot_state-npc0": 0.07672204611436376, "sim_compute_robot_state-npc1": 0.07669226628429485, "sim_compute_robot_state-npc2": 0.07408729139363991, "sim_compute_robot_state-npc3": 0.07464614904151773}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.3170173854101942, "sim_physics": 0.2183247612368676, "survival_time": 1.5500000000000007, "driven_lanedir": 0.30712670008511367, "sim_render-ego": 0.06563300471152028, "in-drivable-lane": 0, "agent_compute-ego": 0.07444872394684822, "deviation-heading": 0.2996283459319333, "set_robot_commands": 0.10492134863330473, "deviation-center-line": 0.13363659833059607, "driven_lanedir_consec": 0.30712670008511367, "sim_compute_sim_state": 0.04362577776755056, "sim_compute_performance-ego": 0.0727936452434909, "sim_compute_robot_state-ego": 0.0802373501562303, "sim_compute_robot_state-npc0": 0.07773786206399241, "sim_compute_robot_state-npc1": 0.07484193002024005, "sim_compute_robot_state-npc2": 0.07420422953944053, "sim_compute_robot_state-npc3": 0.07740851371519027}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.6739360049522848, "sim_physics": 0.20378382065716913, "survival_time": 2.549999999999999, "driven_lanedir": 0.27812483555624423, "sim_render-ego": 0.06450654478634105, "in-drivable-lane": 1.2499999999999984, "agent_compute-ego": 0.07596637688431085, "deviation-heading": 0.550481042305602, "set_robot_commands": 0.10269753605711694, "deviation-center-line": 0.07768321004180687, "driven_lanedir_consec": 0.27812483555624423, "sim_compute_sim_state": 0.03965565270068599, "sim_compute_performance-ego": 0.06960250817093194, "sim_compute_robot_state-ego": 0.0733328286339255, "sim_compute_robot_state-npc0": 0.07260991077797085, "sim_compute_robot_state-npc1": 0.07433451390733906, "sim_compute_robot_state-npc2": 0.07219679215375115, "sim_compute_robot_state-npc3": 0.07202864160724715}}
set_robot_commands_max0.110781953448341
set_robot_commands_mean0.10284258300434428
set_robot_commands_median0.10269753605711694
set_robot_commands_min0.09518232227356964
sim_compute_performance-ego_max0.07680705418953529
sim_compute_performance-ego_mean0.0736558909906962
sim_compute_performance-ego_median0.07401034865580813
sim_compute_performance-ego_min0.06922459996436253
sim_compute_robot_state-ego_max0.0942600598702064
sim_compute_robot_state-ego_mean0.082275663133399
sim_compute_robot_state-ego_median0.0802373501562303
sim_compute_robot_state-ego_min0.0733328286339255
sim_compute_robot_state-npc0_max0.07845115661621094
sim_compute_robot_state-npc0_mean0.07465280983260889
sim_compute_robot_state-npc0_median0.07417391387509628
sim_compute_robot_state-npc0_min0.07139181806927636
sim_compute_robot_state-npc1_max0.07843859990437825
sim_compute_robot_state-npc1_mean0.07442763099390538
sim_compute_robot_state-npc1_median0.07443924177260626
sim_compute_robot_state-npc1_min0.06864189116422795
sim_compute_robot_state-npc2_max0.07934060096740722
sim_compute_robot_state-npc2_mean0.07417746624439912
sim_compute_robot_state-npc2_median0.07354340288374159
sim_compute_robot_state-npc2_min0.06964894562713371
sim_compute_robot_state-npc3_max0.07740851371519027
sim_compute_robot_state-npc3_mean0.07420709398394476
sim_compute_robot_state-npc3_median0.07406127452850342
sim_compute_robot_state-npc3_min0.07039062444828759
sim_compute_sim_state_max0.04581197871956774
sim_compute_sim_state_mean0.04206230823872391
sim_compute_sim_state_median0.04204245567321777
sim_compute_sim_state_min0.03965565270068599
sim_physics_max0.23385113256948967
sim_physics_mean0.20408434662450017
sim_physics_median0.20378382065716913
sim_physics_min0.1801835364765591
sim_render-ego_max0.07031467225816515
sim_render-ego_mean0.06624030820703583
sim_render-ego_median0.06625636418660481
sim_render-ego_min0.06313815589778679
simulation-passed1
survival_time_max6.0499999999999865
survival_time_mean3.046666666666663
survival_time_min1.0500000000000005
No reset possible
21115step1-simulationsuccessno0:24:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20432step1-simulationsuccessno0:10:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20420step1-simulationhost-errorno0:07:58
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20420-300852', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20420-300852', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20420-300852', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20417step1-simulationsuccessno0:08:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19470step1-simulationsuccessno0:06:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19447step1-simulationhost-errorno0:02:13
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job19447-995485', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job19447-995485', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job19447-995485', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18709step1-simulationsuccessno0:09:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18367step1-simulationsuccessno0:06:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18046step1-simulationsuccessno0:05:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible