Duckietown Challenges Home Challenges Submissions

Submission 2860

Submission2860
Competingyes
Challengeaido2-LFV-sim-testing
UserAlexander Karavaev
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22735
Next
User labelchallenge-aido_LF-baseline-duckietown
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22735step1-simulationsuccessyes0:39:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.25218368833369853
survival_time_median4.749999999999991
deviation-center-line_median0.20983213757884128
in-drivable-lane_median0.1999999999999993


other stats
agent_compute-ego_max0.09105118284834192
agent_compute-ego_mean0.08587026076211965
agent_compute-ego_median0.08517571997969117
agent_compute-ego_min0.08247286689524748
deviation-center-line_max0.37412453759010306
deviation-center-line_mean0.2155734050031605
deviation-center-line_min0.10371361973602056
deviation-heading_max2.069370988539005
deviation-heading_mean1.6670455793043777
deviation-heading_median1.635119640893918
deviation-heading_min0.9825521661260276
driven_any_max1.170710411824839
driven_any_mean0.4132824323960856
driven_any_median0.35486274775616694
driven_any_min0.1780800336916172
driven_lanedir_consec_max0.5114936323244628
driven_lanedir_consec_mean0.3058586493358029
driven_lanedir_consec_min0.11405801011318316
driven_lanedir_max0.5114936323244628
driven_lanedir_mean0.3058586493358029
driven_lanedir_median0.25218368833369853
driven_lanedir_min0.11405801011318316
in-drivable-lane_max7.650000000000095
in-drivable-lane_mean0.7933333333333387
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.170710411824839, "sim_physics": 0.08288164854049683, "survival_time": 14.950000000000076, "driven_lanedir": 0.5114936323244628, "sim_render-ego": 0.05319547732671102, "in-drivable-lane": 7.650000000000095, "agent_compute-ego": 0.08992759148279826, "deviation-heading": 2.069370988539005, "set_robot_commands": 0.05130298693974813, "deviation-center-line": 0.37412453759010306, "driven_lanedir_consec": 0.5114936323244628, "sim_compute_sim_state": 0.032537830670674645, "sim_compute_performance-ego": 0.05681891202926636, "sim_compute_robot_state-ego": 0.05453875541687011, "sim_compute_robot_state-npc0": 0.06231400728225708, "sim_compute_robot_state-npc1": 0.05660987218221029, "sim_compute_robot_state-npc2": 0.05607960303624471, "sim_compute_robot_state-npc3": 0.05631816625595093}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.36692157244246953, "sim_physics": 0.08846360080096187, "survival_time": 4.899999999999991, "driven_lanedir": 0.25214523942886835, "sim_render-ego": 0.05518189741640675, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.08247286689524748, "deviation-heading": 1.7653657159567786, "set_robot_commands": 0.05121390430294737, "deviation-center-line": 0.20983213757884128, "driven_lanedir_consec": 0.25214523942886835, "sim_compute_sim_state": 0.03281571670454376, "sim_compute_performance-ego": 0.05858660474115489, "sim_compute_robot_state-ego": 0.05777047118362116, "sim_compute_robot_state-npc0": 0.0641951220376151, "sim_compute_robot_state-npc1": 0.0583872381521731, "sim_compute_robot_state-npc2": 0.058102267129080634, "sim_compute_robot_state-npc3": 0.05747341379827382}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.48345668693643706, "sim_physics": 0.08733441322807252, "survival_time": 6.349999999999985, "driven_lanedir": 0.4300286713380452, "sim_render-ego": 0.05362377016563115, "in-drivable-lane": 0.04999999999999982, "agent_compute-ego": 0.08659400526932844, "deviation-heading": 1.8881773426523991, "set_robot_commands": 0.05167493482274333, "deviation-center-line": 0.25047209567107825, "driven_lanedir_consec": 0.4300286713380452, "sim_compute_sim_state": 0.033184181048175485, "sim_compute_performance-ego": 0.05701401665454774, "sim_compute_robot_state-ego": 0.054637112955408774, "sim_compute_robot_state-npc0": 0.06370740988123136, "sim_compute_robot_state-npc1": 0.05799332190686324, "sim_compute_robot_state-npc2": 0.056522046487162435, "sim_compute_robot_state-npc3": 0.05705133385545626}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4955070992249157, "sim_physics": 0.08537300183222844, "survival_time": 6.499999999999985, "driven_lanedir": 0.42874206180997954, "sim_render-ego": 0.05397853667919452, "in-drivable-lane": 0.1999999999999993, "agent_compute-ego": 0.08461781465090239, "deviation-heading": 1.8917967573230448, "set_robot_commands": 0.05309905455662654, "deviation-center-line": 0.20427977572214137, "driven_lanedir_consec": 0.42874206180997954, "sim_compute_sim_state": 0.032499845211322494, "sim_compute_performance-ego": 0.058048151089594914, "sim_compute_robot_state-ego": 0.055400582460256725, "sim_compute_robot_state-npc0": 0.0638733882170457, "sim_compute_robot_state-npc1": 0.05859310443584736, "sim_compute_robot_state-npc2": 0.05716376121227558, "sim_compute_robot_state-npc3": 0.05711763455317571}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.22630257231887344, "sim_physics": 0.08180605797540574, "survival_time": 3.149999999999997, "driven_lanedir": 0.19274797603123983, "sim_render-ego": 0.053622158746870735, "in-drivable-lane": 0, "agent_compute-ego": 0.0873981203351702, "deviation-heading": 1.4137512103671486, "set_robot_commands": 0.05132218012734065, "deviation-center-line": 0.17397024821029894, "driven_lanedir_consec": 0.19274797603123983, "sim_compute_sim_state": 0.03415422590952071, "sim_compute_performance-ego": 0.057044994263421925, "sim_compute_robot_state-ego": 0.05517861956641787, "sim_compute_robot_state-npc0": 0.06680573735918317, "sim_compute_robot_state-npc1": 0.06024372388446142, "sim_compute_robot_state-npc2": 0.05756720664009216, "sim_compute_robot_state-npc3": 0.057853940933469745}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.2664667144323718, "sim_physics": 0.0814829656522568, "survival_time": 3.649999999999995, "driven_lanedir": 0.1575796771632696, "sim_render-ego": 0.05484141062383782, "in-drivable-lane": 0.7499999999999973, "agent_compute-ego": 0.08517571997969117, "deviation-heading": 1.5888994278309865, "set_robot_commands": 0.051306120336872256, "deviation-center-line": 0.13020760794673267, "driven_lanedir_consec": 0.1575796771632696, "sim_compute_sim_state": 0.03370447028173159, "sim_compute_performance-ego": 0.05866886818245666, "sim_compute_robot_state-ego": 0.05701434448973773, "sim_compute_robot_state-npc0": 0.06605274056735104, "sim_compute_robot_state-npc1": 0.06003521239920838, "sim_compute_robot_state-npc2": 0.05916915854362592, "sim_compute_robot_state-npc3": 0.05915646030478282}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.5397224274752584, "sim_physics": 0.08445363518194104, "survival_time": 7.049999999999983, "driven_lanedir": 0.5075765760347429, "sim_render-ego": 0.05250414719818332, "in-drivable-lane": 0, "agent_compute-ego": 0.09105118284834192, "deviation-heading": 1.648973699691825, "set_robot_commands": 0.052535872087411, "deviation-center-line": 0.32947311010944863, "driven_lanedir_consec": 0.5075765760347429, "sim_compute_sim_state": 0.03225494952912026, "sim_compute_performance-ego": 0.05587052960767814, "sim_compute_robot_state-ego": 0.05428407885504107, "sim_compute_robot_state-npc0": 0.06184380274292425, "sim_compute_robot_state-npc1": 0.05620940695417688, "sim_compute_robot_state-npc2": 0.05547889580963351, "sim_compute_robot_state-npc3": 0.05524934606349215}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3026382869669593, "sim_physics": 0.08848261542436553, "survival_time": 4.099999999999993, "driven_lanedir": 0.25218368833369853, "sim_render-ego": 0.05358379352383497, "in-drivable-lane": 0.04999999999999982, "agent_compute-ego": 0.08357538537281316, "deviation-heading": 1.536323415898871, "set_robot_commands": 0.05013872937458318, "deviation-center-line": 0.21658240503182383, "driven_lanedir_consec": 0.25218368833369853, "sim_compute_sim_state": 0.03254719187573689, "sim_compute_performance-ego": 0.05672152740199391, "sim_compute_robot_state-ego": 0.05418586440202666, "sim_compute_robot_state-npc0": 0.06458212108146853, "sim_compute_robot_state-npc1": 0.05909150402720382, "sim_compute_robot_state-npc2": 0.057856867953044615, "sim_compute_robot_state-npc3": 0.05779612936624667}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.5356986453649202, "sim_physics": 0.08822539704186576, "survival_time": 6.999999999999983, "driven_lanedir": 0.4429807871634335, "sim_render-ego": 0.05255190644945417, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.08888246161597116, "deviation-heading": 2.061635742829224, "set_robot_commands": 0.05271640675408499, "deviation-center-line": 0.23457868778466737, "driven_lanedir_consec": 0.4429807871634335, "sim_compute_sim_state": 0.032529430730002265, "sim_compute_performance-ego": 0.056716188362666536, "sim_compute_robot_state-ego": 0.05466761929648263, "sim_compute_robot_state-npc0": 0.06194566147668021, "sim_compute_robot_state-npc1": 0.05765414067677089, "sim_compute_robot_state-npc2": 0.056849839006151474, "sim_compute_robot_state-npc3": 0.05655909436089652}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.35486274775616694, "sim_physics": 0.08827905905874152, "survival_time": 4.749999999999991, "driven_lanedir": 0.25300522529531344, "sim_render-ego": 0.05287644486678274, "in-drivable-lane": 0.6999999999999975, "agent_compute-ego": 0.08327339573910361, "deviation-heading": 1.5878413075379836, "set_robot_commands": 0.05057114049008018, "deviation-center-line": 0.1191174151016822, "driven_lanedir_consec": 0.25300522529531344, "sim_compute_sim_state": 0.032388235393323396, "sim_compute_performance-ego": 0.056492687526502106, "sim_compute_robot_state-ego": 0.05775879056830155, "sim_compute_robot_state-npc0": 0.0622470403972425, "sim_compute_robot_state-npc1": 0.05648699810630397, "sim_compute_robot_state-npc2": 0.05537967179950915, "sim_compute_robot_state-npc3": 0.054930998149671055}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.1780800336916172, "sim_physics": 0.09908286730448404, "survival_time": 2.549999999999999, "driven_lanedir": 0.11405801011318316, "sim_render-ego": 0.05486430841333726, "in-drivable-lane": 0.1999999999999993, "agent_compute-ego": 0.08318150277231254, "deviation-heading": 1.5419948809531043, "set_robot_commands": 0.05011044764051251, "deviation-center-line": 0.10371361973602056, "driven_lanedir_consec": 0.11405801011318316, "sim_compute_sim_state": 0.03315352458579868, "sim_compute_performance-ego": 0.056989524878707586, "sim_compute_robot_state-ego": 0.05543794351465562, "sim_compute_robot_state-npc0": 0.0637347511216706, "sim_compute_robot_state-npc1": 0.059572303996366614, "sim_compute_robot_state-npc2": 0.05734159432205499, "sim_compute_robot_state-npc3": 0.05739024106193991}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.23434109684843313, "sim_physics": 0.08188884808466984, "survival_time": 3.2499999999999964, "driven_lanedir": 0.21561105937740743, "sim_render-ego": 0.04977302918067345, "in-drivable-lane": 0, "agent_compute-ego": 0.0830416936140794, "deviation-heading": 0.9825521661260276, "set_robot_commands": 0.051395460275503305, "deviation-center-line": 0.30398195222892455, "driven_lanedir_consec": 0.21561105937740743, "sim_compute_sim_state": 0.03187708854675293, "sim_compute_performance-ego": 0.05422178781949557, "sim_compute_robot_state-ego": 0.05495402262761043, "sim_compute_robot_state-npc0": 0.06081647139329176, "sim_compute_robot_state-npc1": 0.053957297251774713, "sim_compute_robot_state-npc2": 0.05542619411761944, "sim_compute_robot_state-npc3": 0.05538208668048565}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.26646811728130165, "sim_physics": 0.08991686611959379, "survival_time": 3.649999999999995, "driven_lanedir": 0.15896732478047337, "sim_render-ego": 0.05624907637295658, "in-drivable-lane": 0.6999999999999975, "agent_compute-ego": 0.08783687304144036, "deviation-heading": 1.635119640893918, "set_robot_commands": 0.050937907336509394, "deviation-center-line": 0.15011388737425768, "driven_lanedir_consec": 0.15896732478047337, "sim_compute_sim_state": 0.0339503288269043, "sim_compute_performance-ego": 0.05749516617761899, "sim_compute_robot_state-ego": 0.05559496683617161, "sim_compute_robot_state-npc0": 0.06519126892089844, "sim_compute_robot_state-npc1": 0.059252529928129016, "sim_compute_robot_state-npc2": 0.057640928111664234, "sim_compute_robot_state-npc3": 0.058600246089778533}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.535688530228512, "sim_physics": 0.08621761117662702, "survival_time": 6.999999999999983, "driven_lanedir": 0.4656862071605814, "sim_render-ego": 0.055223378113337926, "in-drivable-lane": 0.2499999999999991, "agent_compute-ego": 0.08787794283458165, "deviation-heading": 1.912642083658619, "set_robot_commands": 0.05181933130536761, "deviation-center-line": 0.22956038862420117, "driven_lanedir_consec": 0.4656862071605814, "sim_compute_sim_state": 0.03285773311342512, "sim_compute_performance-ego": 0.05903068270002093, "sim_compute_robot_state-ego": 0.05592107772827149, "sim_compute_robot_state-npc0": 0.06460491078240531, "sim_compute_robot_state-npc1": 0.05915452071598598, "sim_compute_robot_state-npc2": 0.05823150702885219, "sim_compute_robot_state-npc3": 0.05789847714560373}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.2423715431482093, "sim_physics": 0.09187323299806507, "survival_time": 3.349999999999996, "driven_lanedir": 0.20507360368234595, "sim_render-ego": 0.05449714945323432, "in-drivable-lane": 0, "agent_compute-ego": 0.08314735498001326, "deviation-heading": 1.4812393093067262, "set_robot_commands": 0.05164864881714778, "deviation-center-line": 0.20359320633718628, "driven_lanedir_consec": 0.20507360368234595, "sim_compute_sim_state": 0.03269633962147271, "sim_compute_performance-ego": 0.05871498406823002, "sim_compute_robot_state-ego": 0.05543705242783276, "sim_compute_robot_state-npc0": 0.061740248950559705, "sim_compute_robot_state-npc1": 0.05619428406900434, "sim_compute_robot_state-npc2": 0.054300511061255614, "sim_compute_robot_state-npc3": 0.05422591095540061}}
set_robot_commands_max0.05309905455662654
set_robot_commands_mean0.051452875011165224
set_robot_commands_median0.05132218012734065
set_robot_commands_min0.05011044764051251
sim_compute_performance-ego_max0.05903068270002093
sim_compute_performance-ego_mean0.05722897503355708
sim_compute_performance-ego_median0.05701401665454774
sim_compute_performance-ego_min0.05422178781949557
sim_compute_robot_state-ego_max0.05777047118362116
sim_compute_robot_state-ego_mean0.05551875348858041
sim_compute_robot_state-ego_median0.055400582460256725
sim_compute_robot_state-ego_min0.05418586440202666
sim_compute_robot_state-npc0_max0.06680573735918317
sim_compute_robot_state-npc0_mean0.06357697881412165
sim_compute_robot_state-npc0_median0.0637347511216706
sim_compute_robot_state-npc0_min0.06081647139329176
sim_compute_robot_state-npc1_max0.06024372388446142
sim_compute_robot_state-npc1_mean0.05796236391243201
sim_compute_robot_state-npc1_median0.0583872381521731
sim_compute_robot_state-npc1_min0.053957297251774713
sim_compute_robot_state-npc2_max0.05916915854362592
sim_compute_robot_state-npc2_mean0.056874003483884446
sim_compute_robot_state-npc2_median0.05716376121227558
sim_compute_robot_state-npc2_min0.054300511061255614
sim_compute_robot_state-npc3_max0.05915646030478282
sim_compute_robot_state-npc3_mean0.05686689863830827
sim_compute_robot_state-npc3_median0.05711763455317571
sim_compute_robot_state-npc3_min0.05422591095540061
sim_compute_sim_state_max0.03415422590952071
sim_compute_sim_state_mean0.03287673946990035
sim_compute_sim_state_median0.03269633962147271
sim_compute_sim_state_min0.03187708854675293
sim_physics_max0.09908286730448404
sim_physics_mean0.08705078802798506
sim_physics_median0.08733441322807252
sim_physics_min0.0814829656522568
sim_render-ego_max0.05624907637295658
sim_render-ego_mean0.05377109896869645
sim_render-ego_median0.05362377016563115
sim_render-ego_min0.04977302918067345
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean5.479999999999998
survival_time_min2.549999999999999
No reset possible
20954step1-simulationsuccessno0:50:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20282step1-simulationsuccessno0:19:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19334step1-simulationsuccessno0:19:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19304step1-simulationhost-errorno0:07:01
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job19304-32762', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job19304-32762', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job19304-32762', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18590step1-simulationerrorno0:23:49
Waited 1204 seconds [...]
Waited 1204 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18562step1-simulationerrorno0:21:21
Waited 1201 seconds [...]
Waited 1201 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible