Duckietown Challenges Home Challenges Submissions

Submission 2928

Submission2928
Competingyes
Challengeaido2-LFVI-sim-testing
UserBhairav Mehta
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22709
Next
User labelchallenge-aido_LF-template-pytorch
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22709step1-simulationsuccessyes0:14:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.10407850696719768
survival_time_median1.3500000000000003
deviation-center-line_median0.07935366019771346
in-drivable-lane_median0


other stats
agent_compute-ego_max0.1936461002595963
agent_compute-ego_mean0.17366470128496994
agent_compute-ego_median0.17245185736453894
agent_compute-ego_min0.1638822283063616
deviation-center-line_max0.11329928690807677
deviation-center-line_mean0.07846300654667009
deviation-center-line_min0.032409236397548925
deviation-heading_max0.9411919545625084
deviation-heading_mean0.7076452082752028
deviation-heading_median0.7622874851734545
deviation-heading_min0.44428440435091177
driven_any_max0.2628529286754375
driven_any_mean0.1772026742099274
driven_any_median0.17143802982765077
driven_any_min0.1269254890598757
driven_lanedir_consec_max0.16271008248963503
driven_lanedir_consec_mean0.10574752603185728
driven_lanedir_consec_min0.047418833310535424
driven_lanedir_max0.16271008248963503
driven_lanedir_mean0.10574752603185728
driven_lanedir_median0.10407850696719768
driven_lanedir_min0.047418833310535424
in-drivable-lane_max0.6000000000000005
in-drivable-lane_mean0.17000000000000018
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 0.226020028235494, "sim_physics": 0.12203219277518136, "survival_time": 1.7500000000000009, "driven_lanedir": 0.08654801542399593, "sim_render-ego": 0.06351450511387416, "in-drivable-lane": 0.5000000000000004, "agent_compute-ego": 0.1638822283063616, "deviation-heading": 0.7978594262175611, "set_robot_commands": 0.0783759457724435, "deviation-center-line": 0.07225914045159813, "driven_lanedir_consec": 0.08654801542399593, "sim_compute_sim_state": 0.03710479055132185, "sim_compute_performance-ego": 0.06641155651637486, "sim_compute_robot_state-ego": 0.06478015354701451, "sim_compute_robot_state-npc0": 0.07370881353105818, "sim_compute_robot_state-npc1": 0.06829301970345633, "sim_compute_robot_state-npc2": 0.06629984719412667, "sim_compute_robot_state-npc3": 0.06625949995858328}, "udem1-1-0": {"driven_any": 0.19796123229659057, "sim_physics": 0.13174055604373708, "survival_time": 1.7000000000000008, "driven_lanedir": 0.05676340460595197, "sim_render-ego": 0.07268070473390467, "in-drivable-lane": 0.6000000000000005, "agent_compute-ego": 0.1781140635995304, "deviation-heading": 0.8576618778741829, "set_robot_commands": 0.08487992426928352, "deviation-center-line": 0.0578250744099173, "driven_lanedir_consec": 0.05676340460595197, "sim_compute_sim_state": 0.038379486869363225, "sim_compute_performance-ego": 0.06590121633866254, "sim_compute_robot_state-ego": 0.06426229196436264, "sim_compute_robot_state-npc0": 0.07707880525028005, "sim_compute_robot_state-npc1": 0.07095037488376393, "sim_compute_robot_state-npc2": 0.07069291086757884, "sim_compute_robot_state-npc3": 0.07046603455263026}, "udem1-2-0": {"driven_any": 0.17143802982765077, "sim_physics": 0.15401307741800943, "survival_time": 1.2000000000000004, "driven_lanedir": 0.14090818374093717, "sim_render-ego": 0.06712985038757324, "in-drivable-lane": 0, "agent_compute-ego": 0.17216860254605612, "deviation-heading": 0.44428440435091177, "set_robot_commands": 0.07927520076433818, "deviation-center-line": 0.10924605007966089, "driven_lanedir_consec": 0.14090818374093717, "sim_compute_sim_state": 0.04070764780044556, "sim_compute_performance-ego": 0.06331908702850342, "sim_compute_robot_state-ego": 0.0646067460378011, "sim_compute_robot_state-npc0": 0.07528601090113322, "sim_compute_robot_state-npc1": 0.06983419259389241, "sim_compute_robot_state-npc2": 0.07152884205182393, "sim_compute_robot_state-npc3": 0.06819771726926167}, "udem1-3-0": {"driven_any": 0.13852700267795226, "sim_physics": 0.12523233890533447, "survival_time": 1.2000000000000004, "driven_lanedir": 0.10407850696719768, "sim_render-ego": 0.06426910559336345, "in-drivable-lane": 0, "agent_compute-ego": 0.17729471127192178, "deviation-heading": 0.5345248218447747, "set_robot_commands": 0.07730292280515035, "deviation-center-line": 0.10184501129475498, "driven_lanedir_consec": 0.10407850696719768, "sim_compute_sim_state": 0.03709969917933146, "sim_compute_performance-ego": 0.0641868809858958, "sim_compute_robot_state-ego": 0.06216353178024292, "sim_compute_robot_state-npc0": 0.0688688059647878, "sim_compute_robot_state-npc1": 0.06417105595270793, "sim_compute_robot_state-npc2": 0.06422126293182373, "sim_compute_robot_state-npc3": 0.06596569220225017}, "udem1-4-0": {"driven_any": 0.14161356519552518, "sim_physics": 0.11634216705958048, "survival_time": 1.2000000000000004, "driven_lanedir": 0.08476166284401465, "sim_render-ego": 0.06440457701683044, "in-drivable-lane": 0, "agent_compute-ego": 0.17664223909378052, "deviation-heading": 0.8417087740253795, "set_robot_commands": 0.07836831609408061, "deviation-center-line": 0.060091245678630174, "driven_lanedir_consec": 0.08476166284401465, "sim_compute_sim_state": 0.03754156827926636, "sim_compute_performance-ego": 0.06778471668561299, "sim_compute_robot_state-ego": 0.06236044565836588, "sim_compute_robot_state-npc0": 0.07536571224530537, "sim_compute_robot_state-npc1": 0.067769189675649, "sim_compute_robot_state-npc2": 0.06728749473889668, "sim_compute_robot_state-npc3": 0.0683544675509135}, "udem1-5-0": {"driven_any": 0.2232801826460644, "sim_physics": 0.11671210737789378, "survival_time": 1.7000000000000008, "driven_lanedir": 0.1179701245467859, "sim_render-ego": 0.06283866657930262, "in-drivable-lane": 0.2500000000000002, "agent_compute-ego": 0.16595596425673542, "deviation-heading": 0.8443542324198109, "set_robot_commands": 0.07677379776449765, "deviation-center-line": 0.06534187426511774, "driven_lanedir_consec": 0.1179701245467859, "sim_compute_sim_state": 0.03871790100546444, "sim_compute_performance-ego": 0.06475526445052203, "sim_compute_robot_state-ego": 0.06127431813408347, "sim_compute_robot_state-npc0": 0.07360235382528867, "sim_compute_robot_state-npc1": 0.07101806472329532, "sim_compute_robot_state-npc2": 0.07030540354111615, "sim_compute_robot_state-npc3": 0.06811556395362406}, "udem1-6-0": {"driven_any": 0.13363311490769458, "sim_physics": 0.12994972519252612, "survival_time": 1.1500000000000004, "driven_lanedir": 0.09527939082349768, "sim_render-ego": 0.0628249748893406, "in-drivable-lane": 0, "agent_compute-ego": 0.169420574022376, "deviation-heading": 0.5988256650089281, "set_robot_commands": 0.0758714364922565, "deviation-center-line": 0.08459454885384131, "driven_lanedir_consec": 0.09527939082349768, "sim_compute_sim_state": 0.03632035462752632, "sim_compute_performance-ego": 0.06327742079029912, "sim_compute_robot_state-ego": 0.06055793554886528, "sim_compute_robot_state-npc0": 0.06926229725713315, "sim_compute_robot_state-npc1": 0.06446358431940494, "sim_compute_robot_state-npc2": 0.06482963976652725, "sim_compute_robot_state-npc3": 0.06720036009083623}, "udem1-7-0": {"driven_any": 0.1269254890598757, "sim_physics": 0.1235283660888672, "survival_time": 1.2500000000000004, "driven_lanedir": 0.08745936511040209, "sim_render-ego": 0.06315938949584961, "in-drivable-lane": 0, "agent_compute-ego": 0.1847828197479248, "deviation-heading": 0.629304139940524, "set_robot_commands": 0.07654838562011719, "deviation-center-line": 0.09354020529256069, "driven_lanedir_consec": 0.08745936511040209, "sim_compute_sim_state": 0.03754948616027832, "sim_compute_performance-ego": 0.06485164642333985, "sim_compute_robot_state-ego": 0.06340003967285156, "sim_compute_robot_state-npc0": 0.07260515213012696, "sim_compute_robot_state-npc1": 0.06748380661010742, "sim_compute_robot_state-npc2": 0.06534461021423339, "sim_compute_robot_state-npc3": 0.06550566673278808}, "udem1-8-0": {"driven_any": 0.2038555128653818, "sim_physics": 0.11510230700174968, "survival_time": 1.5000000000000009, "driven_lanedir": 0.14046633158950828, "sim_render-ego": 0.06561904748280843, "in-drivable-lane": 0, "agent_compute-ego": 0.16594078540802, "deviation-heading": 0.853221767209334, "set_robot_commands": 0.0805949846903483, "deviation-center-line": 0.06770763713353496, "driven_lanedir_consec": 0.14046633158950828, "sim_compute_sim_state": 0.03952716986338298, "sim_compute_performance-ego": 0.06830883026123047, "sim_compute_robot_state-ego": 0.06293853918711344, "sim_compute_robot_state-npc0": 0.07590877215067546, "sim_compute_robot_state-npc1": 0.07077756722768148, "sim_compute_robot_state-npc2": 0.06785737673441569, "sim_compute_robot_state-npc3": 0.07052890459696452}, "udem1-9-0": {"driven_any": 0.17120178101086905, "sim_physics": 0.1297724030234597, "survival_time": 1.6500000000000008, "driven_lanedir": 0.047418833310535424, "sim_render-ego": 0.06682791131915468, "in-drivable-lane": 0.6000000000000005, "agent_compute-ego": 0.17245185736453894, "deviation-heading": 0.7622874851734545, "set_robot_commands": 0.08024412935430353, "deviation-center-line": 0.032409236397548925, "driven_lanedir_consec": 0.047418833310535424, "sim_compute_sim_state": 0.038032676234389794, "sim_compute_performance-ego": 0.06563304409836278, "sim_compute_robot_state-ego": 0.0617150321151271, "sim_compute_robot_state-npc0": 0.08100846319487601, "sim_compute_robot_state-npc1": 0.07328620101466324, "sim_compute_robot_state-npc2": 0.06923695766564572, "sim_compute_robot_state-npc3": 0.06653157869974773}, "udem1-10-0": {"driven_any": 0.15894183040576687, "sim_physics": 0.11890090836419, "survival_time": 1.3500000000000003, "driven_lanedir": 0.07370170822038391, "sim_render-ego": 0.06526282098558214, "in-drivable-lane": 0.2500000000000002, "agent_compute-ego": 0.1665361104188142, "deviation-heading": 0.7941932528603886, "set_robot_commands": 0.07734452353583442, "deviation-center-line": 0.041576002732649824, "driven_lanedir_consec": 0.07370170822038391, "sim_compute_sim_state": 0.03811501573633264, "sim_compute_performance-ego": 0.06775123984725387, "sim_compute_robot_state-ego": 0.06263996936656811, "sim_compute_robot_state-npc0": 0.07387700787297001, "sim_compute_robot_state-npc1": 0.06752317040054887, "sim_compute_robot_state-npc2": 0.06542277336120605, "sim_compute_robot_state-npc3": 0.0676791579635055}, "udem1-11-0": {"driven_any": 0.1749519295874412, "sim_physics": 0.11925950646400452, "survival_time": 1.2000000000000004, "driven_lanedir": 0.14920971405190508, "sim_render-ego": 0.06506540377934773, "in-drivable-lane": 0, "agent_compute-ego": 0.16974510749181113, "deviation-heading": 0.4885567221087282, "set_robot_commands": 0.07872288425763448, "deviation-center-line": 0.11329928690807677, "driven_lanedir_consec": 0.14920971405190508, "sim_compute_sim_state": 0.0386295219262441, "sim_compute_performance-ego": 0.06477199991544087, "sim_compute_robot_state-ego": 0.06360909342765808, "sim_compute_robot_state-npc0": 0.07832688093185425, "sim_compute_robot_state-npc1": 0.06927291552225749, "sim_compute_robot_state-npc2": 0.06895007689793904, "sim_compute_robot_state-npc3": 0.06776036818822224}, "udem1-12-0": {"driven_any": 0.2628529286754375, "sim_physics": 0.1234522116811652, "survival_time": 1.900000000000001, "driven_lanedir": 0.16271008248963503, "sim_render-ego": 0.06980030160201223, "in-drivable-lane": 0.20000000000000015, "agent_compute-ego": 0.17452376767208702, "deviation-heading": 0.9411919545625084, "set_robot_commands": 0.08049311763361881, "deviation-center-line": 0.10287515677260207, "driven_lanedir_consec": 0.16271008248963503, "sim_compute_sim_state": 0.03825703420137104, "sim_compute_performance-ego": 0.0694472350572285, "sim_compute_robot_state-ego": 0.06687121014845998, "sim_compute_robot_state-npc0": 0.07582385916458934, "sim_compute_robot_state-npc1": 0.06665663342726857, "sim_compute_robot_state-npc2": 0.0664693556333843, "sim_compute_robot_state-npc3": 0.06830739974975586}, "udem1-13-0": {"driven_any": 0.1463939866910194, "sim_physics": 0.12010735014210576, "survival_time": 1.1500000000000004, "driven_lanedir": 0.1173040843193145, "sim_render-ego": 0.06699322617572287, "in-drivable-lane": 0, "agent_compute-ego": 0.17386558781499448, "deviation-heading": 0.5064884477432491, "set_robot_commands": 0.07814673755479895, "deviation-center-line": 0.094980967731844, "driven_lanedir_consec": 0.1173040843193145, "sim_compute_sim_state": 0.038601989331452743, "sim_compute_performance-ego": 0.06641293608624002, "sim_compute_robot_state-ego": 0.062363842259282654, "sim_compute_robot_state-npc0": 0.07622518746749214, "sim_compute_robot_state-npc1": 0.07045203706492549, "sim_compute_robot_state-npc2": 0.06834895714469578, "sim_compute_robot_state-npc3": 0.06438441898511804}, "udem1-14-0": {"driven_any": 0.18044349906614737, "sim_physics": 0.11797150488822691, "survival_time": 1.5500000000000007, "driven_lanedir": 0.1216334824337939, "sim_render-ego": 0.06412178470242408, "in-drivable-lane": 0.15000000000000013, "agent_compute-ego": 0.1936461002595963, "deviation-heading": 0.7202151527883072, "set_robot_commands": 0.07685945110936318, "deviation-center-line": 0.07935366019771346, "driven_lanedir_consec": 0.1216334824337939, "sim_compute_sim_state": 0.0378918263220018, "sim_compute_performance-ego": 0.06635021394298922, "sim_compute_robot_state-ego": 0.06566325310737856, "sim_compute_robot_state-npc0": 0.07481781898006316, "sim_compute_robot_state-npc1": 0.06896448904468168, "sim_compute_robot_state-npc2": 0.06646873104956842, "sim_compute_robot_state-npc3": 0.06797670548962008}}
set_robot_commands_max0.08487992426928352
set_robot_commands_mean0.07865345051453795
set_robot_commands_median0.07836831609408061
set_robot_commands_min0.0758714364922565
sim_compute_performance-ego_max0.0694472350572285
sim_compute_performance-ego_mean0.06594421922853042
sim_compute_performance-ego_median0.06590121633866254
sim_compute_performance-ego_min0.06327742079029912
sim_compute_robot_state-ego_max0.06687121014845998
sim_compute_robot_state-ego_mean0.06328042679701168
sim_compute_robot_state-ego_median0.06293853918711344
sim_compute_robot_state-ego_min0.06055793554886528
sim_compute_robot_state-npc0_max0.08100846319487601
sim_compute_robot_state-npc0_mean0.07478439605784225
sim_compute_robot_state-npc0_median0.07528601090113322
sim_compute_robot_state-npc0_min0.0688688059647878
sim_compute_robot_state-npc1_max0.07328620101466324
sim_compute_robot_state-npc1_mean0.06872775347762028
sim_compute_robot_state-npc1_median0.06896448904468168
sim_compute_robot_state-npc1_min0.06417105595270793
sim_compute_robot_state-npc2_max0.07152884205182393
sim_compute_robot_state-npc2_mean0.06755094931953212
sim_compute_robot_state-npc2_median0.06728749473889668
sim_compute_robot_state-npc2_min0.06422126293182373
sim_compute_robot_state-npc3_max0.07052890459696452
sim_compute_robot_state-npc3_mean0.0675489023989214
sim_compute_robot_state-npc3_median0.06776036818822224
sim_compute_robot_state-npc3_min0.06438441898511804
sim_compute_sim_state_max0.04070764780044556
sim_compute_sim_state_mean0.03816507787254484
sim_compute_sim_state_median0.03811501573633264
sim_compute_sim_state_min0.03632035462752632
sim_physics_max0.15401307741800943
sim_physics_mean0.12427444816173544
sim_physics_median0.12203219277518136
sim_physics_min0.11510230700174968
sim_render-ego_max0.07268070473390467
sim_render-ego_mean0.06563415132380607
sim_render-ego_median0.06506540377934773
sim_render-ego_min0.0628249748893406
simulation-passed1
survival_time_max1.900000000000001
survival_time_mean1.4300000000000004
survival_time_min1.1500000000000004
No reset possible
21647step1-simulationsuccessno0:21:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20935step1-simulationsuccessno0:17:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20776step1-simulationsuccessno0:20:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20250step1-simulationsuccessno0:07:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20249step1-simulationsuccessno0:05:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20062step1-simulationsuccessno0:08:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20044step1-simulationhost-errorno0:01:32
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20044-956824', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20044-956824', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20044-956824', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19813step1-simulationsuccessno0:06:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19746step1-simulationsuccessno0:09:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19260step1-simulationsuccessno0:08:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19250step1-simulationsuccessno0:22:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19100step1-simulationsuccessno0:08:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible