Duckietown Challenges Home Challenges Submissions

Submission 2387

Submission2387
Competingyes
Challengeaido2-LFVI-sim-testing
UserAndrea Censi 🇨🇭
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 23012
Next
User labelBaseline solution using reinforcement learning
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
23012step1-simulationsuccessyes0:16:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.14523351468985934
survival_time_median1.4000000000000006
deviation-center-line_median0.08497713018479955
in-drivable-lane_median0


other stats
agent_compute-ego_max0.18725702592304777
agent_compute-ego_mean0.17567452989060445
agent_compute-ego_median0.180315993048928
agent_compute-ego_min0.14577611287434897
deviation-center-line_max0.126934975442118
deviation-center-line_mean0.08525081949317097
deviation-center-line_min0.04858696627920683
deviation-heading_max1.033819055483286
deviation-heading_mean0.6765089999820442
deviation-heading_median0.7312061793015299
deviation-heading_min0.3939765693483576
driven_any_max0.3526677570839242
driven_any_mean0.2128572173226389
driven_any_median0.20436390945672597
driven_any_min0.1477013427501714
driven_lanedir_consec_max0.29929259841296796
driven_lanedir_consec_mean0.15770514620743484
driven_lanedir_consec_min0.08407039710299014
driven_lanedir_max0.29929259841296796
driven_lanedir_mean0.15770514620743484
driven_lanedir_median0.14523351468985934
driven_lanedir_min0.08407039710299014
in-drivable-lane_max0.20000000000000015
in-drivable-lane_mean0.02666666666666669
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 0.22323844186245775, "sim_physics": 0.19766045411427816, "survival_time": 1.5000000000000009, "driven_lanedir": 0.14019536993983284, "sim_render-ego": 0.06592878500620523, "in-drivable-lane": 0.10000000000000007, "agent_compute-ego": 0.1835347334543864, "deviation-heading": 0.8357187579461436, "set_robot_commands": 0.10422461827596027, "deviation-center-line": 0.08550713097523943, "driven_lanedir_consec": 0.14019536993983284, "sim_compute_sim_state": 0.041216190656026205, "sim_compute_performance-ego": 0.06867361863454183, "sim_compute_robot_state-ego": 0.07402807871500651, "sim_compute_robot_state-npc0": 0.07172816594441732, "sim_compute_robot_state-npc1": 0.06997857093811036, "sim_compute_robot_state-npc2": 0.07103020350138346, "sim_compute_robot_state-npc3": 0.07106603781382242}, "udem1-1-0": {"driven_any": 0.18012995616110353, "sim_physics": 0.23936511039733888, "survival_time": 1.2500000000000004, "driven_lanedir": 0.08407039710299014, "sim_render-ego": 0.07072819709777832, "in-drivable-lane": 0, "agent_compute-ego": 0.1769804859161377, "deviation-heading": 1.033819055483286, "set_robot_commands": 0.10455557823181152, "deviation-center-line": 0.08162634830061137, "driven_lanedir_consec": 0.08407039710299014, "sim_compute_sim_state": 0.03940306663513184, "sim_compute_performance-ego": 0.07518525123596191, "sim_compute_robot_state-ego": 0.07947986602783202, "sim_compute_robot_state-npc0": 0.0730130672454834, "sim_compute_robot_state-npc1": 0.07171165466308593, "sim_compute_robot_state-npc2": 0.07157830238342285, "sim_compute_robot_state-npc3": 0.07082037925720215}, "udem1-2-0": {"driven_any": 0.250471755422026, "sim_physics": 0.27827554089682444, "survival_time": 1.4000000000000006, "driven_lanedir": 0.2281620942726299, "sim_render-ego": 0.06667562893458776, "in-drivable-lane": 0, "agent_compute-ego": 0.18725702592304777, "deviation-heading": 0.3939765693483576, "set_robot_commands": 0.10474030460630145, "deviation-center-line": 0.126934975442118, "driven_lanedir_consec": 0.2281620942726299, "sim_compute_sim_state": 0.043453412396567206, "sim_compute_performance-ego": 0.07771669115339007, "sim_compute_robot_state-ego": 0.08003389835357666, "sim_compute_robot_state-npc0": 0.07430659021650042, "sim_compute_robot_state-npc1": 0.07002839020320348, "sim_compute_robot_state-npc2": 0.07195189169475011, "sim_compute_robot_state-npc3": 0.07111591952187675}, "udem1-3-0": {"driven_any": 0.25261300258240266, "sim_physics": 0.20352717103629275, "survival_time": 1.4500000000000006, "driven_lanedir": 0.22515444372813945, "sim_render-ego": 0.06048565075315278, "in-drivable-lane": 0, "agent_compute-ego": 0.17132508343663708, "deviation-heading": 0.4268127941473753, "set_robot_commands": 0.09689487259963464, "deviation-center-line": 0.11739770828591332, "driven_lanedir_consec": 0.22515444372813945, "sim_compute_sim_state": 0.0390486635010818, "sim_compute_performance-ego": 0.0674538201299207, "sim_compute_robot_state-ego": 0.06924776373238399, "sim_compute_robot_state-npc0": 0.06668769080063393, "sim_compute_robot_state-npc1": 0.07080064970871498, "sim_compute_robot_state-npc2": 0.06878680196301691, "sim_compute_robot_state-npc3": 0.074057019990066}, "udem1-4-0": {"driven_any": 0.1477013427501714, "sim_physics": 0.1939692043122791, "survival_time": 1.0500000000000005, "driven_lanedir": 0.09754920975519132, "sim_render-ego": 0.0567583924248105, "in-drivable-lane": 0, "agent_compute-ego": 0.14999411219642275, "deviation-heading": 0.652922639023533, "set_robot_commands": 0.09229127566019694, "deviation-center-line": 0.04858696627920683, "driven_lanedir_consec": 0.09754920975519132, "sim_compute_sim_state": 0.03588951201665969, "sim_compute_performance-ego": 0.06631424313499815, "sim_compute_robot_state-ego": 0.07062849544343494, "sim_compute_robot_state-npc0": 0.0710958753313337, "sim_compute_robot_state-npc1": 0.0685850325084868, "sim_compute_robot_state-npc2": 0.06773336728413899, "sim_compute_robot_state-npc3": 0.06418541499546596}, "udem1-5-0": {"driven_any": 0.24615540760076915, "sim_physics": 0.20518101964678084, "survival_time": 1.7500000000000009, "driven_lanedir": 0.1507164284825966, "sim_render-ego": 0.0607144832611084, "in-drivable-lane": 0.20000000000000015, "agent_compute-ego": 0.1775348254612514, "deviation-heading": 0.8964526573143058, "set_robot_commands": 0.10329758780343191, "deviation-center-line": 0.07681729198409296, "driven_lanedir_consec": 0.1507164284825966, "sim_compute_sim_state": 0.04161548614501953, "sim_compute_performance-ego": 0.0646083150591169, "sim_compute_robot_state-ego": 0.07477055277143206, "sim_compute_robot_state-npc0": 0.07292981147766113, "sim_compute_robot_state-npc1": 0.07240567888532366, "sim_compute_robot_state-npc2": 0.06957259859357562, "sim_compute_robot_state-npc3": 0.07308523314339774}, "udem1-6-0": {"driven_any": 0.15333606025511992, "sim_physics": 0.23998095772483133, "survival_time": 1.1000000000000003, "driven_lanedir": 0.11929183405305156, "sim_render-ego": 0.06640114567496559, "in-drivable-lane": 0, "agent_compute-ego": 0.180315993048928, "deviation-heading": 0.4912747985844088, "set_robot_commands": 0.10748335448178378, "deviation-center-line": 0.08069394423003465, "driven_lanedir_consec": 0.11929183405305156, "sim_compute_sim_state": 0.039895187724720345, "sim_compute_performance-ego": 0.0735530203038996, "sim_compute_robot_state-ego": 0.07648320631547408, "sim_compute_robot_state-npc0": 0.07308147170326927, "sim_compute_robot_state-npc1": 0.07896590232849121, "sim_compute_robot_state-npc2": 0.07044253566048363, "sim_compute_robot_state-npc3": 0.06907225738872182}, "udem1-7-0": {"driven_any": 0.3526677570839242, "sim_physics": 0.2249833784605327, "survival_time": 1.900000000000001, "driven_lanedir": 0.29929259841296796, "sim_render-ego": 0.08263530229267321, "in-drivable-lane": 0, "agent_compute-ego": 0.18696998295031095, "deviation-heading": 0.7444837771663263, "set_robot_commands": 0.1000728732661197, "deviation-center-line": 0.08497713018479955, "driven_lanedir_consec": 0.29929259841296796, "sim_compute_sim_state": 0.03726961738184879, "sim_compute_performance-ego": 0.07718517905787418, "sim_compute_robot_state-ego": 0.07641659912310149, "sim_compute_robot_state-npc0": 0.07437463183152049, "sim_compute_robot_state-npc1": 0.07248008251190186, "sim_compute_robot_state-npc2": 0.06906144869954962, "sim_compute_robot_state-npc3": 0.06865576066468891}, "udem1-8-0": {"driven_any": 0.2372517209062659, "sim_physics": 0.1988253116607666, "survival_time": 1.5000000000000009, "driven_lanedir": 0.18298050574022895, "sim_render-ego": 0.05840079784393311, "in-drivable-lane": 0, "agent_compute-ego": 0.1712867816289266, "deviation-heading": 0.7312061793015299, "set_robot_commands": 0.09620696703592936, "deviation-center-line": 0.06781389732873722, "driven_lanedir_consec": 0.18298050574022895, "sim_compute_sim_state": 0.0375299612681071, "sim_compute_performance-ego": 0.06407826741536458, "sim_compute_robot_state-ego": 0.06659525235493978, "sim_compute_robot_state-npc0": 0.06913192272186279, "sim_compute_robot_state-npc1": 0.06530350844065348, "sim_compute_robot_state-npc2": 0.06991778214772543, "sim_compute_robot_state-npc3": 0.06722100575764973}, "udem1-9-0": {"driven_any": 0.17415719061358204, "sim_physics": 0.20092302560806272, "survival_time": 1.2000000000000004, "driven_lanedir": 0.11310527977275942, "sim_render-ego": 0.054279992977778115, "in-drivable-lane": 0, "agent_compute-ego": 0.14577611287434897, "deviation-heading": 0.7855761619095886, "set_robot_commands": 0.09289162357648212, "deviation-center-line": 0.060580475542081624, "driven_lanedir_consec": 0.11310527977275942, "sim_compute_sim_state": 0.03403161962827047, "sim_compute_performance-ego": 0.057645956675211586, "sim_compute_robot_state-ego": 0.06122162938117981, "sim_compute_robot_state-npc0": 0.05876157681147257, "sim_compute_robot_state-npc1": 0.06015729904174805, "sim_compute_robot_state-npc2": 0.062277386585871376, "sim_compute_robot_state-npc3": 0.06622812151908875}, "udem1-10-0": {"driven_any": 0.1775376408210182, "sim_physics": 0.2206935187180837, "survival_time": 1.2000000000000004, "driven_lanedir": 0.12381222931095914, "sim_render-ego": 0.06753432750701904, "in-drivable-lane": 0, "agent_compute-ego": 0.18400327364603675, "deviation-heading": 0.7708886854166557, "set_robot_commands": 0.10112765431404114, "deviation-center-line": 0.062021067259149576, "driven_lanedir_consec": 0.12381222931095914, "sim_compute_sim_state": 0.03997581203778585, "sim_compute_performance-ego": 0.06960904598236084, "sim_compute_robot_state-ego": 0.07437817255655925, "sim_compute_robot_state-npc0": 0.07178406914075215, "sim_compute_robot_state-npc1": 0.07006735603014629, "sim_compute_robot_state-npc2": 0.0692408283551534, "sim_compute_robot_state-npc3": 0.07510911424954732}, "udem1-11-0": {"driven_any": 0.1794310167596238, "sim_physics": 0.2290826141834259, "survival_time": 1.2000000000000004, "driven_lanedir": 0.14568196093737473, "sim_render-ego": 0.06957744558652242, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.18094202876091003, "deviation-heading": 0.4482060771664075, "set_robot_commands": 0.10366002718607584, "deviation-center-line": 0.10656259478632273, "driven_lanedir_consec": 0.14568196093737473, "sim_compute_sim_state": 0.0417493482430776, "sim_compute_performance-ego": 0.07357832789421082, "sim_compute_robot_state-ego": 0.07852040727933247, "sim_compute_robot_state-npc0": 0.0737403432528178, "sim_compute_robot_state-npc1": 0.07384456197420756, "sim_compute_robot_state-npc2": 0.07205276687939961, "sim_compute_robot_state-npc3": 0.07278125484784444}, "udem1-12-0": {"driven_any": 0.22965941428413308, "sim_physics": 0.2604655186335246, "survival_time": 1.5000000000000009, "driven_lanedir": 0.14238482842007372, "sim_render-ego": 0.064309557278951, "in-drivable-lane": 0, "agent_compute-ego": 0.1840127070744832, "deviation-heading": 0.9103947187317212, "set_robot_commands": 0.1070529063542684, "deviation-center-line": 0.09694679977669649, "driven_lanedir_consec": 0.14238482842007372, "sim_compute_sim_state": 0.04143364429473877, "sim_compute_performance-ego": 0.0719437837600708, "sim_compute_robot_state-ego": 0.08220643202463786, "sim_compute_robot_state-npc0": 0.07656638622283936, "sim_compute_robot_state-npc1": 0.07322540283203124, "sim_compute_robot_state-npc2": 0.07516391277313232, "sim_compute_robot_state-npc3": 0.07450987497965494}, "udem1-13-0": {"driven_any": 0.18414364328026048, "sim_physics": 0.23552048206329343, "survival_time": 1.2000000000000004, "driven_lanedir": 0.14523351468985934, "sim_render-ego": 0.0629586676756541, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.18709927797317505, "deviation-heading": 0.427058333848968, "set_robot_commands": 0.10191234946250916, "deviation-center-line": 0.09424601893995624, "driven_lanedir_consec": 0.14523351468985934, "sim_compute_sim_state": 0.04154495398203532, "sim_compute_performance-ego": 0.06965481241544087, "sim_compute_robot_state-ego": 0.08094162742296855, "sim_compute_robot_state-npc0": 0.0719952682654063, "sim_compute_robot_state-npc1": 0.06954748431841533, "sim_compute_robot_state-npc2": 0.07010817527770996, "sim_compute_robot_state-npc3": 0.0769634743531545}, "udem1-14-0": {"driven_any": 0.20436390945672597, "sim_physics": 0.21701391254152572, "survival_time": 1.4000000000000006, "driven_lanedir": 0.16794649849286758, "sim_render-ego": 0.06366738251277379, "in-drivable-lane": 0, "agent_compute-ego": 0.16808552401406424, "deviation-heading": 0.5988437943420544, "set_robot_commands": 0.09359755686351232, "deviation-center-line": 0.0880499430826044, "driven_lanedir_consec": 0.16794649849286758, "sim_compute_sim_state": 0.04059592315128872, "sim_compute_performance-ego": 0.0699651667049953, "sim_compute_robot_state-ego": 0.07955270154135567, "sim_compute_robot_state-npc0": 0.07301721402576991, "sim_compute_robot_state-npc1": 0.07220934970038277, "sim_compute_robot_state-npc2": 0.07139637640544347, "sim_compute_robot_state-npc3": 0.07405575684138707}}
set_robot_commands_max0.10748335448178378
set_robot_commands_mean0.10066730331453724
set_robot_commands_median0.10191234946250916
set_robot_commands_min0.09229127566019694
sim_compute_performance-ego_max0.07771669115339007
sim_compute_performance-ego_mean0.06981103330382386
sim_compute_performance-ego_median0.06965481241544087
sim_compute_performance-ego_min0.057645956675211586
sim_compute_robot_state-ego_max0.08220643202463786
sim_compute_robot_state-ego_mean0.07496697886954769
sim_compute_robot_state-ego_median0.07641659912310149
sim_compute_robot_state-ego_min0.06122162938117981
sim_compute_robot_state-npc0_max0.07656638622283936
sim_compute_robot_state-npc0_mean0.07148093899944936
sim_compute_robot_state-npc0_median0.07292981147766113
sim_compute_robot_state-npc0_min0.05876157681147257
sim_compute_robot_state-npc1_max0.07896590232849121
sim_compute_robot_state-npc1_mean0.07062072827232686
sim_compute_robot_state-npc1_median0.07080064970871498
sim_compute_robot_state-npc1_min0.06015729904174805
sim_compute_robot_state-npc2_max0.07516391277313232
sim_compute_robot_state-npc2_mean0.07002095854698379
sim_compute_robot_state-npc2_median0.07010817527770996
sim_compute_robot_state-npc2_min0.062277386585871376
sim_compute_robot_state-npc3_max0.0769634743531545
sim_compute_robot_state-npc3_mean0.07126177502157123
sim_compute_robot_state-npc3_median0.07111591952187675
sim_compute_robot_state-npc3_min0.06418541499546596
sim_compute_sim_state_max0.043453412396567206
sim_compute_sim_state_mean0.03964349327082394
sim_compute_sim_state_median0.03997581203778585
sim_compute_sim_state_min0.03403161962827047
sim_physics_max0.27827554089682444
sim_physics_mean0.22303114799985607
sim_physics_median0.2206935187180837
sim_physics_min0.1939692043122791
sim_render-ego_max0.08263530229267321
sim_render-ego_mean0.06473705045519423
sim_render-ego_median0.064309557278951
sim_render-ego_min0.054279992977778115
simulation-passed1
survival_time_max1.900000000000001
survival_time_mean1.373333333333334
survival_time_min1.0500000000000005
No reset possible
21768step1-simulationsuccessno0:16:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21343step1-simulationsuccessno0:19:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20604step1-simulationsuccessno0:05:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20016step1-simulationsuccessno0:05:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19656step1-simulationsuccessno0:05:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19655step1-simulationsuccessno0:05:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18905step1-simulationsuccessno0:07:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18535step1-simulationsuccessno0:06:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17867step1-simulationerrorno0:21:01
Waited 1203 seconds [...]
Waited 1203 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17567step1-simulationsuccessno0:03:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17564step1-simulationhost-errorno0:01:18
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job17564-429', 'pull']:

   >  Command '['docker-compose', '-p', 'job17564-429', 'pull']' returned non-zero exit status 1.

stdout | 

stderr | Pulling evaluator      ...
stderr | Pulling scenario_maker ...
stderr | Pulling simulator      ...
stderr | Pulling solution       ...
stderr | 
stderr | ERROR: for simulator  write /var/lib/docker/tmp/GetImageBlob507463442: no space left on device
stderr | 
stderr | ERROR: for solution  write /var/lib/docker/tmp/GetImageBlob229330070: no space left on device
stderr | 
stderr | ERROR: for scenario_maker  failed to register layer: Error processing tar file(exit status 1): write /usr/local/lib/python3.7/site-packages/plotly/validators/layout/image/_x.py: no space left on device
stderr | 
stderr | ERROR: for evaluator  failed to register layer: Error processing tar file(exit status 1): write /usr/local/lib/python3.7/site-packages/plotly/validators/layout/image/_x.py: no space left on device
stderr | failed to register layer: Error processing tar file(exit status 1): write /usr/local/lib/python3.7/site-packages/plotly/validators/layout/image/_x.py: no space left on device
stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible