Duckietown Challenges Home Challenges Submissions

Submission 2366

Submission2366
Competingyes
Challengeaido2-LFVI-sim-testing
UserAndrea Censi 🇨🇭
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 23030
Next
User labelrandom_agent
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
23030step1-simulationsuccessyes0:24:52
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6537162971975778
survival_time_median2.5999999999999988
deviation-center-line_median0.14884708769804722
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.16246684959956578
agent_compute-ego_mean0.1193457550406858
agent_compute-ego_median0.11584102023731578
agent_compute-ego_min0.0816840070944566
deviation-center-line_max0.5141120095458062
deviation-center-line_mean0.1769000227005324
deviation-center-line_min0.06216029860587694
deviation-heading_max1.9929437250081896
deviation-heading_mean0.692977823395608
deviation-heading_median0.4928440244444201
deviation-heading_min0.24234723632033853
driven_any_max3.5343244022418343
driven_any_mean1.3470385992234175
driven_any_median1.001633421251194
driven_any_min0.3412551960104244
driven_lanedir_consec_max2.494308053441959
driven_lanedir_consec_mean0.881391085527715
driven_lanedir_consec_min0.3199180625488627
driven_lanedir_max2.494308053441959
driven_lanedir_mean0.8962528018383226
driven_lanedir_median0.7206363324756615
driven_lanedir_min0.3199180625488627
in-drivable-lane_max3.549999999999991
in-drivable-lane_mean0.839999999999998
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.24855565119393264, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.06925922104074986, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.155336527884761, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.10752986352654952, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.04359757749340202, "sim_compute_performance-ego": 0.07697745214534711, "sim_compute_robot_state-ego": 0.08291390274144426, "sim_compute_robot_state-npc0": 0.07685048670708379, "sim_compute_robot_state-npc1": 0.07677476617354381, "sim_compute_robot_state-npc2": 0.07438194600841667, "sim_compute_robot_state-npc3": 0.07647331455085851}, "udem1-1-0": {"driven_any": 0.47798123795353353, "sim_physics": 0.2497809018407549, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.07189416885375977, "in-drivable-lane": 0, "agent_compute-ego": 0.16246684959956578, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.10845349516187396, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.04594646181379046, "sim_compute_performance-ego": 0.07773692267281669, "sim_compute_robot_state-ego": 0.0905194708279201, "sim_compute_robot_state-npc0": 0.07805607148579188, "sim_compute_robot_state-npc1": 0.08239365475518363, "sim_compute_robot_state-npc2": 0.07980709416525704, "sim_compute_robot_state-npc3": 0.07643545525414604}, "udem1-2-0": {"driven_any": 2.1864003300000694, "sim_physics": 0.18134581926956916, "survival_time": 5.14999999999999, "driven_lanedir": 1.826231945363564, "sim_render-ego": 0.05972729145901874, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.12698824891766297, "deviation-heading": 0.8115272458314897, "set_robot_commands": 0.0858080988948785, "deviation-center-line": 0.1927316041609401, "driven_lanedir_consec": 1.826231945363564, "sim_compute_sim_state": 0.03874928974410863, "sim_compute_performance-ego": 0.06571476204881391, "sim_compute_robot_state-ego": 0.06663452305840058, "sim_compute_robot_state-npc0": 0.0656357228177265, "sim_compute_robot_state-npc1": 0.06356838605936291, "sim_compute_robot_state-npc2": 0.06600616742106317, "sim_compute_robot_state-npc3": 0.06525416744565501}, "udem1-3-0": {"driven_any": 0.9410513078477724, "sim_physics": 0.2482065757115682, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.06695430974165599, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.14053413271903992, "deviation-heading": 0.35154027506548424, "set_robot_commands": 0.10042938093344372, "deviation-center-line": 0.08975118546699407, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.04281287888685862, "sim_compute_performance-ego": 0.0750250518321991, "sim_compute_robot_state-ego": 0.08010945717493693, "sim_compute_robot_state-npc0": 0.07606008152167003, "sim_compute_robot_state-npc1": 0.08567437529563904, "sim_compute_robot_state-npc2": 0.08693686624368031, "sim_compute_robot_state-npc3": 0.07538183530171712}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.2506513936179025, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.0698632626306443, "in-drivable-lane": 0, "agent_compute-ego": 0.15233826637268066, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.10605581601460774, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.04399532363528297, "sim_compute_performance-ego": 0.07691823868524461, "sim_compute_robot_state-ego": 0.08187186150323778, "sim_compute_robot_state-npc0": 0.07834412938072569, "sim_compute_robot_state-npc1": 0.0800840173448835, "sim_compute_robot_state-npc2": 0.07827677045549665, "sim_compute_robot_state-npc3": 0.07187821751549131}, "udem1-5-0": {"driven_any": 3.5343244022418343, "sim_physics": 0.1636652675217497, "survival_time": 8.349999999999984, "driven_lanedir": 2.494308053441959, "sim_render-ego": 0.05409811642355548, "in-drivable-lane": 2, "agent_compute-ego": 0.11723034396143016, "deviation-heading": 1.9929437250081896, "set_robot_commands": 0.08138465310285191, "deviation-center-line": 0.5141120095458062, "driven_lanedir_consec": 2.494308053441959, "sim_compute_sim_state": 0.03422128511760049, "sim_compute_performance-ego": 0.0591054776471532, "sim_compute_robot_state-ego": 0.061902017650490034, "sim_compute_robot_state-npc0": 0.06106009169253047, "sim_compute_robot_state-npc1": 0.0603737174393888, "sim_compute_robot_state-npc2": 0.05882169386583888, "sim_compute_robot_state-npc3": 0.05787203126324865}, "udem1-6-0": {"driven_any": 2.176040550989443, "sim_physics": 0.17167812643699276, "survival_time": 5.14999999999999, "driven_lanedir": 1.9748003483573369, "sim_render-ego": 0.05149866076349055, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.11002605864145223, "deviation-heading": 0.7877172510198454, "set_robot_commands": 0.07634600389350965, "deviation-center-line": 0.30661477465861414, "driven_lanedir_consec": 1.9748003483573369, "sim_compute_sim_state": 0.03224121019678208, "sim_compute_performance-ego": 0.056757017246727805, "sim_compute_robot_state-ego": 0.05930232770234636, "sim_compute_robot_state-npc0": 0.0569587749184914, "sim_compute_robot_state-npc1": 0.0559333157770842, "sim_compute_robot_state-npc2": 0.055721366289749885, "sim_compute_robot_state-npc3": 0.05531118679972528}, "udem1-7-0": {"driven_any": 1.0050236108346422, "sim_physics": 0.17299080848693849, "survival_time": 2.499999999999999, "driven_lanedir": 0.8701796426684156, "sim_render-ego": 0.051539006233215334, "in-drivable-lane": 0, "agent_compute-ego": 0.1100844383239746, "deviation-heading": 0.9227446658975216, "set_robot_commands": 0.07611077308654784, "deviation-center-line": 0.14884708769804722, "driven_lanedir_consec": 0.8546116490381102, "sim_compute_sim_state": 0.031402487754821774, "sim_compute_performance-ego": 0.05753192901611328, "sim_compute_robot_state-ego": 0.06577604293823242, "sim_compute_robot_state-npc0": 0.05519410610198974, "sim_compute_robot_state-npc1": 0.05492759227752685, "sim_compute_robot_state-npc2": 0.05492892742156982, "sim_compute_robot_state-npc3": 0.055024819374084474}, "udem1-8-0": {"driven_any": 0.9970416603019576, "sim_physics": 0.1528076850450956, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8041152574540316, "sim_render-ego": 0.05063549830363347, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.1097752176798307, "deviation-heading": 0.6817155913229235, "set_robot_commands": 0.07505703889406644, "deviation-center-line": 0.1898768080806288, "driven_lanedir_consec": 0.8041152574540316, "sim_compute_sim_state": 0.03255288876019991, "sim_compute_performance-ego": 0.05739063024520874, "sim_compute_robot_state-ego": 0.0626486402291518, "sim_compute_robot_state-npc0": 0.05478745240431566, "sim_compute_robot_state-npc1": 0.05652853617301354, "sim_compute_robot_state-npc2": 0.055612646616422214, "sim_compute_robot_state-npc3": 0.05676229641987728}, "udem1-9-0": {"driven_any": 0.4374998710472299, "sim_physics": 0.1954207053551307, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4177527119659043, "sim_render-ego": 0.051992443891671986, "in-drivable-lane": 0, "agent_compute-ego": 0.1213149199118981, "deviation-heading": 0.3796137446285097, "set_robot_commands": 0.07735528395726131, "deviation-center-line": 0.08207813392324871, "driven_lanedir_consec": 0.4177527119659043, "sim_compute_sim_state": 0.0335764243052556, "sim_compute_performance-ego": 0.057682459171001725, "sim_compute_robot_state-ego": 0.06005234901721661, "sim_compute_robot_state-npc0": 0.05679120467259334, "sim_compute_robot_state-npc1": 0.059519327603853665, "sim_compute_robot_state-npc2": 0.056780035679156966, "sim_compute_robot_state-npc3": 0.05792041925283579}, "udem1-10-0": {"driven_any": 0.3412551960104244, "sim_physics": 0.17329116301103073, "survival_time": 1.1000000000000003, "driven_lanedir": 0.3199180625488627, "sim_render-ego": 0.051693081855773926, "in-drivable-lane": 0, "agent_compute-ego": 0.11584102023731578, "deviation-heading": 0.3796986873122644, "set_robot_commands": 0.07827983119271019, "deviation-center-line": 0.06276575300040536, "driven_lanedir_consec": 0.3199180625488627, "sim_compute_sim_state": 0.03320745988325639, "sim_compute_performance-ego": 0.06003393910147927, "sim_compute_robot_state-ego": 0.06028139591217041, "sim_compute_robot_state-npc0": 0.05778358199379661, "sim_compute_robot_state-npc1": 0.06072930856184526, "sim_compute_robot_state-npc2": 0.056870070370760833, "sim_compute_robot_state-npc3": 0.056995695287531074}, "udem1-11-0": {"driven_any": 0.9951985237696338, "sim_physics": 0.14803444862365722, "survival_time": 2.499999999999999, "driven_lanedir": 0.7206363324756615, "sim_render-ego": 0.05113977909088135, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.10650689601898192, "deviation-heading": 0.30074636349085415, "set_robot_commands": 0.07351184844970703, "deviation-center-line": 0.16567670185555422, "driven_lanedir_consec": 0.7185075497455284, "sim_compute_sim_state": 0.03309750556945801, "sim_compute_performance-ego": 0.05565319061279297, "sim_compute_robot_state-ego": 0.059306068420410155, "sim_compute_robot_state-npc0": 0.05434576988220215, "sim_compute_robot_state-npc1": 0.05825599670410156, "sim_compute_robot_state-npc2": 0.05735128402709961, "sim_compute_robot_state-npc3": 0.05626834392547608}, "udem1-12-0": {"driven_any": 1.964687870253437, "sim_physics": 0.13596351603244214, "survival_time": 4.699999999999991, "driven_lanedir": 0.35162821871931205, "sim_render-ego": 0.044710098428929104, "in-drivable-lane": 3.549999999999991, "agent_compute-ego": 0.09815599056000406, "deviation-heading": 0.24234723632033853, "set_robot_commands": 0.06760663428205124, "deviation-center-line": 0.13126670634798973, "driven_lanedir_consec": 0.35162821871931205, "sim_compute_sim_state": 0.02883693258813087, "sim_compute_performance-ego": 0.04965711147227186, "sim_compute_robot_state-ego": 0.050647598631838535, "sim_compute_robot_state-npc0": 0.049829566732366035, "sim_compute_robot_state-npc1": 0.04851810982886781, "sim_compute_robot_state-npc2": 0.04774807361846275, "sim_compute_robot_state-npc3": 0.04942530013145285}, "udem1-13-0": {"driven_any": 1.001633421251194, "sim_physics": 0.10993269773629996, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6537162971975778, "sim_render-ego": 0.03860729015790499, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.0816840070944566, "deviation-heading": 0.4928440244444201, "set_robot_commands": 0.057307926508096546, "deviation-center-line": 0.13361893417935136, "driven_lanedir_consec": 0.6537162971975778, "sim_compute_sim_state": 0.024158060550689697, "sim_compute_performance-ego": 0.04162236818900475, "sim_compute_robot_state-ego": 0.043001807652986966, "sim_compute_robot_state-npc0": 0.0423891544342041, "sim_compute_robot_state-npc1": 0.04213312039008507, "sim_compute_robot_state-npc2": 0.04177883038154015, "sim_compute_robot_state-npc3": 0.04189970860114464}, "udem1-14-0": {"driven_any": 2.218769782727203, "sim_physics": 0.1073713189079648, "survival_time": 5.249999999999989, "driven_lanedir": 0.8125963661433656, "sim_render-ego": 0.03809627578372047, "in-drivable-lane": 2.499999999999993, "agent_compute-ego": 0.0819034076872326, "deviation-heading": 1.7163026916317128, "set_robot_commands": 0.05665628569466727, "deviation-center-line": 0.24126933160503777, "driven_lanedir_consec": 0.6077880672015845, "sim_compute_sim_state": 0.02411738123212542, "sim_compute_performance-ego": 0.04171615101042248, "sim_compute_robot_state-ego": 0.04335651851835705, "sim_compute_robot_state-npc0": 0.042714623042515346, "sim_compute_robot_state-npc1": 0.04189298493521554, "sim_compute_robot_state-npc2": 0.04327527227855864, "sim_compute_robot_state-npc3": 0.04199775060017904}}
set_robot_commands_max0.10845349516187396
set_robot_commands_mean0.08185952890618818
set_robot_commands_median0.07735528395726131
set_robot_commands_min0.05665628569466727
sim_compute_performance-ego_max0.07773692267281669
sim_compute_performance-ego_mean0.06063484673977316
sim_compute_performance-ego_median0.057682459171001725
sim_compute_performance-ego_min0.04162236818900475
sim_compute_robot_state-ego_max0.0905194708279201
sim_compute_robot_state-ego_mean0.06455493213194267
sim_compute_robot_state-ego_median0.061902017650490034
sim_compute_robot_state-ego_min0.043001807652986966
sim_compute_robot_state-npc0_max0.07834412938072569
sim_compute_robot_state-npc0_mean0.06045338785253353
sim_compute_robot_state-npc0_median0.0569587749184914
sim_compute_robot_state-npc0_min0.0423891544342041
sim_compute_robot_state-npc1_max0.08567437529563904
sim_compute_robot_state-npc1_mean0.06182048062130636
sim_compute_robot_state-npc1_median0.059519327603853665
sim_compute_robot_state-npc1_min0.04189298493521554
sim_compute_robot_state-npc2_max0.08693686624368031
sim_compute_robot_state-npc2_mean0.06095313632287159
sim_compute_robot_state-npc2_median0.056870070370760833
sim_compute_robot_state-npc2_min0.04177883038154015
sim_compute_robot_state-npc3_max0.07647331455085851
sim_compute_robot_state-npc3_mean0.059660036114894885
sim_compute_robot_state-npc3_median0.056995695287531074
sim_compute_robot_state-npc3_min0.04189970860114464
sim_compute_sim_state_max0.04594646181379046
sim_compute_sim_state_mean0.03483421116878419
sim_compute_sim_state_median0.03320745988325639
sim_compute_sim_state_min0.02411738123212542
sim_physics_max0.2506513936179025
sim_physics_mean0.1806464052527353
sim_physics_median0.17299080848693849
sim_physics_min0.1073713189079648
sim_render-ego_max0.07189416885375977
sim_render-ego_mean0.054780566977240354
sim_render-ego_median0.051693081855773926
sim_render-ego_min0.03809627578372047
simulation-passed1
survival_time_max8.349999999999984
survival_time_mean3.3333333333333286
survival_time_min1.0500000000000005
No reset possible
21771step1-simulationsuccessno0:24:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21359step1-simulationsuccessno0:32:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20626step1-simulationsuccessno0:08:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20032step1-simulationsuccessno0:06:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19672step1-simulationsuccessno0:07:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18929step1-simulationsuccessno0:11:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18924step1-simulationhost-errorno0:15:04
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job18924-808550', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job18924-808550', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job18924-808550', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18555step1-simulationsuccessno0:11:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17920step1-simulationsuccessno0:05:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17582step1-simulationsuccessno0:07:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17555step1-simulationabortedno0:05:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17536step1-simulationsuccessno0:07:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible