AI Driving Olympics Home Challenges Submissions Jobs

Submission 2865

Submission2865
Competingyes
Challengeaido2-LFVI-sim-validation
UserAlexKaravaev
Date submitted
Completecomplete
Resulterror
Jobsstep1-simulation: 22235
Next
User labelchallenge-aido_LF-baseline-duckietown
Admin priority60
User priority60

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
222352865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationerroryes-0:09:07
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
(hidden)
208182865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationsuccessno-0:28:43(hidden)
driven_lanedir_consec_median0.4906378651204748
survival_time_median14.950000000000076
deviation-center-line_median0.23916579592115045
in-drivable-lane_median3.4000000000000483


other stats
agent_compute-ego_max0.11445613543192544
agent_compute-ego_mean0.10948192346225652
agent_compute-ego_median0.10965794801712037
agent_compute-ego_min0.1025368468514804
deviation-center-line_max0.8598179747822606
deviation-center-line_mean0.4164109816235886
deviation-center-line_min0.1524104407527288
deviation-heading_max5.536030364590776
deviation-heading_mean3.1885992625542134
deviation-heading_median1.9089265344806616
deviation-heading_min1.448341673463684
driven_any_max1.1745667911136657
driven_any_mean0.7993065462620965
driven_any_median1.1745375875148232
driven_any_min0.20621070958903517
driven_lanedir_consec_max0.64427541241931
driven_lanedir_consec_mean0.4197719404475316
driven_lanedir_consec_min0.1532115720161615
driven_lanedir_max0.7380366950743553
driven_lanedir_mean0.4433955772366987
driven_lanedir_median0.4906378651204748
driven_lanedir_min0.1532115720161615
in-drivable-lane_max7.950000000000094
in-drivable-lane_mean3.260000000000036
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 0.2666651060745704, "sim_physics": 0.22426826567263217, "survival_time": 3.699999999999995, "driven_lanedir": 0.1532115720161615, "sim_render-ego": 0.06775473581778037, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.11164505417282516, "deviation-heading": 1.5314631276581263, "set_robot_commands": 0.1118812335504068, "deviation-center-line": 0.1524104407527288, "driven_lanedir_consec": 0.1532115720161615, "sim_compute_sim_state": 0.04338183274140229, "sim_compute_performance-ego": 0.07434112961227829, "sim_compute_robot_state-ego": 0.08290572746379955, "sim_compute_robot_state-npc0": 0.07458131377761429, "sim_compute_robot_state-npc1": 0.07648387148573592, "sim_compute_robot_state-npc2": 0.0750597264315631, "sim_compute_robot_state-npc3": 0.07875589744464771}, "udem1-1-0": {"driven_any": 1.1745375875148232, "sim_physics": 0.2307570195198059, "survival_time": 14.950000000000076, "driven_lanedir": 0.4906378651204748, "sim_render-ego": 0.06981953859329224, "in-drivable-lane": 7.950000000000094, "agent_compute-ego": 0.11445613543192544, "deviation-heading": 1.9089265344806616, "set_robot_commands": 0.10993390719095866, "deviation-center-line": 0.23916579592115045, "driven_lanedir_consec": 0.4906378651204748, "sim_compute_sim_state": 0.04418567816416422, "sim_compute_performance-ego": 0.08068432966868083, "sim_compute_robot_state-ego": 0.08500124375025431, "sim_compute_robot_state-npc0": 0.0760547129313151, "sim_compute_robot_state-npc1": 0.07591456254323324, "sim_compute_robot_state-npc2": 0.07792675813039143, "sim_compute_robot_state-npc3": 0.07701671044031778}, "udem1-2-0": {"driven_any": 1.174552537018389, "sim_physics": 0.21528377056121825, "survival_time": 14.950000000000076, "driven_lanedir": 0.6669058075208681, "sim_render-ego": 0.06567161401112874, "in-drivable-lane": 4.1500000000000385, "agent_compute-ego": 0.10911363283793132, "deviation-heading": 5.536030364590776, "set_robot_commands": 0.1030753747622172, "deviation-center-line": 0.6709099403033454, "driven_lanedir_consec": 0.642548906230078, "sim_compute_sim_state": 0.04081465562184652, "sim_compute_performance-ego": 0.07333365122477213, "sim_compute_robot_state-ego": 0.07943152507146199, "sim_compute_robot_state-npc0": 0.07270699580510458, "sim_compute_robot_state-npc1": 0.0711501677831014, "sim_compute_robot_state-npc2": 0.07263823350270589, "sim_compute_robot_state-npc3": 0.07194299538930257}, "udem1-3-0": {"driven_any": 1.1745667911136657, "sim_physics": 0.20135415156682332, "survival_time": 14.950000000000076, "driven_lanedir": 0.7380366950743553, "sim_render-ego": 0.06528722763061523, "in-drivable-lane": 3.4000000000000483, "agent_compute-ego": 0.10965794801712037, "deviation-heading": 5.518234612577819, "set_robot_commands": 0.10125334898630778, "deviation-center-line": 0.8598179747822606, "driven_lanedir_consec": 0.64427541241931, "sim_compute_sim_state": 0.039270044167836506, "sim_compute_performance-ego": 0.07115239938100179, "sim_compute_robot_state-ego": 0.07627529621124268, "sim_compute_robot_state-npc0": 0.07111894845962524, "sim_compute_robot_state-npc1": 0.06980561176935832, "sim_compute_robot_state-npc2": 0.07071214437484741, "sim_compute_robot_state-npc3": 0.07056262095769246}, "udem1-4-0": {"driven_any": 0.20621070958903517, "sim_physics": 0.20105756562331628, "survival_time": 2.8999999999999977, "driven_lanedir": 0.16818594645163398, "sim_render-ego": 0.06471192425694959, "in-drivable-lane": 0, "agent_compute-ego": 0.1025368468514804, "deviation-heading": 1.448341673463684, "set_robot_commands": 0.0960175949951698, "deviation-center-line": 0.15975075635845737, "driven_lanedir_consec": 0.16818594645163398, "sim_compute_sim_state": 0.03916913065417059, "sim_compute_performance-ego": 0.06909774911814723, "sim_compute_robot_state-ego": 0.07507125262556405, "sim_compute_robot_state-npc0": 0.07022135422147553, "sim_compute_robot_state-npc1": 0.07067679536753688, "sim_compute_robot_state-npc2": 0.06912464520026898, "sim_compute_robot_state-npc3": 0.07039469686047785}}
set_robot_commands_max0.1118812335504068
set_robot_commands_mean0.10443229189701204
set_robot_commands_median0.1030753747622172
set_robot_commands_min0.0960175949951698
sim_compute_performance-ego_max0.08068432966868083
sim_compute_performance-ego_mean0.07372185180097605
sim_compute_performance-ego_median0.07333365122477213
sim_compute_performance-ego_min0.06909774911814723
sim_compute_robot_state-ego_max0.08500124375025431
sim_compute_robot_state-ego_mean0.07973700902446453
sim_compute_robot_state-ego_median0.07943152507146199
sim_compute_robot_state-ego_min0.07507125262556405
sim_compute_robot_state-npc0_max0.0760547129313151
sim_compute_robot_state-npc0_mean0.07293666503902695
sim_compute_robot_state-npc0_median0.07270699580510458
sim_compute_robot_state-npc0_min0.07022135422147553
sim_compute_robot_state-npc1_max0.07648387148573592
sim_compute_robot_state-npc1_mean0.07280620178979316
sim_compute_robot_state-npc1_median0.0711501677831014
sim_compute_robot_state-npc1_min0.06980561176935832
sim_compute_robot_state-npc2_max0.07792675813039143
sim_compute_robot_state-npc2_mean0.07309230152795536
sim_compute_robot_state-npc2_median0.07263823350270589
sim_compute_robot_state-npc2_min0.06912464520026898
sim_compute_robot_state-npc3_max0.07875589744464771
sim_compute_robot_state-npc3_mean0.07373458421848768
sim_compute_robot_state-npc3_median0.07194299538930257
sim_compute_robot_state-npc3_min0.07039469686047785
sim_compute_sim_state_max0.04418567816416422
sim_compute_sim_state_mean0.04136426826988403
sim_compute_sim_state_median0.04081465562184652
sim_compute_sim_state_min0.03916913065417059
sim_physics_max0.2307570195198059
sim_physics_mean0.2145441545887592
sim_physics_median0.21528377056121825
sim_physics_min0.20105756562331628
sim_render-ego_max0.06981953859329224
sim_render-ego_mean0.06664900806195324
sim_render-ego_median0.06567161401112874
sim_render-ego_min0.06471192425694959
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean10.290000000000044
survival_time_min2.8999999999999977
201732865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationsuccessno-0:25:12(hidden)
driven_lanedir_consec_median0.42092024400740513
survival_time_median6.199999999999986
deviation-center-line_median0.39887631758881914
in-drivable-lane_median0.7499999999999973


other stats
agent_compute-ego_max0.12848677558283653
agent_compute-ego_mean0.10449558528299778
agent_compute-ego_median0.1003039589634648
agent_compute-ego_min0.09175318962818868
deviation-center-line_max0.6814184811135475
deviation-center-line_mean0.39868467575164573
deviation-center-line_min0.14808359038981317
deviation-heading_max5.682032472228424
deviation-heading_mean3.0991916846309637
deviation-heading_median2.196694509997698
deviation-heading_min1.4831008157431294
driven_any_max1.1745078055093283
driven_any_mean0.6555289219567875
driven_any_median0.4713653277787152
driven_any_min0.1900877791161716
driven_lanedir_consec_max0.6296003873719687
driven_lanedir_consec_mean0.3870279121704267
driven_lanedir_consec_min0.1476423557944071
driven_lanedir_max0.8653004836600253
driven_lanedir_mean0.4573524156901631
driven_lanedir_median0.42092024400740513
driven_lanedir_min0.1476423557944071
in-drivable-lane_max4.150000000000059
in-drivable-lane_mean1.4000000000000168
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 0.26723155429561457, "sim_physics": 0.1482917875856967, "survival_time": 3.699999999999995, "driven_lanedir": 0.15577862660395891, "sim_render-ego": 0.05678098910563701, "in-drivable-lane": 0.7499999999999973, "agent_compute-ego": 0.09175318962818868, "deviation-heading": 1.5475188810563432, "set_robot_commands": 0.08243754425564327, "deviation-center-line": 0.15955196419172768, "driven_lanedir_consec": 0.15577862660395891, "sim_compute_sim_state": 0.034379649806667016, "sim_compute_performance-ego": 0.06056144753017941, "sim_compute_robot_state-ego": 0.06450259041141819, "sim_compute_robot_state-npc0": 0.060341035997545395, "sim_compute_robot_state-npc1": 0.06002761866595294, "sim_compute_robot_state-npc2": 0.05916655708003689, "sim_compute_robot_state-npc3": 0.060257492838679136}, "udem1-1-0": {"driven_any": 0.4713653277787152, "sim_physics": 0.1531128921816426, "survival_time": 6.199999999999986, "driven_lanedir": 0.42092024400740513, "sim_render-ego": 0.05556020236784412, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.12848677558283653, "deviation-heading": 2.196694509997698, "set_robot_commands": 0.09240400791168212, "deviation-center-line": 0.39887631758881914, "driven_lanedir_consec": 0.42092024400740513, "sim_compute_sim_state": 0.03503909803205921, "sim_compute_performance-ego": 0.059861367748629664, "sim_compute_robot_state-ego": 0.06515246245168871, "sim_compute_robot_state-npc0": 0.06554988891847673, "sim_compute_robot_state-npc1": 0.06384849163793749, "sim_compute_robot_state-npc2": 0.06162073343030868, "sim_compute_robot_state-npc3": 0.06205556469578897}, "udem1-2-0": {"driven_any": 1.1744521430841075, "sim_physics": 0.20364309708277384, "survival_time": 14.950000000000076, "driven_lanedir": 0.6971203683850193, "sim_render-ego": 0.06236762205759684, "in-drivable-lane": 4.150000000000059, "agent_compute-ego": 0.09743817726771035, "deviation-heading": 4.5866117441292245, "set_robot_commands": 0.09243290821711224, "deviation-center-line": 0.605493025474321, "driven_lanedir_consec": 0.6296003873719687, "sim_compute_sim_state": 0.037642247676849365, "sim_compute_performance-ego": 0.06497902154922486, "sim_compute_robot_state-ego": 0.0707712467511495, "sim_compute_robot_state-npc0": 0.0669618312517802, "sim_compute_robot_state-npc1": 0.0662138319015503, "sim_compute_robot_state-npc2": 0.06739532629648845, "sim_compute_robot_state-npc3": 0.06705923398335775}, "udem1-3-0": {"driven_any": 1.1745078055093283, "sim_physics": 0.21288268009821573, "survival_time": 14.950000000000076, "driven_lanedir": 0.8653004836600253, "sim_render-ego": 0.06408434947331747, "in-drivable-lane": 2.0000000000000284, "agent_compute-ego": 0.10449582497278848, "deviation-heading": 5.682032472228424, "set_robot_commands": 0.09722180604934692, "deviation-center-line": 0.6814184811135475, "driven_lanedir_consec": 0.581197947074394, "sim_compute_sim_state": 0.039928827285766605, "sim_compute_performance-ego": 0.0702626895904541, "sim_compute_robot_state-ego": 0.07384480158487956, "sim_compute_robot_state-npc0": 0.0709614634513855, "sim_compute_robot_state-npc1": 0.0706700332959493, "sim_compute_robot_state-npc2": 0.07016832033793131, "sim_compute_robot_state-npc3": 0.06969287633895874}, "udem1-4-0": {"driven_any": 0.1900877791161716, "sim_physics": 0.16485004071836118, "survival_time": 2.6999999999999984, "driven_lanedir": 0.1476423557944071, "sim_render-ego": 0.06168314704188594, "in-drivable-lane": 0, "agent_compute-ego": 0.1003039589634648, "deviation-heading": 1.4831008157431294, "set_robot_commands": 0.09520169099171956, "deviation-center-line": 0.14808359038981317, "driven_lanedir_consec": 0.1476423557944071, "sim_compute_sim_state": 0.04013618716487178, "sim_compute_performance-ego": 0.06676040755377875, "sim_compute_robot_state-ego": 0.06879414010930944, "sim_compute_robot_state-npc0": 0.07266768261238381, "sim_compute_robot_state-npc1": 0.06789219379425049, "sim_compute_robot_state-npc2": 0.0689603708408497, "sim_compute_robot_state-npc3": 0.06881527105967204}}
set_robot_commands_max0.09722180604934692
set_robot_commands_mean0.0919395914851008
set_robot_commands_median0.09243290821711224
set_robot_commands_min0.08243754425564327
sim_compute_performance-ego_max0.0702626895904541
sim_compute_performance-ego_mean0.06448498679445336
sim_compute_performance-ego_median0.06497902154922486
sim_compute_performance-ego_min0.059861367748629664
sim_compute_robot_state-ego_max0.07384480158487956
sim_compute_robot_state-ego_mean0.06861304826168908
sim_compute_robot_state-ego_median0.06879414010930944
sim_compute_robot_state-ego_min0.06450259041141819
sim_compute_robot_state-npc0_max0.07266768261238381
sim_compute_robot_state-npc0_mean0.06729638044631432
sim_compute_robot_state-npc0_median0.0669618312517802
sim_compute_robot_state-npc0_min0.060341035997545395
sim_compute_robot_state-npc1_max0.0706700332959493
sim_compute_robot_state-npc1_mean0.0657304338591281
sim_compute_robot_state-npc1_median0.0662138319015503
sim_compute_robot_state-npc1_min0.06002761866595294
sim_compute_robot_state-npc2_max0.07016832033793131
sim_compute_robot_state-npc2_mean0.06546226159712301
sim_compute_robot_state-npc2_median0.06739532629648845
sim_compute_robot_state-npc2_min0.05916655708003689
sim_compute_robot_state-npc3_max0.06969287633895874
sim_compute_robot_state-npc3_mean0.06557608778329133
sim_compute_robot_state-npc3_median0.06705923398335775
sim_compute_robot_state-npc3_min0.060257492838679136
sim_compute_sim_state_max0.04013618716487178
sim_compute_sim_state_mean0.0374252019932428
sim_compute_sim_state_median0.037642247676849365
sim_compute_sim_state_min0.034379649806667016
sim_physics_max0.21288268009821573
sim_physics_mean0.176556099533338
sim_physics_median0.16485004071836118
sim_physics_min0.1482917875856967
sim_render-ego_max0.06408434947331747
sim_render-ego_mean0.06009526200925628
sim_render-ego_median0.06168314704188594
sim_render-ego_min0.05556020236784412
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean8.500000000000027
survival_time_min2.6999999999999984
201592865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationhost-errorno-0:07:12
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job20159-239117', 'ps', '-q', 'solution']' returned non-zero exit status 255.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job20159-239117', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job20159-239117', 'ps', '-q', 'solution']' returned non-zero exit status 255.

stdout | 

stderr | 
(hidden)
200892865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationsuccessno-0:16:17(hidden)
driven_lanedir_consec_median0.46565017005301224
survival_time_median7.449999999999981
deviation-center-line_median0.2882218494101193
in-drivable-lane_median0.7999999999999972


other stats
agent_compute-ego_max0.09799424558877944
agent_compute-ego_mean0.08887003378280664
agent_compute-ego_median0.08672890087102084
agent_compute-ego_min0.08428518851598103
deviation-center-line_max0.537077720055675
deviation-center-line_mean0.29272662848603404
deviation-center-line_min0.1215526816296228
deviation-heading_max4.157728977389364
deviation-heading_mean2.454983461450658
deviation-heading_median2.14963671161184
deviation-heading_min1.4717839531985006
driven_any_max1.1767794278679349
driven_any_mean0.7149927785762101
driven_any_median0.5740810466368345
driven_any_min0.22999383000584195
driven_lanedir_consec_max0.6351395165131395
driven_lanedir_consec_mean0.4178435033021303
driven_lanedir_consec_min0.11584373405072056
driven_lanedir_max0.9089832756234724
driven_lanedir_mean0.47261225512419697
driven_lanedir_median0.46565017005301224
driven_lanedir_min0.11584373405072056
in-drivable-lane_max7.9000000000000945
in-drivable-lane_mean2.2500000000000213
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 0.22999383000584195, "sim_physics": 0.04745914787054062, "survival_time": 3.1999999999999966, "driven_lanedir": 0.11584373405072056, "sim_render-ego": 0.05693260207772255, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.09799424558877944, "deviation-heading": 1.4717839531985006, "set_robot_commands": 0.0786939263343811, "deviation-center-line": 0.1215526816296228, "driven_lanedir_consec": 0.11584373405072056, "sim_compute_sim_state": 0.03260214999318123, "sim_compute_performance-ego": 0.06253881752490997, "sim_compute_robot_state-ego": 0.0637086071074009, "sim_compute_robot_state-npc0": 0.06426065042614937, "sim_compute_robot_state-npc1": 0.060049597173929214, "sim_compute_robot_state-npc2": 0.05903516337275505, "sim_compute_robot_state-npc3": 0.05914441496133804}, "udem1-1-0": {"driven_any": 1.17674843821732, "sim_physics": 0.04425849437713623, "survival_time": 14.950000000000076, "driven_lanedir": 0.46565017005301224, "sim_render-ego": 0.05763531525929769, "in-drivable-lane": 7.9000000000000945, "agent_compute-ego": 0.08986928701400757, "deviation-heading": 2.617330634817127, "set_robot_commands": 0.07611050367355347, "deviation-center-line": 0.30045600046214327, "driven_lanedir_consec": 0.46565017005301224, "sim_compute_sim_state": 0.03379124561945597, "sim_compute_performance-ego": 0.061678139368693034, "sim_compute_robot_state-ego": 0.062055567105611165, "sim_compute_robot_state-npc0": 0.06609498023986816, "sim_compute_robot_state-npc1": 0.06181816101074219, "sim_compute_robot_state-npc2": 0.06169082244237264, "sim_compute_robot_state-npc3": 0.06078333536783854}, "udem1-2-0": {"driven_any": 1.1767794278679349, "sim_physics": 0.047953482468922934, "survival_time": 14.950000000000076, "driven_lanedir": 0.9089832756234724, "sim_render-ego": 0.05755265235900879, "in-drivable-lane": 2.2000000000000153, "agent_compute-ego": 0.08428518851598103, "deviation-heading": 4.157728977389364, "set_robot_commands": 0.07373157501220703, "deviation-center-line": 0.537077720055675, "driven_lanedir_consec": 0.6351395165131395, "sim_compute_sim_state": 0.03545847177505493, "sim_compute_performance-ego": 0.060297420024871824, "sim_compute_robot_state-ego": 0.0608399232228597, "sim_compute_robot_state-npc0": 0.06523253917694091, "sim_compute_robot_state-npc1": 0.06196925163269043, "sim_compute_robot_state-npc2": 0.06151816765467326, "sim_compute_robot_state-npc3": 0.06133298635482788}, "udem1-3-0": {"driven_any": 0.5740810466368345, "sim_physics": 0.046321958503467126, "survival_time": 7.449999999999981, "driven_lanedir": 0.5009109077731269, "sim_render-ego": 0.05696725365299506, "in-drivable-lane": 0.34999999999999876, "agent_compute-ego": 0.08672890087102084, "deviation-heading": 2.14963671161184, "set_robot_commands": 0.07317808650484021, "deviation-center-line": 0.2882218494101193, "driven_lanedir_consec": 0.5009109077731269, "sim_compute_sim_state": 0.03310026898480102, "sim_compute_performance-ego": 0.060359919631241155, "sim_compute_robot_state-ego": 0.05962708492407063, "sim_compute_robot_state-npc0": 0.0634948631261019, "sim_compute_robot_state-npc1": 0.06173218496693861, "sim_compute_robot_state-npc2": 0.0597441276447885, "sim_compute_robot_state-npc3": 0.059402296207095154}, "udem1-4-0": {"driven_any": 0.4173611501531193, "sim_physics": 0.04459647698835893, "survival_time": 5.4999999999999885, "driven_lanedir": 0.3716731881206527, "sim_render-ego": 0.05518029169602827, "in-drivable-lane": 0, "agent_compute-ego": 0.08547254692424427, "deviation-heading": 1.8784370302364597, "set_robot_commands": 0.07332774292338978, "deviation-center-line": 0.21632489087260984, "driven_lanedir_consec": 0.3716731881206527, "sim_compute_sim_state": 0.033261498537930576, "sim_compute_performance-ego": 0.06069793701171875, "sim_compute_robot_state-ego": 0.061456487395546655, "sim_compute_robot_state-npc0": 0.06355471827767112, "sim_compute_robot_state-npc1": 0.05967957323247736, "sim_compute_robot_state-npc2": 0.05815083113583651, "sim_compute_robot_state-npc3": 0.0593770525672219}}
set_robot_commands_max0.0786939263343811
set_robot_commands_mean0.07500836688967433
set_robot_commands_median0.07373157501220703
set_robot_commands_min0.07317808650484021
sim_compute_performance-ego_max0.06253881752490997
sim_compute_performance-ego_mean0.061114446712286954
sim_compute_performance-ego_median0.06069793701171875
sim_compute_performance-ego_min0.060297420024871824
sim_compute_robot_state-ego_max0.0637086071074009
sim_compute_robot_state-ego_mean0.0615375339510978
sim_compute_robot_state-ego_median0.061456487395546655
sim_compute_robot_state-ego_min0.05962708492407063
sim_compute_robot_state-npc0_max0.06609498023986816
sim_compute_robot_state-npc0_mean0.06452755024934628
sim_compute_robot_state-npc0_median0.06426065042614937
sim_compute_robot_state-npc0_min0.0634948631261019
sim_compute_robot_state-npc1_max0.06196925163269043
sim_compute_robot_state-npc1_mean0.06104975360335556
sim_compute_robot_state-npc1_median0.06173218496693861
sim_compute_robot_state-npc1_min0.05967957323247736
sim_compute_robot_state-npc2_max0.06169082244237264
sim_compute_robot_state-npc2_mean0.060027822450085186
sim_compute_robot_state-npc2_median0.0597441276447885
sim_compute_robot_state-npc2_min0.05815083113583651
sim_compute_robot_state-npc3_max0.06133298635482788
sim_compute_robot_state-npc3_mean0.0600080170916643
sim_compute_robot_state-npc3_median0.059402296207095154
sim_compute_robot_state-npc3_min0.05914441496133804
sim_compute_sim_state_max0.03545847177505493
sim_compute_sim_state_mean0.033642726982084745
sim_compute_sim_state_median0.033261498537930576
sim_compute_sim_state_min0.03260214999318123
sim_physics_max0.047953482468922934
sim_physics_mean0.04611791204168516
sim_physics_median0.046321958503467126
sim_physics_min0.04425849437713623
sim_render-ego_max0.05763531525929769
sim_render-ego_mean0.056853623009010465
sim_render-ego_median0.05696725365299506
sim_render-ego_min0.05518029169602827
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean9.210000000000024
survival_time_min3.1999999999999966
192762865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationsuccessno-0:18:48(hidden)
driven_lanedir_consec_median0.4822454603534423
survival_time_median14.950000000000076
deviation-center-line_median0.20142330058984173
in-drivable-lane_median1.9500000000000275


other stats
agent_compute-ego_max0.09636053244272869
agent_compute-ego_mean0.08900605922516325
agent_compute-ego_median0.08741215253487611
agent_compute-ego_min0.08608087062835694
deviation-center-line_max0.7174437868070064
deviation-center-line_mean0.3440412661051905
deviation-center-line_min0.12727579259173313
deviation-heading_max5.998400968351701
deviation-heading_mean3.3238359902178267
deviation-heading_median2.395191670207236
deviation-heading_min1.5458517968651286
driven_any_max1.1778660440785529
driven_any_mean0.8385038621210625
driven_any_median1.177846078789077
driven_any_min0.2887073076450035
driven_lanedir_consec_max0.800097753503334
driven_lanedir_consec_mean0.48760543512440513
driven_lanedir_consec_min0.23357527563858135
driven_lanedir_max0.8747175051645386
driven_lanedir_mean0.5488077899449735
driven_lanedir_median0.4822454603534423
driven_lanedir_min0.23357527563858135
in-drivable-lane_max7.700000000000095
in-drivable-lane_mean2.3700000000000294
in-drivable-lane_min0.04999999999999982
per-episodes
details{"udem1-0-0": {"driven_any": 0.2887073076450035, "sim_physics": 0.032169308417882674, "survival_time": 3.899999999999994, "driven_lanedir": 0.23357527563858135, "sim_render-ego": 0.05410576783693754, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.08741215253487611, "deviation-heading": 1.5458517968651286, "set_robot_commands": 0.07315016709841214, "deviation-center-line": 0.12727579259173313, "driven_lanedir_consec": 0.23357527563858135, "sim_compute_sim_state": 0.034203385695433006, "sim_compute_performance-ego": 0.059111472887870595, "sim_compute_robot_state-ego": 0.06073657365945669, "sim_compute_robot_state-npc0": 0.061772438196035534, "sim_compute_robot_state-npc1": 0.06042895255944668, "sim_compute_robot_state-npc2": 0.06073366678678072, "sim_compute_robot_state-npc3": 0.060979522191561185}, "udem1-1-0": {"driven_any": 1.177846078789077, "sim_physics": 0.03147789080937703, "survival_time": 14.950000000000076, "driven_lanedir": 0.4822454603534423, "sim_render-ego": 0.05539740085601807, "in-drivable-lane": 7.700000000000095, "agent_compute-ego": 0.08608087062835694, "deviation-heading": 2.395191670207236, "set_robot_commands": 0.07369386514027913, "deviation-center-line": 0.20142330058984173, "driven_lanedir_consec": 0.4822454603534423, "sim_compute_sim_state": 0.03500017484029134, "sim_compute_performance-ego": 0.06087045590082804, "sim_compute_robot_state-ego": 0.06245906352996826, "sim_compute_robot_state-npc0": 0.06416738828023275, "sim_compute_robot_state-npc1": 0.06048127333323161, "sim_compute_robot_state-npc2": 0.06193707386652628, "sim_compute_robot_state-npc3": 0.061327155431111655}, "udem1-2-0": {"driven_any": 1.1778660440785529, "sim_physics": 0.029440244833628337, "survival_time": 14.950000000000076, "driven_lanedir": 0.8334043340588759, "sim_render-ego": 0.05333714644114176, "in-drivable-lane": 1.9500000000000275, "agent_compute-ego": 0.08879526376724244, "deviation-heading": 5.998400968351701, "set_robot_commands": 0.07113379955291749, "deviation-center-line": 0.7174437868070064, "driven_lanedir_consec": 0.800097753503334, "sim_compute_sim_state": 0.03321646372477213, "sim_compute_performance-ego": 0.05645381689071655, "sim_compute_robot_state-ego": 0.05846914768218994, "sim_compute_robot_state-npc0": 0.06099478721618652, "sim_compute_robot_state-npc1": 0.05836862961451213, "sim_compute_robot_state-npc2": 0.056878317991892496, "sim_compute_robot_state-npc3": 0.05738462607065837}, "udem1-3-0": {"driven_any": 1.1778635521834349, "sim_physics": 0.03069431702295939, "survival_time": 14.950000000000076, "driven_lanedir": 0.8747175051645386, "sim_render-ego": 0.05468050003051758, "in-drivable-lane": 2.0500000000000256, "agent_compute-ego": 0.09636053244272869, "deviation-heading": 5.107221609460501, "set_robot_commands": 0.07320452690124511, "deviation-center-line": 0.5398667108667882, "driven_lanedir_consec": 0.6020123116172389, "sim_compute_sim_state": 0.03364793062210083, "sim_compute_performance-ego": 0.05857104778289795, "sim_compute_robot_state-ego": 0.06092108647028605, "sim_compute_robot_state-npc0": 0.06201037883758545, "sim_compute_robot_state-npc1": 0.059653141498565675, "sim_compute_robot_state-npc2": 0.05961342255274455, "sim_compute_robot_state-npc3": 0.05938352743784587}, "udem1-4-0": {"driven_any": 0.3702363279092443, "sim_physics": 0.03251497599543357, "survival_time": 4.899999999999991, "driven_lanedir": 0.3200963745094292, "sim_render-ego": 0.055628761953237106, "in-drivable-lane": 0.04999999999999982, "agent_compute-ego": 0.08638147675261205, "deviation-heading": 1.5725139062045677, "set_robot_commands": 0.07353678528143436, "deviation-center-line": 0.13419673967058318, "driven_lanedir_consec": 0.3200963745094292, "sim_compute_sim_state": 0.03362737626445537, "sim_compute_performance-ego": 0.05948836219554045, "sim_compute_robot_state-ego": 0.06242233636427899, "sim_compute_robot_state-npc0": 0.06373729997751665, "sim_compute_robot_state-npc1": 0.06068739599111129, "sim_compute_robot_state-npc2": 0.060711629536687114, "sim_compute_robot_state-npc3": 0.05993295202449876}}
set_robot_commands_max0.07369386514027913
set_robot_commands_mean0.07294382879485764
set_robot_commands_median0.07320452690124511
set_robot_commands_min0.07113379955291749
sim_compute_performance-ego_max0.06087045590082804
sim_compute_performance-ego_mean0.05889903113157072
sim_compute_performance-ego_median0.059111472887870595
sim_compute_performance-ego_min0.05645381689071655
sim_compute_robot_state-ego_max0.06245906352996826
sim_compute_robot_state-ego_mean0.06100164154123598
sim_compute_robot_state-ego_median0.06092108647028605
sim_compute_robot_state-ego_min0.05846914768218994
sim_compute_robot_state-npc0_max0.06416738828023275
sim_compute_robot_state-npc0_mean0.06253645850151138
sim_compute_robot_state-npc0_median0.06201037883758545
sim_compute_robot_state-npc0_min0.06099478721618652
sim_compute_robot_state-npc1_max0.06068739599111129
sim_compute_robot_state-npc1_mean0.05992387859937348
sim_compute_robot_state-npc1_median0.06042895255944668
sim_compute_robot_state-npc1_min0.05836862961451213
sim_compute_robot_state-npc2_max0.06193707386652628
sim_compute_robot_state-npc2_mean0.05997482214692623
sim_compute_robot_state-npc2_median0.060711629536687114
sim_compute_robot_state-npc2_min0.056878317991892496
sim_compute_robot_state-npc3_max0.061327155431111655
sim_compute_robot_state-npc3_mean0.05980155663113517
sim_compute_robot_state-npc3_median0.05993295202449876
sim_compute_robot_state-npc3_min0.05738462607065837
sim_compute_sim_state_max0.03500017484029134
sim_compute_sim_state_mean0.033939066229410536
sim_compute_sim_state_median0.03364793062210083
sim_compute_sim_state_min0.03321646372477213
sim_physics_max0.03251497599543357
sim_physics_mean0.031259347415856197
sim_physics_median0.03147789080937703
sim_physics_min0.029440244833628337
sim_render-ego_max0.055628761953237106
sim_render-ego_mean0.05462991542357041
sim_render-ego_median0.05468050003051758
sim_render-ego_min0.05333714644114176
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean10.730000000000045
survival_time_min3.899999999999994
192742865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationhost-errorno-0:01:03
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 306, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown-shell/lib/dt_shell/env_checks.py", line 60, in check_docker_environment
    containers = client.containers.list(filters=dict(status='running'))
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 925, in list
    containers.append(self.get(r['Id']))
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 862, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/container.py", line 755, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 230, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 571, in run_single
    timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 646, in run_one
    client = check_docker_environment()
  File "/project/src/duckietown-shell/lib/dt_shell/env_checks.py", line 67, in check_docker_environment
    raise InvalidEnvironment(msg)
dt_shell.exceptions.InvalidEnvironment: I cannot communicate with Docker:
UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Make sure the docker service is running.
(hidden)
185892865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationerrorno-0:24:03
Waited 1201 seconds [...]
Waited 1201 seconds for container to finish. Giving up. 
(hidden)
185812865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationhost-errorno-0:02:44
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 481, in get_cr
    uploaded = upload_files(wd, aws_config)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1016, in upload_files
    uploaded = upload(aws_config, toupload)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1171, in upload
    sha256hex = compute_sha256hex(realfile)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1217, in compute_sha256hex
    res: bytes = subprocess.check_output(cmd)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['shasum', '-a', '256', '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-validation/submission2865/step1-simulation-idsc-rudolf-9560-job18581/tmp/tmpprwdqviv/attempts/udem1-0-0.attempt0/drawing.svg']' returned non-zero exit status 1.
(hidden)
185662865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationerrorno-0:21:16
Waited 1201 seconds [...]
Waited 1201 seconds for container to finish. Giving up. 
(hidden)
185652865AlexKaravaevchallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationerrorno-0:21:14
Waited 1201 seconds [...]
Waited 1201 seconds for container to finish. Giving up. 
(hidden)