Duckietown Challenges Home Challenges Submissions

Submission 5985

Submission5985
Competingyes
Challengeaido3-LFV-sim-testing
UserFrank (Chude) Qian 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 29850
Next
User label2019-12-09|FrankNet|v3-LFSIM
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
29850step1-simulationsuccessyes0:54:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.287550376498553
survival_time_median14.950000000000076
deviation-center-line_median0.9070332640735664
in-drivable-lane_median1.2499999999999956


other stats
agent_compute-ego_max0.07299140691757203
agent_compute-ego_mean0.06305844352859667
agent_compute-ego_median0.06695802927017212
agent_compute-ego_min0.04394786119461059
deviation-center-line_max1.1818040928264244
deviation-center-line_mean0.8025320740477856
deviation-center-line_min0.22307400914119593
deviation-heading_max3.8751441023170217
deviation-heading_mean2.7740756275258662
deviation-heading_median2.9844942525175813
deviation-heading_min0.9443903829884468
driven_any_max6.125731577826896
driven_any_mean4.524750469559961
driven_any_median4.760248971791701
driven_any_min1.339017857380343
driven_lanedir_consec_max5.6192201123069285
driven_lanedir_consec_mean3.797607825607813
driven_lanedir_consec_min1.286349121351606
driven_lanedir_max5.6192201123069285
driven_lanedir_mean3.797607825607813
driven_lanedir_median4.287550376498553
driven_lanedir_min1.286349121351606
in-drivable-lane_max5.150000000000024
in-drivable-lane_mean1.6033333333333402
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 4.3829683467615315, "sim_physics": 0.17919276158014932, "survival_time": 14.950000000000076, "driven_lanedir": 4.287550376498553, "sim_render-ego": 0.012411152521769206, "in-drivable-lane": 0, "agent_compute-ego": 0.06695802927017212, "deviation-heading": 2.9844942525175813, "set_robot_commands": 0.009347201188405357, "deviation-center-line": 0.9978607045804516, "driven_lanedir_consec": 4.287550376498553, "sim_compute_sim_state": 0.005350996653238932, "sim_compute_performance-ego": 0.007683248519897461, "sim_compute_robot_state-ego": 0.009663247267405192, "sim_compute_robot_state-npc0": 0.008871235052744548, "sim_compute_robot_state-npc1": 0.008764340877532958, "sim_compute_robot_state-npc2": 0.00893116315205892, "sim_compute_robot_state-npc3": 0.008816877206166585}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 5.118778827182721, "sim_physics": 0.21237647851308183, "survival_time": 14.950000000000076, "driven_lanedir": 4.507729473173159, "sim_render-ego": 0.01389183521270752, "in-drivable-lane": 1.2499999999999956, "agent_compute-ego": 0.07221120595932007, "deviation-heading": 3.028407390959487, "set_robot_commands": 0.010109410285949708, "deviation-center-line": 1.0467008400628492, "driven_lanedir_consec": 4.507729473173159, "sim_compute_sim_state": 0.006007920900980631, "sim_compute_performance-ego": 0.00823191245396932, "sim_compute_robot_state-ego": 0.01026759147644043, "sim_compute_robot_state-npc0": 0.009612011909484864, "sim_compute_robot_state-npc1": 0.009885197480519612, "sim_compute_robot_state-npc2": 0.00987558921178182, "sim_compute_robot_state-npc3": 0.009718742370605467}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 5.537547463407202, "sim_physics": 0.21510707934697468, "survival_time": 14.950000000000076, "driven_lanedir": 4.74667543844416, "sim_render-ego": 0.013165403207143147, "in-drivable-lane": 1.5000000000000098, "agent_compute-ego": 0.07229929288228353, "deviation-heading": 3.646786647996812, "set_robot_commands": 0.010191832383473714, "deviation-center-line": 1.1818040928264244, "driven_lanedir_consec": 4.74667543844416, "sim_compute_sim_state": 0.005586400826772054, "sim_compute_performance-ego": 0.007876845200856526, "sim_compute_robot_state-ego": 0.01011816660563151, "sim_compute_robot_state-npc0": 0.009567235310872396, "sim_compute_robot_state-npc1": 0.00945292075475057, "sim_compute_robot_state-npc2": 0.009288681348164875, "sim_compute_robot_state-npc3": 0.009507569471995036}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 4.760248971791701, "sim_physics": 0.1814605323562908, "survival_time": 13.350000000000056, "driven_lanedir": 4.633885959023437, "sim_render-ego": 0.01194653528906433, "in-drivable-lane": 0, "agent_compute-ego": 0.06698118792044536, "deviation-heading": 2.696291427648497, "set_robot_commands": 0.00948166400752264, "deviation-center-line": 0.6916450792324704, "driven_lanedir_consec": 4.633885959023437, "sim_compute_sim_state": 0.004719211814108859, "sim_compute_performance-ego": 0.007277411914496832, "sim_compute_robot_state-ego": 0.009294835815715432, "sim_compute_robot_state-npc0": 0.008594055747271477, "sim_compute_robot_state-npc1": 0.008263931738749872, "sim_compute_robot_state-npc2": 0.00820236884699332, "sim_compute_robot_state-npc3": 0.008219078685460466}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 4.181075554447752, "sim_physics": 0.1995266842842102, "survival_time": 14.950000000000076, "driven_lanedir": 2.4847535894546087, "sim_render-ego": 0.01274798075358073, "in-drivable-lane": 5.150000000000024, "agent_compute-ego": 0.06663281361262004, "deviation-heading": 2.787511964335597, "set_robot_commands": 0.009354098637898763, "deviation-center-line": 0.916578860535522, "driven_lanedir_consec": 2.4847535894546087, "sim_compute_sim_state": 0.004882332483927409, "sim_compute_performance-ego": 0.00746161699295044, "sim_compute_robot_state-ego": 0.00939483642578125, "sim_compute_robot_state-npc0": 0.008746498425801595, "sim_compute_robot_state-npc1": 0.008573171297709146, "sim_compute_robot_state-npc2": 0.008396185239156088, "sim_compute_robot_state-npc3": 0.00860965092976888}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 2.6192424671739754, "sim_physics": 0.22302692963017356, "survival_time": 7.199999999999982, "driven_lanedir": 2.0659178920994643, "sim_render-ego": 0.013230340348349676, "in-drivable-lane": 1.099999999999996, "agent_compute-ego": 0.07295993467171986, "deviation-heading": 1.5240802845536334, "set_robot_commands": 0.009924085603819952, "deviation-center-line": 0.4926395907016424, "driven_lanedir_consec": 2.0659178920994643, "sim_compute_sim_state": 0.005334312717119853, "sim_compute_performance-ego": 0.007609498169687059, "sim_compute_robot_state-ego": 0.009600611196623908, "sim_compute_robot_state-npc0": 0.009179587165514628, "sim_compute_robot_state-npc1": 0.009010429183642067, "sim_compute_robot_state-npc2": 0.008767942587534586, "sim_compute_robot_state-npc3": 0.008740663528442383}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 5.692718904776961, "sim_physics": 0.20420873403549197, "survival_time": 14.950000000000076, "driven_lanedir": 5.00201049253231, "sim_render-ego": 0.01277112325032552, "in-drivable-lane": 1.2999999999999954, "agent_compute-ego": 0.07142425775527954, "deviation-heading": 3.25623067255731, "set_robot_commands": 0.009922684033711751, "deviation-center-line": 0.9070332640735664, "driven_lanedir_consec": 5.00201049253231, "sim_compute_sim_state": 0.004928563435872396, "sim_compute_performance-ego": 0.007477575143178304, "sim_compute_robot_state-ego": 0.010088785489400228, "sim_compute_robot_state-npc0": 0.009233501752217613, "sim_compute_robot_state-npc1": 0.008961335817972819, "sim_compute_robot_state-npc2": 0.009158907731374104, "sim_compute_robot_state-npc3": 0.008725887934366862}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 6.021307058588554, "sim_physics": 0.19623600085576376, "survival_time": 14.950000000000076, "driven_lanedir": 4.560613931433271, "sim_render-ego": 0.012045046488444011, "in-drivable-lane": 2.9500000000000277, "agent_compute-ego": 0.06381570974985759, "deviation-heading": 3.6868274456318817, "set_robot_commands": 0.00926026503245036, "deviation-center-line": 0.9159630351092196, "driven_lanedir_consec": 4.560613931433271, "sim_compute_sim_state": 0.004762855370839437, "sim_compute_performance-ego": 0.007016512552897135, "sim_compute_robot_state-ego": 0.008974614143371583, "sim_compute_robot_state-npc0": 0.008345177173614502, "sim_compute_robot_state-npc1": 0.008194518089294434, "sim_compute_robot_state-npc2": 0.008167760372161865, "sim_compute_robot_state-npc3": 0.008132007916768392}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 5.781629584311985, "sim_physics": 0.19504515568415323, "survival_time": 14.950000000000076, "driven_lanedir": 3.834017872256752, "sim_render-ego": 0.012343523502349851, "in-drivable-lane": 4.65000000000004, "agent_compute-ego": 0.07299140691757203, "deviation-heading": 3.2450858405997565, "set_robot_commands": 0.01010127067565918, "deviation-center-line": 0.9552096839168374, "driven_lanedir_consec": 3.834017872256752, "sim_compute_sim_state": 0.0051690785090128585, "sim_compute_performance-ego": 0.007356138229370117, "sim_compute_robot_state-ego": 0.009225807189941404, "sim_compute_robot_state-npc0": 0.009008071422576903, "sim_compute_robot_state-npc1": 0.008660730520884195, "sim_compute_robot_state-npc2": 0.00871784528096517, "sim_compute_robot_state-npc3": 0.008684466679890951}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 3.4950756019794116, "sim_physics": 0.1829323121479579, "survival_time": 10.500000000000014, "driven_lanedir": 2.6619949775655267, "sim_render-ego": 0.01248603207724435, "in-drivable-lane": 2.2500000000000187, "agent_compute-ego": 0.06988145169757662, "deviation-heading": 3.2339586752493004, "set_robot_commands": 0.009651561010451544, "deviation-center-line": 0.7847018998242521, "driven_lanedir_consec": 2.6619949775655267, "sim_compute_sim_state": 0.005053607622782389, "sim_compute_performance-ego": 0.007334339051019578, "sim_compute_robot_state-ego": 0.009113857859656923, "sim_compute_robot_state-npc0": 0.008689490954081218, "sim_compute_robot_state-npc1": 0.00874700319199335, "sim_compute_robot_state-npc2": 0.008537007513500396, "sim_compute_robot_state-npc3": 0.00878488336290632}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 4.118375714440843, "sim_physics": 0.15238091965352207, "survival_time": 12.100000000000035, "driven_lanedir": 3.070889864575808, "sim_render-ego": 0.009194327780037872, "in-drivable-lane": 2.100000000000004, "agent_compute-ego": 0.0516871144948912, "deviation-heading": 3.8751441023170217, "set_robot_commands": 0.007204828183513042, "deviation-center-line": 0.6632537926149755, "driven_lanedir_consec": 3.070889864575808, "sim_compute_sim_state": 0.0037589871193751817, "sim_compute_performance-ego": 0.005295252011827201, "sim_compute_robot_state-ego": 0.006664370702317924, "sim_compute_robot_state-npc0": 0.006520888036932827, "sim_compute_robot_state-npc1": 0.006249874091345416, "sim_compute_robot_state-npc2": 0.006216243278881735, "sim_compute_robot_state-npc3": 0.006254394192340945}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 3.684783822944692, "sim_physics": 0.14021396366032687, "survival_time": 8.79999999999999, "driven_lanedir": 3.650755591789237, "sim_render-ego": 0.0091806176033887, "in-drivable-lane": 0, "agent_compute-ego": 0.05401681769977917, "deviation-heading": 1.0109120125199156, "set_robot_commands": 0.007272949272936041, "deviation-center-line": 0.6224423572612885, "driven_lanedir_consec": 3.650755591789237, "sim_compute_sim_state": 0.003785622390833768, "sim_compute_performance-ego": 0.005223054777492176, "sim_compute_robot_state-ego": 0.006663986227729104, "sim_compute_robot_state-npc0": 0.006597612391818653, "sim_compute_robot_state-npc1": 0.006515493447130377, "sim_compute_robot_state-npc2": 0.006332794373685663, "sim_compute_robot_state-npc3": 0.0063262243162501945}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 6.125731577826896, "sim_physics": 0.11745169957478842, "survival_time": 14.950000000000076, "driven_lanedir": 5.6192201123069285, "sim_render-ego": 0.00820540189743042, "in-drivable-lane": 0.7499999999999973, "agent_compute-ego": 0.04394786119461059, "deviation-heading": 2.8530673333309613, "set_robot_commands": 0.00613985538482666, "deviation-center-line": 0.9270450097907748, "driven_lanedir_consec": 5.6192201123069285, "sim_compute_sim_state": 0.003298764228820801, "sim_compute_performance-ego": 0.004827845096588135, "sim_compute_robot_state-ego": 0.005991083780924479, "sim_compute_robot_state-npc0": 0.005918153127034505, "sim_compute_robot_state-npc1": 0.005524468421936035, "sim_compute_robot_state-npc2": 0.005570945739746094, "sim_compute_robot_state-npc3": 0.005484824975331625}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 1.339017857380343, "sim_physics": 0.13391682467883145, "survival_time": 3.949999999999994, "driven_lanedir": 1.286349121351606, "sim_render-ego": 0.01023178764536411, "in-drivable-lane": 0, "agent_compute-ego": 0.04909969583342347, "deviation-heading": 0.9443903829884468, "set_robot_commands": 0.007260301445103898, "deviation-center-line": 0.22307400914119593, "driven_lanedir_consec": 1.286349121351606, "sim_compute_sim_state": 0.004230704488633554, "sim_compute_performance-ego": 0.006117425387418723, "sim_compute_robot_state-ego": 0.007220533829701098, "sim_compute_robot_state-npc0": 0.007001894938794872, "sim_compute_robot_state-npc1": 0.006847438932974128, "sim_compute_robot_state-npc2": 0.0070552765568600425, "sim_compute_robot_state-npc3": 0.006693788721591611}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 5.012755290384834, "sim_physics": 0.13614205201466878, "survival_time": 14.950000000000076, "driven_lanedir": 4.5517526916123625, "sim_render-ego": 0.009724797407786052, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.05096987326939901, "deviation-heading": 2.8379459796817934, "set_robot_commands": 0.007302385965983073, "deviation-center-line": 0.7120288910453122, "driven_lanedir_consec": 4.5517526916123625, "sim_compute_sim_state": 0.004306108951568604, "sim_compute_performance-ego": 0.0058032957712809245, "sim_compute_robot_state-ego": 0.007180186907450358, "sim_compute_robot_state-npc0": 0.007127174536387126, "sim_compute_robot_state-npc1": 0.00692331075668335, "sim_compute_robot_state-npc2": 0.006953980922698975, "sim_compute_robot_state-npc3": 0.006851398944854736}}
set_robot_commands_max0.010191832383473714
set_robot_commands_mean0.008834959540780377
set_robot_commands_median0.009354098637898763
set_robot_commands_min0.00613985538482666
sim_compute_performance-ego_max0.00823191245396932
sim_compute_performance-ego_mean0.006839464751528662
sim_compute_performance-ego_median0.007334339051019578
sim_compute_performance-ego_min0.004827845096588135
sim_compute_robot_state-ego_max0.01026759147644043
sim_compute_robot_state-ego_mean0.008630834327872724
sim_compute_robot_state-ego_median0.009225807189941404
sim_compute_robot_state-ego_min0.005991083780924479
sim_compute_robot_state-npc0_max0.009612011909484864
sim_compute_robot_state-npc0_mean0.008200839196343181
sim_compute_robot_state-npc0_median0.008689490954081218
sim_compute_robot_state-npc0_min0.005918153127034505
sim_compute_robot_state-npc1_max0.009885197480519612
sim_compute_robot_state-npc1_mean0.008038277640207888
sim_compute_robot_state-npc1_median0.008573171297709146
sim_compute_robot_state-npc1_min0.005524468421936035
sim_compute_robot_state-npc2_max0.00987558921178182
sim_compute_robot_state-npc2_mean0.008011512810370911
sim_compute_robot_state-npc2_median0.008396185239156088
sim_compute_robot_state-npc2_min0.005570945739746094
sim_compute_robot_state-npc3_max0.009718742370605467
sim_compute_robot_state-npc3_mean0.007970030615782697
sim_compute_robot_state-npc3_median0.00860965092976888
sim_compute_robot_state-npc3_min0.005484824975331625
sim_compute_sim_state_max0.006007920900980631
sim_compute_sim_state_mean0.004745031167592449
sim_compute_sim_state_median0.004882332483927409
sim_compute_sim_state_min0.003298764228820801
sim_physics_max0.22302692963017356
sim_physics_mean0.17794787520109231
sim_physics_median0.1829323121479579
sim_physics_min0.11745169957478842
sim_render-ego_max0.01389183521270752
sim_render-ego_mean0.011571726998999034
sim_render-ego_median0.012343523502349851
sim_render-ego_min0.00820540189743042
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean12.696666666666715
survival_time_min3.949999999999994
No reset possible
29845step1-simulationerroryes0:58:57
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
29807step1-simulationhost-erroryes0:17:26
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 919, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 862, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/container.py", line 755, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/containers/2c76e016adda296a95ef3740856c299f6f745b390acee27081b2a1c245d5caa9/json
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
29797step1-simulationabortedyes0:34:28
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
29791step1-simulationabortedyes0:03:57
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
29788step1-simulationabortedyes0:04:18
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible