Duckietown Challenges Home Challenges Submissions

Submission 2377

Submission2377
Competingyes
Challengeaido2-LFV-sim-testing
UserLiam Paull 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 23019
Next
User labelminimal_agent_python2 (Python 2)
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
23019step1-simulationsuccessyes0:29:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4295977528045558
survival_time_median3.7499999999999942
deviation-center-line_median0.16525966154163726
in-drivable-lane_median0.9499999999999966


other stats
agent_compute-ego_max0.06964980810880661
agent_compute-ego_mean0.06370677889882928
agent_compute-ego_median0.06426587525536032
agent_compute-ego_min0.057423023240906854
deviation-center-line_max0.44830395948846297
deviation-center-line_mean0.17891237550140934
deviation-center-line_min0.10012774388669717
deviation-heading_max1.272726254754917
deviation-heading_mean0.5883512828498624
deviation-heading_median0.5156622477370731
deviation-heading_min0.2495279988048577
driven_any_max2.0298273185719493
driven_any_mean1.0098349503422042
driven_any_median1.0248273191286312
driven_any_min0.3648701917101209
driven_lanedir_consec_max1.211071354427033
driven_lanedir_consec_mean0.6125599618319294
driven_lanedir_consec_min0.32721267753615213
driven_lanedir_max1.211071354427033
driven_lanedir_mean0.6125599618319294
driven_lanedir_median0.4295977528045558
driven_lanedir_min0.32721267753615213
in-drivable-lane_max4.249999999999985
in-drivable-lane_mean1.2033333333333296
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9948273195001778, "sim_physics": 0.1771785461739318, "survival_time": 3.649999999999995, "driven_lanedir": 0.6707441932890994, "sim_render-ego": 0.06237254077441072, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.05988680826474543, "deviation-heading": 0.5590533399929921, "set_robot_commands": 0.09804084203014636, "deviation-center-line": 0.14435558631904788, "driven_lanedir_consec": 0.6707441932890994, "sim_compute_sim_state": 0.03707967718986616, "sim_compute_performance-ego": 0.06854550805810379, "sim_compute_robot_state-ego": 0.07130100302500268, "sim_compute_robot_state-npc0": 0.06536698014768835, "sim_compute_robot_state-npc1": 0.06764278019944282, "sim_compute_robot_state-npc2": 0.06680025139900103, "sim_compute_robot_state-npc3": 0.06765945316994026}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4548365617710475, "sim_physics": 0.15756037428572373, "survival_time": 1.850000000000001, "driven_lanedir": 0.4295977528045558, "sim_render-ego": 0.057813180459512246, "in-drivable-lane": 0, "agent_compute-ego": 0.05954426043742412, "deviation-heading": 0.6191860504142408, "set_robot_commands": 0.0911558061032682, "deviation-center-line": 0.13177328856984177, "driven_lanedir_consec": 0.4295977528045558, "sim_compute_sim_state": 0.03551011472134977, "sim_compute_performance-ego": 0.06391222412521774, "sim_compute_robot_state-ego": 0.061564419720623945, "sim_compute_robot_state-npc0": 0.06724905967712402, "sim_compute_robot_state-npc1": 0.06741528253297548, "sim_compute_robot_state-npc2": 0.06371018048879262, "sim_compute_robot_state-npc3": 0.06160971280690786}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.57982731857201, "sim_physics": 0.15979171650750296, "survival_time": 5.599999999999988, "driven_lanedir": 1.2014595555356244, "sim_render-ego": 0.0552915803023747, "in-drivable-lane": 1.1999999999999955, "agent_compute-ego": 0.057423023240906854, "deviation-heading": 0.5156622477370731, "set_robot_commands": 0.08544308585780007, "deviation-center-line": 0.2085822874042474, "driven_lanedir_consec": 1.2014595555356244, "sim_compute_sim_state": 0.035165171538080485, "sim_compute_performance-ego": 0.06372510748250144, "sim_compute_robot_state-ego": 0.06820572912693024, "sim_compute_robot_state-npc0": 0.0632713862827846, "sim_compute_robot_state-npc1": 0.06357124447822571, "sim_compute_robot_state-npc2": 0.06469536253384181, "sim_compute_robot_state-npc3": 0.06355673926217216}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.5798273185720382, "sim_physics": 0.1658642590045929, "survival_time": 5.599999999999988, "driven_lanedir": 1.2017887891468408, "sim_render-ego": 0.06572959678513664, "in-drivable-lane": 1.1999999999999955, "agent_compute-ego": 0.06011446458952768, "deviation-heading": 0.5031987685845647, "set_robot_commands": 0.09424558069024767, "deviation-center-line": 0.2031955759348632, "driven_lanedir_consec": 1.2017887891468408, "sim_compute_sim_state": 0.03822559118270874, "sim_compute_performance-ego": 0.06436075695923396, "sim_compute_robot_state-ego": 0.0696533705507006, "sim_compute_robot_state-npc0": 0.06921209820679255, "sim_compute_robot_state-npc1": 0.06812990989003863, "sim_compute_robot_state-npc2": 0.06855521031788417, "sim_compute_robot_state-npc3": 0.06976360508373805}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.3798605253583843, "sim_physics": 0.19080091267824173, "survival_time": 1.6000000000000008, "driven_lanedir": 0.366714522485295, "sim_render-ego": 0.06563552469015121, "in-drivable-lane": 0, "agent_compute-ego": 0.06964980810880661, "deviation-heading": 0.42218734391092383, "set_robot_commands": 0.10518090426921844, "deviation-center-line": 0.10686452122645386, "driven_lanedir_consec": 0.366714522485295, "sim_compute_sim_state": 0.04264979809522629, "sim_compute_performance-ego": 0.07319007068872452, "sim_compute_robot_state-ego": 0.08477694541215897, "sim_compute_robot_state-npc0": 0.07423854619264603, "sim_compute_robot_state-npc1": 0.07672356069087982, "sim_compute_robot_state-npc2": 0.07665587216615677, "sim_compute_robot_state-npc3": 0.07560074329376221}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.0548273189057833, "sim_physics": 0.17475623279422908, "survival_time": 3.8499999999999943, "driven_lanedir": 0.3963714216744775, "sim_render-ego": 0.06475336830337326, "in-drivable-lane": 1.7499999999999971, "agent_compute-ego": 0.0659879833072811, "deviation-heading": 1.272726254754917, "set_robot_commands": 0.10176002205192268, "deviation-center-line": 0.20372147663934215, "driven_lanedir_consec": 0.3963714216744775, "sim_compute_sim_state": 0.038688953820761145, "sim_compute_performance-ego": 0.0734505219893022, "sim_compute_robot_state-ego": 0.07970291298705262, "sim_compute_robot_state-npc0": 0.07148391859872001, "sim_compute_robot_state-npc1": 0.0722972262989391, "sim_compute_robot_state-npc2": 0.07119433601181228, "sim_compute_robot_state-npc3": 0.06829344452201545}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.9948273195001682, "sim_physics": 0.15652815609762114, "survival_time": 3.649999999999995, "driven_lanedir": 0.6530046235032079, "sim_render-ego": 0.06115302321029036, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.06394566248541009, "deviation-heading": 0.8187446392859457, "set_robot_commands": 0.09614379765236214, "deviation-center-line": 0.19436286934102945, "driven_lanedir_consec": 0.6530046235032079, "sim_compute_sim_state": 0.03885316195553296, "sim_compute_performance-ego": 0.0688059199346255, "sim_compute_robot_state-ego": 0.07318120786588486, "sim_compute_robot_state-npc0": 0.06661191051953459, "sim_compute_robot_state-npc1": 0.06965935393555524, "sim_compute_robot_state-npc2": 0.07111017997950723, "sim_compute_robot_state-npc3": 0.07129546060954055}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 2.0298273185719493, "sim_physics": 0.18736032700874436, "survival_time": 7.099999999999983, "driven_lanedir": 0.741234184731216, "sim_render-ego": 0.06407987399840019, "in-drivable-lane": 4.249999999999985, "agent_compute-ego": 0.06563132413676087, "deviation-heading": 0.5416787058937298, "set_robot_commands": 0.10116228083489646, "deviation-center-line": 0.21649269357451997, "driven_lanedir_consec": 0.741234184731216, "sim_compute_sim_state": 0.04147259953995826, "sim_compute_performance-ego": 0.0718407714870614, "sim_compute_robot_state-ego": 0.07872488297207254, "sim_compute_robot_state-npc0": 0.07170204713310994, "sim_compute_robot_state-npc1": 0.07100059952534421, "sim_compute_robot_state-npc2": 0.07251831847177424, "sim_compute_robot_state-npc3": 0.07284513997360015}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.0248273191286528, "sim_physics": 0.15889360109965006, "survival_time": 3.7499999999999942, "driven_lanedir": 0.38223306402252977, "sim_render-ego": 0.0633326816558838, "in-drivable-lane": 2.099999999999994, "agent_compute-ego": 0.06563564618428548, "deviation-heading": 0.41826512024881946, "set_robot_commands": 0.09921127319335936, "deviation-center-line": 0.14440586530294292, "driven_lanedir_consec": 0.38223306402252977, "sim_compute_sim_state": 0.03951605478922526, "sim_compute_performance-ego": 0.0712567965189616, "sim_compute_robot_state-ego": 0.0756987476348877, "sim_compute_robot_state-npc0": 0.06900405883789062, "sim_compute_robot_state-npc1": 0.06913928667704264, "sim_compute_robot_state-npc2": 0.0713950252532959, "sim_compute_robot_state-npc3": 0.07011337916056315}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.4548365617710395, "sim_physics": 0.1733114139453785, "survival_time": 1.850000000000001, "driven_lanedir": 0.43889108545437505, "sim_render-ego": 0.0582147031217008, "in-drivable-lane": 0, "agent_compute-ego": 0.060069573892129434, "deviation-heading": 0.491310634741224, "set_robot_commands": 0.0996196849926098, "deviation-center-line": 0.11665316537398888, "driven_lanedir_consec": 0.43889108545437505, "sim_compute_sim_state": 0.039001426181277714, "sim_compute_performance-ego": 0.06872554083128234, "sim_compute_robot_state-ego": 0.07258920411805848, "sim_compute_robot_state-npc0": 0.0674774389009218, "sim_compute_robot_state-npc1": 0.06595929893287453, "sim_compute_robot_state-npc2": 0.06922377766789617, "sim_compute_robot_state-npc3": 0.06950911960086308}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.3648701917101209, "sim_physics": 0.22243431306654407, "survival_time": 1.5500000000000007, "driven_lanedir": 0.34552145550754965, "sim_render-ego": 0.06407014016182191, "in-drivable-lane": 0, "agent_compute-ego": 0.06653946445834252, "deviation-heading": 0.5070424162880827, "set_robot_commands": 0.10276282987286968, "deviation-center-line": 0.10012774388669717, "driven_lanedir_consec": 0.34552145550754965, "sim_compute_sim_state": 0.04073271443766932, "sim_compute_performance-ego": 0.07401910135822912, "sim_compute_robot_state-ego": 0.0854919264393468, "sim_compute_robot_state-npc0": 0.07244193938470656, "sim_compute_robot_state-npc1": 0.0733921604771768, "sim_compute_robot_state-npc2": 0.0719542887903029, "sim_compute_robot_state-npc3": 0.06869043073346538}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.4098472254790875, "sim_physics": 0.1617263064664953, "survival_time": 1.7000000000000008, "driven_lanedir": 0.4054410153244894, "sim_render-ego": 0.06064285250271067, "in-drivable-lane": 0, "agent_compute-ego": 0.06426587525536032, "deviation-heading": 0.2495279988048577, "set_robot_commands": 0.09466158642488368, "deviation-center-line": 0.16731811540993266, "driven_lanedir_consec": 0.4054410153244894, "sim_compute_sim_state": 0.03822943743537454, "sim_compute_performance-ego": 0.06986604017369888, "sim_compute_robot_state-ego": 0.0754256528966567, "sim_compute_robot_state-npc0": 0.06775200366973877, "sim_compute_robot_state-npc1": 0.06844136294196634, "sim_compute_robot_state-npc2": 0.06934970967909869, "sim_compute_robot_state-npc3": 0.0722207041347728}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0248273191286312, "sim_physics": 0.1479312260945638, "survival_time": 3.7499999999999942, "driven_lanedir": 0.32721267753615213, "sim_render-ego": 0.05926061630249024, "in-drivable-lane": 2.299999999999994, "agent_compute-ego": 0.06370999018351237, "deviation-heading": 0.3112188917063982, "set_robot_commands": 0.09476211229960124, "deviation-center-line": 0.16525966154163726, "driven_lanedir_consec": 0.32721267753615213, "sim_compute_sim_state": 0.03852919896443685, "sim_compute_performance-ego": 0.06767628033955891, "sim_compute_robot_state-ego": 0.07201109250386556, "sim_compute_robot_state-npc0": 0.06804041862487793, "sim_compute_robot_state-npc1": 0.07024582227071126, "sim_compute_robot_state-npc2": 0.06791179974873861, "sim_compute_robot_state-npc3": 0.06832898457845052}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 1.5798273185719856, "sim_physics": 0.1685384682246617, "survival_time": 5.599999999999988, "driven_lanedir": 1.211071354427033, "sim_render-ego": 0.06312943143504006, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.06784123820917946, "deviation-heading": 1.0143384205991224, "set_robot_commands": 0.09629139517034804, "deviation-center-line": 0.44830395948846297, "driven_lanedir_consec": 1.211071354427033, "sim_compute_sim_state": 0.03901236397879464, "sim_compute_performance-ego": 0.06798134318419866, "sim_compute_robot_state-ego": 0.07704817823001317, "sim_compute_robot_state-npc0": 0.06915429447378431, "sim_compute_robot_state-npc1": 0.06981987186840602, "sim_compute_robot_state-npc2": 0.06979520193168096, "sim_compute_robot_state-npc3": 0.06876347533294133}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 1.2198273185919868, "sim_physics": 0.2065768377347426, "survival_time": 4.399999999999992, "driven_lanedir": 0.41711373203649504, "sim_render-ego": 0.06569519909945401, "in-drivable-lane": 2.5999999999999917, "agent_compute-ego": 0.06535656072876671, "deviation-heading": 0.5811284097850438, "set_robot_commands": 0.10416370088403876, "deviation-center-line": 0.13226882250813252, "driven_lanedir_consec": 0.41711373203649504, "sim_compute_sim_state": 0.04150789705189792, "sim_compute_performance-ego": 0.07480762763456865, "sim_compute_robot_state-ego": 0.0860214585607702, "sim_compute_robot_state-npc0": 0.07220091332088817, "sim_compute_robot_state-npc1": 0.07444809783588756, "sim_compute_robot_state-npc2": 0.0753856518051841, "sim_compute_robot_state-npc3": 0.07300882718779823}}
set_robot_commands_max0.10518090426921844
set_robot_commands_mean0.09764032682183818
set_robot_commands_median0.09804084203014636
set_robot_commands_min0.08544308585780007
sim_compute_performance-ego_max0.07480762763456865
sim_compute_performance-ego_mean0.0694775740510179
sim_compute_performance-ego_median0.0688059199346255
sim_compute_performance-ego_min0.06372510748250144
sim_compute_robot_state-ego_max0.0860214585607702
sim_compute_robot_state-ego_mean0.075426448802935
sim_compute_robot_state-ego_median0.0754256528966567
sim_compute_robot_state-ego_min0.061564419720623945
sim_compute_robot_state-npc0_max0.07423854619264603
sim_compute_robot_state-npc0_mean0.06901380093141388
sim_compute_robot_state-npc0_median0.06900405883789062
sim_compute_robot_state-npc0_min0.0632713862827846
sim_compute_robot_state-npc1_max0.07672356069087982
sim_compute_robot_state-npc1_mean0.06985905723703106
sim_compute_robot_state-npc1_median0.06965935393555524
sim_compute_robot_state-npc1_min0.06357124447822571
sim_compute_robot_state-npc2_max0.07665587216615677
sim_compute_robot_state-npc2_mean0.07001701108299783
sim_compute_robot_state-npc2_median0.06979520193168096
sim_compute_robot_state-npc2_min0.06371018048879262
sim_compute_robot_state-npc3_max0.07560074329376221
sim_compute_robot_state-npc3_mean0.06941728129670208
sim_compute_robot_state-npc3_median0.06950911960086308
sim_compute_robot_state-npc3_min0.06160971280690786
sim_compute_sim_state_max0.04264979809522629
sim_compute_sim_state_mean0.03894494405881067
sim_compute_sim_state_median0.03885316195553296
sim_compute_sim_state_min0.035165171538080485
sim_physics_max0.22243431306654407
sim_physics_mean0.1739501794121749
sim_physics_median0.1685384682246617
sim_physics_min0.1479312260945638
sim_render-ego_max0.06572959678513664
sim_render-ego_mean0.062078287520183385
sim_render-ego_median0.06312943143504006
sim_render-ego_min0.0552915803023747
simulation-passed1
survival_time_max7.099999999999983
survival_time_mean3.6999999999999944
survival_time_min1.5500000000000007
No reset possible
21387step1-simulationsuccessno0:25:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21372step1-simulationhost-errorno0:28:15
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission2377/step1-simulation-ip-172-31-38-104-5376-job21372/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21352step1-simulationhost-errorno0:04:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission2377/step1-simulation-ip-172-31-42-167-7357-job21352/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20612step1-simulationsuccessno0:07:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19663step1-simulationsuccessno0:12:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18912step1-simulationsuccessno0:17:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18546step1-simulationsuccessno0:17:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17872step1-simulationsuccessno0:09:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17545step1-simulationsuccessno0:09:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17539step1-simulationsuccessno0:09:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible