Duckietown Challenges Home Challenges Submissions

Submission 11418

Submission11418
Competingyes
Challengeaido5-LFV-sim-validation
UserFrank (Chude) Qian 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFVv-sim: 55035
Next
User labelchallenge-aido_LF-baseline-behavior-cloning
Admin priority50
Blessingn/a
User priority50

55035

Click the images to see detailed statistics about the episode.

LFV-norm-loop-000

LFV-norm-small_loop-000

LFV-norm-techtrack-000

LFV-norm-zigzag-000

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
55035LFVv-simsuccessyes0:26:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.6621958617558428
survival_time_median14.650000000000071
deviation-center-line_median0.7017814763680197
in-drivable-lane_median2.0500000000000043


other stats
agent_compute-ego0_max0.04914097152512892
agent_compute-ego0_mean0.04580757318438754
agent_compute-ego0_median0.04553382807134669
agent_compute-ego0_min0.04302166506972786
agent_compute-npc0_max0.02821410274153706
agent_compute-npc0_mean0.02278614661188494
agent_compute-npc0_median0.02119144497018722
agent_compute-npc0_min0.02054759376562825
agent_compute-npc1_max0.03397887945175171
agent_compute-npc1_mean0.029846560329261283
agent_compute-npc1_median0.02841034146137418
agent_compute-npc1_min0.027150460074657965
agent_compute-npc2_max0.031767749786376955
agent_compute-npc2_mean0.027468749133075
agent_compute-npc2_median0.02609725178980301
agent_compute-npc2_min0.02454124582304504
agent_compute-npc3_max0.04179626082772336
agent_compute-npc3_mean0.035801100529301626
agent_compute-npc3_median0.035801100529301626
agent_compute-npc3_min0.029805940230879888
complete-iteration_max1.1477899024915243
complete-iteration_mean0.867269528958251
complete-iteration_median0.9795351322280292
complete-iteration_min0.3622179488854214
deviation-center-line_max1.1970998662303527
deviation-center-line_mean0.7701525457082916
deviation-center-line_min0.47994736386677417
deviation-heading_max6.161231033088842
deviation-heading_mean3.861319375723385
deviation-heading_median3.565205680487884
deviation-heading_min2.1536351088289276
driven_any_max4.783458036063022
driven_any_mean3.2213747044959633
driven_any_median3.328350224548573
driven_any_min1.4453403328236851
driven_lanedir_consec_max3.6280508600691097
driven_lanedir_consec_mean2.5747138874399864
driven_lanedir_consec_min1.3464129661791504
driven_lanedir_max3.6280508600691097
driven_lanedir_mean2.5747138874399864
driven_lanedir_median2.6621958617558428
driven_lanedir_min1.3464129661791504
get_duckie_state_max1.626811945099936e-06
get_duckie_state_mean1.470384359986377e-06
get_duckie_state_median1.4877313051842112e-06
get_duckie_state_min1.279262884477149e-06
get_robot_state_max0.017681341442023917
get_robot_state_mean0.014181842229729878
get_robot_state_median0.01572286528478812
get_robot_state_min0.007600296907319354
get_state_dump_max0.010883513309226052
get_state_dump_mean0.009791544642390822
get_state_dump_median0.010511208735135795
get_state_dump_min0.007260247790065639
get_ui_image_max0.05076086954112203
get_ui_image_mean0.04278010698790688
get_ui_image_median0.04468990975320205
get_ui_image_min0.030979738904101384
in-drivable-lane_max3.3000000000000185
in-drivable-lane_mean1.8500000000000068
in-drivable-lane_min0.0
per-episodes
details{"LFV-norm-loop-000-ego0": {"driven_any": 1.4453403328236851, "get_ui_image": 0.04077394349234445, "step_physics": 0.5739412563187736, "survival_time": 6.949999999999983, "driven_lanedir": 1.3464129661791504, "get_state_dump": 0.010383224487304688, "get_robot_state": 0.01464411871773856, "sim_render-ego0": 0.003651806286403111, "sim_render-npc0": 0.0037187695503234862, "sim_render-npc1": 0.003659224510192871, "sim_render-npc2": 0.003906565053122384, "get_duckie_state": 1.5343938555036273e-06, "in-drivable-lane": 0.0, "deviation-heading": 2.1536351088289276, "agent_compute-ego0": 0.045691217694963725, "agent_compute-npc0": 0.021097780976976663, "agent_compute-npc1": 0.03397887945175171, "agent_compute-npc2": 0.031767749786376955, "complete-iteration": 0.8302730219704765, "set_robot_commands": 0.00249436753136771, "deviation-center-line": 0.47994736386677417, "driven_lanedir_consec": 1.3464129661791504, "sim_compute_sim_state": 0.025967652457101004, "sim_compute_performance-ego0": 0.0019652281488691057, "sim_compute_performance-npc0": 0.001863985402243478, "sim_compute_performance-npc1": 0.0019198196274893625, "sim_compute_performance-npc2": 0.002001547813415527}, "LFV-norm-zigzag-000-ego0": {"driven_any": 4.783458036063022, "get_ui_image": 0.05076086954112203, "step_physics": 0.8112110092911247, "survival_time": 20.60000000000016, "driven_lanedir": 3.6280508600691097, "get_state_dump": 0.010639192982966906, "get_robot_state": 0.016801611851837674, "sim_render-ego0": 0.003604325486153147, "sim_render-npc0": 0.0035737190061850927, "sim_render-npc1": 0.0036033221653529574, "sim_render-npc2": 0.0035412507830751435, "sim_render-npc3": 0.0034922485490110826, "get_duckie_state": 1.279262884477149e-06, "in-drivable-lane": 3.3000000000000185, "deviation-heading": 6.161231033088842, "agent_compute-ego0": 0.04302166506972786, "agent_compute-npc0": 0.02054759376562825, "agent_compute-npc1": 0.027150460074657965, "agent_compute-npc2": 0.02454124582304504, "agent_compute-npc3": 0.029805940230879888, "complete-iteration": 1.128797242485582, "set_robot_commands": 0.0022146442034631317, "deviation-center-line": 1.1970998662303527, "driven_lanedir_consec": 3.6280508600691097, "sim_compute_sim_state": 0.0563200472630831, "sim_compute_performance-ego0": 0.0019763745638129208, "sim_compute_performance-npc0": 0.0018551055224698043, "sim_compute_performance-npc1": 0.0018080256464406304, "sim_compute_performance-npc2": 0.0018151533805717856, "sim_compute_performance-npc3": 0.0018402623783878207}, "LFV-norm-techtrack-000-ego0": {"driven_any": 3.60960504690231, "get_ui_image": 0.04860587601405965, "step_physics": 0.8126879264879678, "survival_time": 15.80000000000009, "driven_lanedir": 3.177375561499363, "get_state_dump": 0.010883513309226052, "get_robot_state": 0.017681341442023917, "sim_render-ego0": 0.0038213902843487377, "sim_render-npc0": 0.00375408329046111, "sim_render-npc1": 0.003786354786966125, "sim_render-npc2": 0.0037654040363684816, "sim_render-npc3": 0.0037355821591446457, "get_duckie_state": 1.626811945099936e-06, "in-drivable-lane": 1.2499999999999956, "deviation-heading": 3.1149546070138623, "agent_compute-ego0": 0.04537643844772964, "agent_compute-npc0": 0.021285108963397775, "agent_compute-npc1": 0.02841034146137418, "agent_compute-npc2": 0.02609725178980301, "agent_compute-npc3": 0.04179626082772336, "complete-iteration": 1.1477899024915243, "set_robot_commands": 0.002285107453162738, "deviation-center-line": 0.6193485361793041, "driven_lanedir_consec": 3.177375561499363, "sim_compute_sim_state": 0.05493359460439592, "sim_compute_performance-ego0": 0.002140597214082062, "sim_compute_performance-npc0": 0.0019484228140172143, "sim_compute_performance-npc1": 0.00195757146889479, "sim_compute_performance-npc2": 0.001934153800507073, "sim_compute_performance-npc3": 0.0019248299042133128}, "LFV-norm-small_loop-000-ego0": {"driven_any": 3.047095402194835, "get_ui_image": 0.030979738904101384, "step_physics": 0.2110787731255113, "survival_time": 13.500000000000057, "driven_lanedir": 2.1470161620123225, "get_state_dump": 0.007260247790065639, "get_robot_state": 0.007600296907319354, "sim_render-ego0": 0.004072330974564781, "sim_render-npc0": 0.003791568903905439, "get_duckie_state": 1.4410687548647948e-06, "in-drivable-lane": 2.850000000000013, "deviation-heading": 4.015456753961906, "agent_compute-ego0": 0.04914097152512892, "agent_compute-npc0": 0.02821410274153706, "complete-iteration": 0.3622179488854214, "set_robot_commands": 0.002161652399605051, "deviation-center-line": 0.7842144165567352, "driven_lanedir_consec": 2.1470161620123225, "sim_compute_sim_state": 0.011345767446989502, "sim_compute_performance-ego0": 0.002154204238384852, "sim_compute_performance-npc0": 0.0020049990763083597}}
set_robot_commands_max0.00249436753136771
set_robot_commands_mean0.002288942896899658
set_robot_commands_median0.002249875828312935
set_robot_commands_min0.002161652399605051
sim_compute_performance-ego0_max0.002154204238384852
sim_compute_performance-ego0_mean0.002059101041287235
sim_compute_performance-ego0_median0.0020584858889474917
sim_compute_performance-ego0_min0.0019652281488691057
sim_compute_performance-npc0_max0.0020049990763083597
sim_compute_performance-npc0_mean0.0019181282037597145
sim_compute_performance-npc0_median0.001906204108130346
sim_compute_performance-npc0_min0.0018551055224698043
sim_compute_performance-npc1_max0.00195757146889479
sim_compute_performance-npc1_mean0.0018951389142749277
sim_compute_performance-npc1_median0.0019198196274893625
sim_compute_performance-npc1_min0.0018080256464406304
sim_compute_performance-npc2_max0.002001547813415527
sim_compute_performance-npc2_mean0.001916951664831462
sim_compute_performance-npc2_median0.001934153800507073
sim_compute_performance-npc2_min0.0018151533805717856
sim_compute_performance-npc3_max0.0019248299042133128
sim_compute_performance-npc3_mean0.0018825461413005664
sim_compute_performance-npc3_median0.0018825461413005664
sim_compute_performance-npc3_min0.0018402623783878207
sim_compute_sim_state_max0.0563200472630831
sim_compute_sim_state_mean0.03714176544289238
sim_compute_sim_state_median0.040450623530748464
sim_compute_sim_state_min0.011345767446989502
sim_render-ego0_max0.004072330974564781
sim_render-ego0_mean0.003787463257867445
sim_render-ego0_median0.003736598285375924
sim_render-ego0_min0.003604325486153147
sim_render-npc0_max0.003791568903905439
sim_render-npc0_mean0.0037095351877187823
sim_render-npc0_median0.0037364264203922984
sim_render-npc0_min0.0035737190061850927
sim_render-npc1_max0.003786354786966125
sim_render-npc1_mean0.003682967154170651
sim_render-npc1_median0.003659224510192871
sim_render-npc1_min0.0036033221653529574
sim_render-npc2_max0.003906565053122384
sim_render-npc2_mean0.003737739957522003
sim_render-npc2_median0.0037654040363684816
sim_render-npc2_min0.0035412507830751435
sim_render-npc3_max0.0037355821591446457
sim_render-npc3_mean0.003613915354077864
sim_render-npc3_median0.003613915354077864
sim_render-npc3_min0.0034922485490110826
simulation-passed1
step_physics_max0.8126879264879678
step_physics_mean0.6022297413058444
step_physics_median0.6925761328049491
step_physics_min0.2110787731255113
survival_time_max20.60000000000016
survival_time_mean14.212500000000071
survival_time_min6.949999999999983
No reset possible
55026LFVv-simhost-erroryes0:01:53
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
55019LFVv-simsuccessyes0:25:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
55010LFVv-simsuccessyes0:25:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54994LFVv-simsuccessyes0:25:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54980LFVv-simsuccessyes0:25:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible