Duckietown Challenges Home Challenges Submissions

Submission 11080

Submission11080
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 56825
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

56825

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
56825LFv-simsuccessyes0:36:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median5.991005218153541
survival_time_median59.99999999999873
deviation-center-line_median3.0938258468329725
in-drivable-lane_median9.9249999999998


other stats
agent_compute-ego0_max0.03554844300415395
agent_compute-ego0_mean0.02423357452381461
agent_compute-ego0_median0.024801596515283898
agent_compute-ego0_min0.01178266206053671
complete-iteration_max0.23211371114510085
complete-iteration_mean0.1859780056192714
complete-iteration_median0.17882701836458154
complete-iteration_min0.1541442746028217
deviation-center-line_max3.4009170443706394
deviation-center-line_mean3.0666856192420457
deviation-center-line_min2.6781737389315983
deviation-heading_max19.30997198270497
deviation-heading_mean15.292406526220924
deviation-heading_median15.136184969322578
deviation-heading_min11.587284183533578
driven_any_max9.6687639204586
driven_any_mean9.334333358764985
driven_any_median9.320929362481513
driven_any_min9.026710789638312
driven_lanedir_consec_max8.63935556617444
driven_lanedir_consec_mean6.263253913323066
driven_lanedir_consec_min4.431649650810741
driven_lanedir_max8.63935556617444
driven_lanedir_mean7.292986090627949
driven_lanedir_median7.322108201888423
driven_lanedir_min5.888372392560509
get_duckie_state_max2.1668099840912195e-06
get_duckie_state_mean1.9289373259659513e-06
get_duckie_state_median1.9527097030245793e-06
get_duckie_state_min1.643519913723427e-06
get_robot_state_max0.003657733868004182
get_robot_state_mean0.003497746564466491
get_robot_state_median0.0035258825375178177
get_robot_state_min0.0032814873148261457
get_state_dump_max0.004644909865850215
get_state_dump_mean0.004394388665366828
get_state_dump_median0.004413376144326597
get_state_dump_min0.004105892506963903
get_ui_image_max0.03538676007006389
get_ui_image_mean0.02899328318364813
get_ui_image_median0.027363549660484956
get_ui_image_min0.02585927334355871
in-drivable-lane_max15.299999999999564
in-drivable-lane_mean9.674999999999796
in-drivable-lane_min3.5500000000000167
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 9.6687639204586, "get_ui_image": 0.02585927334355871, "step_physics": 0.08974301606590404, "survival_time": 59.99999999999873, "driven_lanedir": 8.63935556617444, "get_state_dump": 0.004260956893654092, "get_robot_state": 0.003432340566363561, "sim_render-ego0": 0.003653760158846916, "get_duckie_state": 1.8072962066116777e-06, "in-drivable-lane": 4.69999999999985, "deviation-heading": 11.587284183533578, "agent_compute-ego0": 0.014757132550064072, "complete-iteration": 0.1541442746028217, "set_robot_commands": 0.002057221410276491, "deviation-center-line": 2.6781737389315983, "driven_lanedir_consec": 8.63935556617444, "sim_compute_sim_state": 0.008434456254322265, "sim_compute_performance-ego0": 0.0018729933294825112}, "LF-norm-zigzag-000-ego0": {"driven_any": 9.156956227846743, "get_ui_image": 0.03538676007006389, "step_physics": 0.1335066176771026, "survival_time": 59.99999999999873, "driven_lanedir": 5.888372392560509, "get_state_dump": 0.004565795394999101, "get_robot_state": 0.003657733868004182, "sim_render-ego0": 0.003948925933075586, "get_duckie_state": 2.1668099840912195e-06, "in-drivable-lane": 15.299999999999564, "deviation-heading": 19.30997198270497, "agent_compute-ego0": 0.03554844300415395, "complete-iteration": 0.23211371114510085, "set_robot_commands": 0.0023095901562311966, "deviation-center-line": 2.9701959003866047, "driven_lanedir_consec": 4.431649650810741, "sim_compute_sim_state": 0.01107021017336627, "sim_compute_performance-ego0": 0.0020337162367211692}, "LF-norm-techtrack-000-ego0": {"driven_any": 9.026710789638312, "get_ui_image": 0.0286853772019665, "step_physics": 0.11547254086731872, "survival_time": 59.99999999999873, "driven_lanedir": 6.019738226244154, "get_state_dump": 0.004105892506963903, "get_robot_state": 0.0032814873148261457, "sim_render-ego0": 0.0035134181690454286, "get_duckie_state": 1.643519913723427e-06, "in-drivable-lane": 15.14999999999975, "deviation-heading": 18.25154393459369, "agent_compute-ego0": 0.01178266206053671, "complete-iteration": 0.1813210150681368, "set_robot_commands": 0.0019139113970144304, "deviation-center-line": 3.4009170443706394, "driven_lanedir_consec": 5.5564976827775645, "sim_compute_sim_state": 0.010771805797389504, "sim_compute_performance-ego0": 0.0017224592928286892}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.484902497116275, "get_ui_image": 0.02604172211900341, "step_physics": 0.09309331801015074, "survival_time": 59.99999999999873, "driven_lanedir": 8.624478177532692, "get_state_dump": 0.004644909865850215, "get_robot_state": 0.003619424508672074, "sim_render-ego0": 0.0038240350950369726, "get_duckie_state": 2.0981231994374805e-06, "in-drivable-lane": 3.5500000000000167, "deviation-heading": 12.020826004051465, "agent_compute-ego0": 0.03484606048050371, "complete-iteration": 0.1763330216610263, "set_robot_commands": 0.0021894148843274526, "deviation-center-line": 3.2174557932793397, "driven_lanedir_consec": 6.425512753529518, "sim_compute_sim_state": 0.0060929666450875285, "sim_compute_performance-ego0": 0.001897464882424233}}
set_robot_commands_max0.0023095901562311966
set_robot_commands_mean0.0021175344619623927
set_robot_commands_median0.0021233181473019717
set_robot_commands_min0.0019139113970144304
sim_compute_performance-ego0_max0.0020337162367211692
sim_compute_performance-ego0_mean0.0018816584353641508
sim_compute_performance-ego0_median0.001885229105953372
sim_compute_performance-ego0_min0.0017224592928286892
sim_compute_sim_state_max0.01107021017336627
sim_compute_sim_state_mean0.009092359717541392
sim_compute_sim_state_median0.009603131025855884
sim_compute_sim_state_min0.0060929666450875285
sim_render-ego0_max0.003948925933075586
sim_render-ego0_mean0.003735034839001226
sim_render-ego0_median0.003738897626941944
sim_render-ego0_min0.0035134181690454286
simulation-passed1
step_physics_max0.1335066176771026
step_physics_mean0.10795387315511902
step_physics_median0.10428292943873473
step_physics_min0.08974301606590404
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_min59.99999999999873
No reset possible
56813LFv-simhost-erroryes0:26:58
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 59, in get_services_id
    raise ZValueError(container_ids=container_ids, services=services, res=res, names=names)
zuper_commons.types.exceptions.ZValueError: 

│ container_ids: [16cc04c9ec40b7232c3b6acc4be1c41996899eb63c6c7cdd4ea1ab2af3e67f29]
│      services: dict[3]
│                │ evaluator:
│                │ dict[7]
│                │ │ image: docker.io/andreacensi/aido5-lf-sim-validation-lfv-sim-evaluator@sha256:6d0af9441525e1ed05be582f41e00fc178083c86797d28cd1a255c7025d0fd50
│                │ │ environment:
│                │ │ dict[10]
│                │ │ │ experiment_manager_parameters:
│                │ │ │ |episodes_per_scenario: 1
│                │ │ │ |episode_length_s: 60.0
│                │ │ │ |min_episode_length_s: 0.0
│                │ │ │ |seed: 888
│                │ │ │ |physics_dt: 0.05
│                │ │ │ |max_failures: 2
│                │ │ │ |fifo_dir: /fifos
│                │ │ │ |sim_in: /fifos/simulator-in
│                │ │ │ |sim_out: /fifos/simulator-out
│                │ │ │ |sm_in: /fifos/scenario_maker-in
│                │ │ │ |sm_out: /fifos/scenario_maker-out
│                │ │ │ |timeout_initialization: 120
│                │ │ │ |timeout_regular: 120
│                │ │ │ |port: 10123
│                │ │ │ |scenarios:
│                │ │ │ |- /scenarios
│                │ │ │ |
│                │ │ │ challenge_name: aido5-LF-sim-validation
│                │ │ │ challenge_step_name: LFv-sim
│                │ │ │ submission_id: 11080
│                │ │ │ submitter_name: melisande
│                │ │ │ SUBMISSION_CONTAINER: docker.io/melisande/aido-submissions:2020_11_05_00_37_10@sha256:c0b95c731a100a8d48b3183eedd5f223893e8fca50df37bb2857ee25f657c67e
│                │ │ │ username: ubuntu
│                │ │ │ uid: 0
│                │ │ │ USER: ubuntu
│                │ │ │ HOME: /fake-home/ubuntu
│                │ │ ports: [10123]
│                │ │ labels: {org.duckietown.created_by_runner: true, org.duckietown.runner_name: nogpu-prod-02_67f7c0c46c90}
│                │ │ user: 0:0
│                │ │ volumes:
│                │ │ [
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission11080/LFv-sim-nogpu-prod-02_67f7c0c46c90-job56813-a-wd:/challenges:rw,
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission11080/LFv-sim-nogpu-prod-02_67f7c0c46c90-job56813-a-fifos:/fifos:rw,
│                │ │  /tmp/duckietown/dt-challenges-runner/20_12_03_07_33_49-70945/fake-ubuntu-home:/fake-home/ubuntu:rw]
│                │ │ networks: {evaluation: {aliases: [evaluation]} }
│                │ simulator:
│                │ dict[6]
│                │ │ image: docker.io/duckietown/challenge-aido_lf-simulator-gym@sha256:c0096866077db3574e425d40603d8f5fc8ebbd164da7c0578df94ff4ede58d95
│                │ │ environment:
│                │ │ dict[12]
│                │ │ │ AIDONODE_CONFIG:
│                │ │ │ |env_constructor: Simulator
│                │ │ │ |env_parameters:
│                │ │ │ |  max_steps: 500001 # we don't want the gym to reset itself
│                │ │ │ |  domain_rand: 0
│                │ │ │ |  camera_width: 640
│                │ │ │ |  camera_height: 480
│                │ │ │ |  distortion: true
│                │ │ │ |  num_tris_distractors: 0
│                │ │ │ |  color_ground: [0, 0.3, 0] # green
│                │ │ │ |  enable_leds: true
│                │ │ │ |
│                │ │ │ AIDONODE_DATA_IN: /fifos/simulator-in
│                │ │ │ AIDONODE_DATA_OUT: fifo:/fifos/simulator-out
│                │ │ │ challenge_name: aido5-LF-sim-validation
│                │ │ │ challenge_step_name: LFv-sim
│                │ │ │ submission_id: 11080
│                │ │ │ submitter_name: melisande
│                │ │ │ SUBMISSION_CONTAINER: docker.io/melisande/aido-submissions:2020_11_05_00_37_10@sha256:c0b95c731a100a8d48b3183eedd5f223893e8fca50df37bb2857ee25f657c67e
│                │ │ │ username: ubuntu
│                │ │ │ uid: 0
│                │ │ │ USER: ubuntu
│                │ │ │ HOME: /fake-home/ubuntu
│                │ │ labels: {org.duckietown.created_by_runner: true, org.duckietown.runner_name: nogpu-prod-02_67f7c0c46c90}
│                │ │ user: 0:0
│                │ │ volumes:
│                │ │ [
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission11080/LFv-sim-nogpu-prod-02_67f7c0c46c90-job56813-a-wd:/challenges:rw,
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission11080/LFv-sim-nogpu-prod-02_67f7c0c46c90-job56813-a-fifos:/fifos:rw,
│                │ │  /tmp/duckietown/dt-challenges-runner/20_12_03_07_33_49-70945/fake-ubuntu-home:/fake-home/ubuntu:rw]
│                │ │ networks: {evaluation: {aliases: [evaluation]} }
│                │ solution-ego0:
│                │ dict[6]
│                │ │ image: docker.io/melisande/aido-submissions@sha256:c0b95c731a100a8d48b3183eedd5f223893e8fca50df37bb2857ee25f657c67e
│                │ │ environment:
│                │ │ dict[13]
│                │ │ │ AIDONODE_NAME: ego0
│                │ │ │ AIDONODE_DATA_IN: /fifos/ego0-in
│                │ │ │ AIDO_REQUIRE_GPU: 1
│                │ │ │ AIDONODE_DATA_OUT: fifo:/fifos/ego0-out
│                │ │ │ challenge_name: aido5-LF-sim-validation
│                │ │ │ challenge_step_name: LFv-sim
│                │ │ │ submission_id: 11080
│                │ │ │ submitter_name: melisande
│                │ │ │ SUBMISSION_CONTAINER: docker.io/melisande/aido-submissions:2020_11_05_00_37_10@sha256:c0b95c731a100a8d48b3183eedd5f223893e8fca50df37bb2857ee25f657c67e
│                │ │ │ username: ubuntu
│                │ │ │ uid: 0
│                │ │ │ USER: ubuntu
│                │ │ │ HOME: /fake-home/ubuntu
│                │ │ labels: {org.duckietown.created_by_runner: true, org.duckietown.runner_name: nogpu-prod-02_67f7c0c46c90}
│                │ │ user: 0:0
│                │ │ volumes:
│                │ │ [
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission11080/LFv-sim-nogpu-prod-02_67f7c0c46c90-job56813-a-wd:/challenges:rw,
│                │ │  /tmp/duckietown/aido5-LF-sim-validation/submission11080/LFv-sim-nogpu-prod-02_67f7c0c46c90-job56813-a-fifos:/fifos:rw,
│                │ │  /tmp/duckietown/dt-challenges-runner/20_12_03_07_33_49-70945/fake-ubuntu-home:/fake-home/ubuntu:rw]
│                │ │ networks: {evaluation: {aliases: [evaluation]} }
│           res: {solution-ego0: 16cc04c9ec40b7232c3b6acc4be1c41996899eb63c6c7cdd4ea1ab2af3e67f29}
│         names: dict[1]
│                │ 16cc04c9ec40b7232c3b6acc4be1c41996899eb63c6c7cdd4ea1ab2af3e67f29: nogpu-prod-02_67f7c0c46c90-job56813-888896_solution-ego0_1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: |16cc04c9ec40b7232c3b6acc4be1c41996899eb63c6c7cdd4ea1ab2af3e67f29
│         |
│  names: dict[1]
│         │ 16cc04c9ec40b7232c3b6acc4be1c41996899eb63c6c7cdd4ea1ab2af3e67f29: nogpu-prod-02_67f7c0c46c90-job56813-888896_solution-ego0_1
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
56807LFv-simsuccessyes0:31:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
56801LFv-simsuccessyes0:28:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible