Duckietown Challenges Home Challenges Submissions

Submission 3103

Submission3103
Competingyes
Challengeaido2-LFVI-sim-validation
UserAngel Woo 🇭🇰
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22614
Next
User labelHKU Duckietown Project
Admin priority50
Blessingn/a
User priority50

22614

Click the images to see detailed statistics about the episode.

udem1-0-0

udem1-1-0

udem1-2-0

udem1-3-0

udem1-4-0

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22614step1-simulationsuccessyes0:28:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5960344577630978
survival_time_median14.950000000000076
deviation-center-line_median0.4729581256067054
in-drivable-lane_median4.950000000000059


other stats
agent_compute-ego_max0.20896032621275704
agent_compute-ego_mean0.2057481556232995
agent_compute-ego_median0.20534555241465569
agent_compute-ego_min0.20248953262964883
deviation-center-line_max1.1065022404277178
deviation-center-line_mean0.6024107793282442
deviation-center-line_min0.420707132970694
deviation-heading_max3.1905536033833224
deviation-heading_mean2.461303614592768
deviation-heading_median2.3079695963082294
deviation-heading_min2.21975396913907
driven_any_max0.9588858510103172
driven_any_mean0.8986403844759157
driven_any_median0.9326864229495406
driven_any_min0.8177868647301125
driven_lanedir_consec_max0.9198101823025568
driven_lanedir_consec_mean0.6020364282893003
driven_lanedir_consec_min0.44411408179003287
driven_lanedir_max0.9198101823025568
driven_lanedir_mean0.602065555465755
driven_lanedir_median0.5960344577630978
driven_lanedir_min0.44411408179003287
in-drivable-lane_max5.650000000000072
in-drivable-lane_mean4.100000000000053
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 0.9343848022084076, "sim_physics": 0.1139144508043925, "survival_time": 14.950000000000076, "driven_lanedir": 0.5960344577630978, "sim_render-ego": 0.06283006906509399, "in-drivable-lane": 4.950000000000059, "agent_compute-ego": 0.204780371983846, "deviation-heading": 2.323967401219826, "set_robot_commands": 0.07784392197926839, "deviation-center-line": 1.1065022404277178, "driven_lanedir_consec": 0.5960344577630978, "sim_compute_sim_state": 0.036381514072418214, "sim_compute_performance-ego": 0.06474499225616455, "sim_compute_robot_state-ego": 0.061558807690938314, "sim_compute_robot_state-npc0": 0.07176120281219482, "sim_compute_robot_state-npc1": 0.06560962041219076, "sim_compute_robot_state-npc2": 0.0653529977798462, "sim_compute_robot_state-npc3": 0.06461800654729208}, "udem1-1-0": {"driven_any": 0.9588858510103172, "sim_physics": 0.10559096972147625, "survival_time": 14.950000000000076, "driven_lanedir": 0.5973879663678017, "sim_render-ego": 0.062173454761505126, "in-drivable-lane": 4.550000000000065, "agent_compute-ego": 0.20248953262964883, "deviation-heading": 3.1905536033833224, "set_robot_commands": 0.07691226323445638, "deviation-center-line": 0.420707132970694, "driven_lanedir_consec": 0.5972423304855279, "sim_compute_sim_state": 0.0356901748975118, "sim_compute_performance-ego": 0.06517179250717163, "sim_compute_robot_state-ego": 0.060761152903238934, "sim_compute_robot_state-npc0": 0.07196073293685913, "sim_compute_robot_state-npc1": 0.06689138809839884, "sim_compute_robot_state-npc2": 0.06607143402099609, "sim_compute_robot_state-npc3": 0.06549856106440226}, "udem1-2-0": {"driven_any": 0.8177868647301125, "sim_physics": 0.10932452976703644, "survival_time": 12.800000000000049, "driven_lanedir": 0.44411408179003287, "sim_render-ego": 0.06197608076035976, "in-drivable-lane": 5.350000000000065, "agent_compute-ego": 0.20534555241465569, "deviation-heading": 2.3079695963082294, "set_robot_commands": 0.0767769142985344, "deviation-center-line": 0.4655137936444971, "driven_lanedir_consec": 0.44411408179003287, "sim_compute_sim_state": 0.036598728969693184, "sim_compute_performance-ego": 0.06467641331255436, "sim_compute_robot_state-ego": 0.061722053214907646, "sim_compute_robot_state-npc0": 0.07242604717612267, "sim_compute_robot_state-npc1": 0.06692030653357506, "sim_compute_robot_state-npc2": 0.06583456601947546, "sim_compute_robot_state-npc3": 0.06518376804888248}, "udem1-3-0": {"driven_any": 0.8494579814812006, "sim_physics": 0.11380449870847306, "survival_time": 13.250000000000052, "driven_lanedir": 0.4529810891052857, "sim_render-ego": 0.06510261859533922, "in-drivable-lane": 5.650000000000072, "agent_compute-ego": 0.20896032621275704, "deviation-heading": 2.26427350291339, "set_robot_commands": 0.07777013958625073, "deviation-center-line": 0.4729581256067054, "driven_lanedir_consec": 0.4529810891052857, "sim_compute_sim_state": 0.03775805347370652, "sim_compute_performance-ego": 0.06600021596224803, "sim_compute_robot_state-ego": 0.06279488149678931, "sim_compute_robot_state-npc0": 0.07355179876651403, "sim_compute_robot_state-npc1": 0.06857817667835163, "sim_compute_robot_state-npc2": 0.06665149904646964, "sim_compute_robot_state-npc3": 0.06763066885606298}, "udem1-4-0": {"driven_any": 0.9326864229495406, "sim_physics": 0.10654062747955324, "survival_time": 14.950000000000076, "driven_lanedir": 0.9198101823025568, "sim_render-ego": 0.06337092002232869, "in-drivable-lane": 0, "agent_compute-ego": 0.20716499487559, "deviation-heading": 2.21975396913907, "set_robot_commands": 0.07793591101964314, "deviation-center-line": 0.5463726039916064, "driven_lanedir_consec": 0.9198101823025568, "sim_compute_sim_state": 0.03800375541051229, "sim_compute_performance-ego": 0.06605809052785237, "sim_compute_robot_state-ego": 0.06364762703577677, "sim_compute_robot_state-npc0": 0.07403849522272746, "sim_compute_robot_state-npc1": 0.06993834972381592, "sim_compute_robot_state-npc2": 0.0681225601832072, "sim_compute_robot_state-npc3": 0.06754117727279663}}
set_robot_commands_max0.07793591101964314
set_robot_commands_mean0.0774478300236306
set_robot_commands_median0.07777013958625073
set_robot_commands_min0.0767769142985344
sim_compute_performance-ego_max0.06605809052785237
sim_compute_performance-ego_mean0.06533030091319819
sim_compute_performance-ego_median0.06517179250717163
sim_compute_performance-ego_min0.06467641331255436
sim_compute_robot_state-ego_max0.06364762703577677
sim_compute_robot_state-ego_mean0.062096904468330195
sim_compute_robot_state-ego_median0.061722053214907646
sim_compute_robot_state-ego_min0.060761152903238934
sim_compute_robot_state-npc0_max0.07403849522272746
sim_compute_robot_state-npc0_mean0.07274765538288361
sim_compute_robot_state-npc0_median0.07242604717612267
sim_compute_robot_state-npc0_min0.07176120281219482
sim_compute_robot_state-npc1_max0.06993834972381592
sim_compute_robot_state-npc1_mean0.06758756828926644
sim_compute_robot_state-npc1_median0.06692030653357506
sim_compute_robot_state-npc1_min0.06560962041219076
sim_compute_robot_state-npc2_max0.0681225601832072
sim_compute_robot_state-npc2_mean0.0664066114099989
sim_compute_robot_state-npc2_median0.06607143402099609
sim_compute_robot_state-npc2_min0.0653529977798462
sim_compute_robot_state-npc3_max0.06763066885606298
sim_compute_robot_state-npc3_mean0.06609443635788728
sim_compute_robot_state-npc3_median0.06549856106440226
sim_compute_robot_state-npc3_min0.06461800654729208
sim_compute_sim_state_max0.03800375541051229
sim_compute_sim_state_mean0.0368864453647684
sim_compute_sim_state_median0.036598728969693184
sim_compute_sim_state_min0.0356901748975118
sim_physics_max0.1139144508043925
sim_physics_mean0.1098350152961863
sim_physics_median0.10932452976703644
sim_physics_min0.10559096972147625
sim_render-ego_max0.06510261859533922
sim_render-ego_mean0.06309062864092536
sim_render-ego_median0.06283006906509399
sim_render-ego_min0.06197608076035976
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.180000000000067
survival_time_min12.800000000000049
No reset possible
22610step1-simulationhost-erroryes0:00:42
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 255, in _do_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+ip4://127.0.0.1:5001/api/v0/swarm/connect?stream-channels=true&arg=%2Fip4%2F129.132.0.37%2Ftcp%2F4001%2Fipfs%2FQmW5P8PZhGYGoyGzAGZNKNTKrvbg8m7Wv4QF4o2ghYmuf9

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 541, in get_cr
    allow_host_network=allow_host_network, use_ipfs=use_ipfs)
  File "/project/src/duckietown_challenges_runner/runner.py", line 605, in run_single
    prepare_dir(wd, aws_config, steps2artefacts, use_ipfs=use_ipfs)
  File "/project/src/duckietown_challenges_runner/runner.py", line 877, in prepare_dir
    download_artefacts(aws_config, steps2artefacts, previous_steps_dir, use_ipfs=use_ipfs)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1151, in download_artefacts
    res = client.swarm_connect(peer)
  File "/usr/local/lib/python3.6/dist-packages/ipfsapi/client/__init__.py", line 205, in wrapper
    return value(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/client/base.py", line 15, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/client/swarm.py", line 137, in connect
    return self._client.request('/swarm/connect', args, decoder='json', **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 44, in wrapper
    return func(self, *args, **merged)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 364, in request
    files, headers, data, timeout=timeout)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 286, in _request
    self._do_raise_for_status(res)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 273, in _do_raise_for_status
    six.raise_from(exceptions.ErrorResponse(msg, error), error)
  File "<string>", line 3, in raise_from
ipfshttpclient.exceptions.ErrorResponse: connect QmW5P8PZhGYGoyGzAGZNKNTKrvbg8m7Wv4QF4o2ghYmuf9 failure: dial attempt failed: <peer.ID T35iPP> --> <peer.ID W5P8PZ> dial attempt failed: context deadline exceeded
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20861step1-simulationsuccessno0:37:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20825step1-simulationhost-errorno0:39:25
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-validation/submission3103/step1-simulation-ip-172-31-40-253-32125-job20825/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20785step1-simulationsuccessno0:38:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20772step1-simulationhost-errorno0:02:49
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 483, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-validation/submission3103/step1-simulation-ip-172-31-40-253-32059-job20772/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20771step1-simulationsuccessno0:26:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible