Duckietown Challenges Home Challenges Submissions

Submission 2818

Submission2818
Competingyes
Challengeaido2-LF-sim-testing
UserLiam Paull 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22759
Next
User labelchallenge-aido_LF-template-random
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
22759step1-simulationsuccessyes0:15:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.10099507669212504
in-drivable-lane_median0.4499999999999984


other stats
agent_compute-ego_max0.15117651341008206
agent_compute-ego_mean0.14068361804031848
agent_compute-ego_median0.14280658108847483
agent_compute-ego_min0.11937679076681332
deviation-center-line_max0.2838635700333795
deviation-center-line_mean0.12786165368054708
deviation-center-line_min0.07112529346797256
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3391682685652287
deviation-heading_median0.32928385103533875
deviation-heading_min0.19401296908406307
driven_any_max2.1706243860917107
driven_any_mean0.9759490378276146
driven_any_median1.0117660515775693
driven_any_min0.23626560261560617
driven_lanedir_consec_max1.919400421063832
driven_lanedir_consec_mean0.6896171461819246
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max1.919400421063832
driven_lanedir_mean0.6896171461819246
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.5899999999999982
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.11176469149412932, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.05705622831980387, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.14148211479187012, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.0924577624709518, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.03756400832423457, "sim_compute_performance-ego": 0.06091920093253807, "sim_compute_robot_state-ego": 0.06821718039336028}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.10657774700837976, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.05898066595488904, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.1476132542479272, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.08589900241178625, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.03690014165990493, "sim_compute_performance-ego": 0.06324121063830805, "sim_compute_robot_state-ego": 0.06883692741394043}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.12251743148354922, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.0591580447028665, "in-drivable-lane": 0, "agent_compute-ego": 0.1357131284825942, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.08997802173390108, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.042992143069996554, "sim_compute_performance-ego": 0.061269409516278434, "sim_compute_robot_state-ego": 0.06797632049111758}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.11990645408630372, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.06403008460998535, "in-drivable-lane": 0, "agent_compute-ego": 0.14908554077148437, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.10303014755249024, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.03944601058959961, "sim_compute_performance-ego": 0.06810245513916016, "sim_compute_robot_state-ego": 0.0667952060699463}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.618214457700942, "sim_physics": 0.11308641071561017, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.06059138684333125, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.14953582498091686, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.0956459166128424, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.03727719451807722, "sim_compute_performance-ego": 0.06267236456086364, "sim_compute_robot_state-ego": 0.0696328259721587}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.0038514956833002, "sim_physics": 0.08870693615504674, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7531703123735944, "sim_render-ego": 0.049352096051585914, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.11937679076681332, "deviation-heading": 0.4596437246381204, "set_robot_commands": 0.07476212540451362, "deviation-center-line": 0.08968283443958727, "driven_lanedir_consec": 0.7531703123735944, "sim_compute_sim_state": 0.029975949501504704, "sim_compute_performance-ego": 0.0529802283462213, "sim_compute_robot_state-ego": 0.0599993540316212}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0542898001504375, "sim_physics": 0.07600830282483782, "survival_time": 2.799999999999998, "driven_lanedir": 0.5389398655375865, "sim_render-ego": 0.04772914733205523, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.11945236580712457, "deviation-heading": 0.4109275195546752, "set_robot_commands": 0.0758951987539019, "deviation-center-line": 0.10786249131209884, "driven_lanedir_consec": 0.5389398655375865, "sim_compute_sim_state": 0.032044129712241035, "sim_compute_performance-ego": 0.05164019976343427, "sim_compute_robot_state-ego": 0.053339366401944845}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3164475681403097, "sim_physics": 0.11845144771394275, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3093025166124813, "sim_render-ego": 0.05931589716956729, "in-drivable-lane": 0, "agent_compute-ego": 0.14458686964852469, "deviation-heading": 0.2242052814634115, "set_robot_commands": 0.0844211010705857, "deviation-center-line": 0.09758615464743424, "driven_lanedir_consec": 0.3093025166124813, "sim_compute_sim_state": 0.03566882723853702, "sim_compute_performance-ego": 0.06128812971569243, "sim_compute_robot_state-ego": 0.07168849309285481}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2018528489510765, "sim_physics": 0.10380313760143216, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6922532395203, "sim_render-ego": 0.056251024795790854, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.14532009625838974, "deviation-heading": 0.32928385103533875, "set_robot_commands": 0.08636815265073615, "deviation-center-line": 0.07112529346797256, "driven_lanedir_consec": 0.6922532395203, "sim_compute_sim_state": 0.032322358276884434, "sim_compute_performance-ego": 0.06191707465608241, "sim_compute_robot_state-ego": 0.06369139784473483}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.0435481553257333, "sim_physics": 0.1008928125554865, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5204405548974873, "sim_render-ego": 0.05921255891973322, "in-drivable-lane": 1.1499999999999977, "agent_compute-ego": 0.1456624984741211, "deviation-heading": 0.33296447766287096, "set_robot_commands": 0.08346153172579679, "deviation-center-line": 0.0950267943453044, "driven_lanedir_consec": 0.5204405548974873, "sim_compute_sim_state": 0.03369640437039462, "sim_compute_performance-ego": 0.06498578244989568, "sim_compute_robot_state-ego": 0.06657355915416371}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5954548672193265, "sim_physics": 0.10572023825211958, "survival_time": 3.8499999999999943, "driven_lanedir": 1.1042463557848754, "sim_render-ego": 0.057809030854856815, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.13627429441972214, "deviation-heading": 0.3566810710676767, "set_robot_commands": 0.09021450946857404, "deviation-center-line": 0.22513914436082189, "driven_lanedir_consec": 1.1042463557848754, "sim_compute_sim_state": 0.03597989949313077, "sim_compute_performance-ego": 0.0629664489201137, "sim_compute_robot_state-ego": 0.06540992662504122}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.49030923107489305, "sim_physics": 0.11284977197647096, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4826393189617785, "sim_render-ego": 0.057041517325810025, "in-drivable-lane": 0, "agent_compute-ego": 0.14280658108847483, "deviation-heading": 0.23322336656472228, "set_robot_commands": 0.08812700850622994, "deviation-center-line": 0.10099507669212504, "driven_lanedir_consec": 0.4826393189617785, "sim_compute_sim_state": 0.03591845716748919, "sim_compute_performance-ego": 0.06432822772434779, "sim_compute_robot_state-ego": 0.06650879553386144}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0117660515775693, "sim_physics": 0.10891577309253168, "survival_time": 2.549999999999999, "driven_lanedir": 0.791717039833255, "sim_render-ego": 0.06148292971592323, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.15117651341008206, "deviation-heading": 0.320449356593834, "set_robot_commands": 0.0952617561115938, "deviation-center-line": 0.1017166571540705, "driven_lanedir_consec": 0.791717039833255, "sim_compute_sim_state": 0.03877721580804563, "sim_compute_performance-ego": 0.07104898434059292, "sim_compute_robot_state-ego": 0.07849649822010714}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.44419823353351984, "sim_physics": 0.09960473908318414, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4333246859278659, "sim_render-ego": 0.05808208606861256, "in-drivable-lane": 0, "agent_compute-ego": 0.1395866517667417, "deviation-heading": 0.2988926743481924, "set_robot_commands": 0.0896724506660744, "deviation-center-line": 0.08545702330592322, "driven_lanedir_consec": 0.4333246859278659, "sim_compute_sim_state": 0.03840106504934805, "sim_compute_performance-ego": 0.061319986979166664, "sim_compute_robot_state-ego": 0.06418256406430844}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1706243860917107, "sim_physics": 0.09305192442501292, "survival_time": 5.09999999999999, "driven_lanedir": 1.919400421063832, "sim_render-ego": 0.05624539711896111, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.14258174568999046, "deviation-heading": 0.5178939525116442, "set_robot_commands": 0.0884536247627408, "deviation-center-line": 0.2838635700333795, "driven_lanedir_consec": 1.919400421063832, "sim_compute_sim_state": 0.0359801451365153, "sim_compute_performance-ego": 0.06106500765856575, "sim_compute_robot_state-ego": 0.06379492843852323}}
set_robot_commands_max0.10303014755249024
set_robot_commands_mean0.08824322066018127
set_robot_commands_median0.0884536247627408
set_robot_commands_min0.07476212540451362
sim_compute_performance-ego_max0.07104898434059292
sim_compute_performance-ego_mean0.06198298075608408
sim_compute_performance-ego_median0.06191707465608241
sim_compute_performance-ego_min0.05164019976343427
sim_compute_robot_state-ego_max0.07849649822010714
sim_compute_robot_state-ego_mean0.06634288958317895
sim_compute_robot_state-ego_median0.06657355915416371
sim_compute_robot_state-ego_min0.053339366401944845
sim_compute_sim_state_max0.042992143069996554
sim_compute_sim_state_mean0.036196263327726914
sim_compute_sim_state_median0.0359801451365153
sim_compute_sim_state_min0.029975949501504704
sim_physics_max0.12251743148354922
sim_physics_mean0.10545718789786918
sim_physics_median0.10657774700837976
sim_physics_min0.07600830282483782
sim_render-ego_max0.06403008460998535
sim_render-ego_mean0.057489206385584815
sim_render-ego_median0.05808208606861256
sim_render-ego_min0.04772914733205523
simulation-passed1
survival_time_max5.09999999999999
survival_time_mean2.503333333333331
survival_time_min0.8500000000000002
No reset possible
22757step1-simulationhost-erroryes0:00:10
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 255, in _do_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.6/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+ip4://127.0.0.1:5001/api/v0/swarm/connect?stream-channels=true&arg=%2Fdns4%2Fipfs.duckietown.org%2Ftcp%2F4001%2Fipfs%2FQmPyoL4ZwaTYtGsvFG8A5fG85tQH5sCHWtexkNVjqa52iK

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 541, in get_cr
    allow_host_network=allow_host_network, use_ipfs=use_ipfs)
  File "/project/src/duckietown_challenges_runner/runner.py", line 605, in run_single
    prepare_dir(wd, aws_config, steps2artefacts, use_ipfs=use_ipfs)
  File "/project/src/duckietown_challenges_runner/runner.py", line 877, in prepare_dir
    download_artefacts(aws_config, steps2artefacts, previous_steps_dir, use_ipfs=use_ipfs)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1151, in download_artefacts
    res = client.swarm_connect(peer)
  File "/usr/local/lib/python3.6/dist-packages/ipfsapi/client/__init__.py", line 205, in wrapper
    return value(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/client/base.py", line 15, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/client/swarm.py", line 137, in connect
    return self._client.request('/swarm/connect', args, decoder='json', **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 44, in wrapper
    return func(self, *args, **merged)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 364, in request
    files, headers, data, timeout=timeout)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 286, in _request
    self._do_raise_for_status(res)
  File "/usr/local/lib/python3.6/dist-packages/ipfshttpclient/http.py", line 273, in _do_raise_for_status
    six.raise_from(exceptions.ErrorResponse(msg, error), error)
  File "<string>", line 3, in raise_from
ipfshttpclient.exceptions.ErrorResponse: connect QmPyoL4ZwaTYtGsvFG8A5fG85tQH5sCHWtexkNVjqa52iK failure: dial attempt failed: <peer.ID T35iPP> --> <peer.ID PyoL4Z> dial attempt failed: context deadline exceeded
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible