Duckietown Challenges Home Challenges Submissions

Evaluator 2260

ID2260
evaluatormont02-5bd91ed070bc-1
ownerI don't have one 😀
machineaido-ms-7b09
processmont02-5bd91ed070bc-1
versiond-c:6.1.23;d-c-r:6.0.59
first heard
last heard
statusinactive
# evaluating
# success5 40846
# timeout
# failed3 40854
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
4091010812Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:03:05
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 11 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4090311281Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simsuccessnomont02-5bd91ed070bc-10:04:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.19372326206745316
survival_time_median2.9999999999999973
deviation-center-line_median0.1869472201393102
in-drivable-lane_median0.5999999999999979


other stats
agent_compute-ego_max0.026847834857004992
agent_compute-ego_mean0.021910729229362
agent_compute-ego_median0.02145958808182323
agent_compute-ego_min0.017875905896796555
complete-iteration_max0.22053487795703816
complete-iteration_mean0.20880144525779223
complete-iteration_median0.2115331346369059
complete-iteration_min0.19160463380031895
deviation-center-line_max0.5495671983592727
deviation-center-line_mean0.2688420132995113
deviation-center-line_min0.15190641456015214
deviation-heading_max2.631101092599843
deviation-heading_mean1.435546958428202
deviation-heading_median1.1400183272263291
deviation-heading_min0.8310500866603063
driven_any_max1.7598657195991556
driven_any_mean0.6454860521378534
driven_any_median0.2882989119776772
driven_any_min0.24548066499690363
driven_lanedir_consec_max0.794806045898188
driven_lanedir_consec_mean0.3324413107456489
driven_lanedir_consec_min0.14751267294950132
driven_lanedir_max0.794806045898188
driven_lanedir_mean0.3324413107456489
driven_lanedir_median0.19372326206745316
driven_lanedir_min0.14751267294950132
get_duckie_state_max2.9158592224121095e-06
get_duckie_state_mean2.5621940922303273e-06
get_duckie_state_median2.470457176450041e-06
get_duckie_state_min2.3920027936091188e-06
get_robot_state_max0.01561085233148539
get_robot_state_mean0.01530725333630547
get_robot_state_median0.015519630459488416
get_robot_state_min0.014578900094759666
get_state_dump_max0.014199531303261813
get_state_dump_mean0.014018286255748386
get_state_dump_median0.013998731237943056
get_state_dump_min0.013876151243845622
get_ui_image_max0.03786450062157973
get_ui_image_mean0.03594208860400526
get_ui_image_median0.03608590738248017
get_ui_image_min0.03373203902948098
in-drivable-lane_max7.30000000000001
in-drivable-lane_mean2.262500000000001
in-drivable-lane_min0.549999999999998
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 0.2939120129418644, "get_ui_image": 0.03373203902948098, "step_physics": 0.07400961391261367, "survival_time": 3.049999999999997, "driven_lanedir": 0.19520875111927616, "get_state_dump": 0.014094083035578494, "sim_render-ego": 0.00591020896786549, "get_robot_state": 0.015454632336976096, "get_duckie_state": 2.3920027936091188e-06, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.017875905896796555, "deviation-heading": 1.0871645633522449, "complete-iteration": 0.19160463380031895, "set_robot_commands": 0.004567142392768234, "deviation-center-line": 0.17706217931526982, "driven_lanedir_consec": 0.19520875111927616, "sim_compute_sim_state": 0.021897390240528544, "sim_compute_performance-ego": 0.003922247495807585}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.24548066499690363, "get_ui_image": 0.03786450062157973, "step_physics": 0.08627856452510042, "survival_time": 2.6499999999999986, "driven_lanedir": 0.14751267294950132, "get_state_dump": 0.014199531303261813, "sim_render-ego": 0.0065348193330584834, "get_robot_state": 0.01561085233148539, "get_duckie_state": 2.451662747365124e-06, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.026847834857004992, "deviation-heading": 1.1928720911004134, "complete-iteration": 0.22053487795703816, "set_robot_commands": 0.004804872116952573, "deviation-center-line": 0.15190641456015214, "driven_lanedir_consec": 0.14751267294950132, "sim_compute_sim_state": 0.023810274196120928, "sim_compute_performance-ego": 0.004425512169891933}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 1.7598657195991556, "get_ui_image": 0.03645721673965454, "step_physics": 0.08342824935913086, "survival_time": 14.950000000000076, "driven_lanedir": 0.794806045898188, "get_state_dump": 0.013876151243845622, "sim_render-ego": 0.00632590373357137, "get_robot_state": 0.015584628582000732, "get_duckie_state": 2.9158592224121095e-06, "in-drivable-lane": 7.30000000000001, "agent_compute-ego": 0.021516307989756265, "deviation-heading": 2.631101092599843, "complete-iteration": 0.21010823647181193, "set_robot_commands": 0.004594769477844238, "deviation-center-line": 0.5495671983592727, "driven_lanedir_consec": 0.794806045898188, "sim_compute_sim_state": 0.02396919806798299, "sim_compute_performance-ego": 0.004201109409332275}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 0.28268581101349005, "get_ui_image": 0.0357145980253058, "step_physics": 0.08560622748682055, "survival_time": 2.9499999999999975, "driven_lanedir": 0.19223777301563016, "get_state_dump": 0.013903379440307615, "sim_render-ego": 0.006286908004243495, "get_robot_state": 0.014578900094759666, "get_duckie_state": 2.4892516055349578e-06, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.021402868173890196, "deviation-heading": 0.8310500866603063, "complete-iteration": 0.21295803280199987, "set_robot_commands": 0.00461381168688758, "deviation-center-line": 0.1968322609633506, "driven_lanedir_consec": 0.19223777301563016, "sim_compute_sim_state": 0.02660312490948176, "sim_compute_performance-ego": 0.004086664167501159}}
set_robot_commands_max0.004804872116952573
set_robot_commands_mean0.004645148918613156
set_robot_commands_median0.0046042905823659094
set_robot_commands_min0.004567142392768234
sim_compute_performance-ego_max0.004425512169891933
sim_compute_performance-ego_mean0.004158883310633238
sim_compute_performance-ego_median0.004143886788416717
sim_compute_performance-ego_min0.003922247495807585
sim_compute_sim_state_max0.02660312490948176
sim_compute_sim_state_mean0.024069996853528557
sim_compute_sim_state_median0.02388973613205196
sim_compute_sim_state_min0.021897390240528544
sim_render-ego_max0.0065348193330584834
sim_render-ego_mean0.00626446000968471
sim_render-ego_median0.006306405868907433
sim_render-ego_min0.00591020896786549
simulation-passed1
step_physics_max0.08627856452510042
step_physics_mean0.08233066382091637
step_physics_median0.0845172384229757
step_physics_min0.07400961391261367
survival_time_max14.950000000000076
survival_time_mean5.900000000000018
survival_time_min2.6499999999999986
No reset possible
4089710823Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:03:12
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 64 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4088610831Raphael Jeanreal-exercise-1aido5-LF-sim-validationLFv-simsuccessnomont02-5bd91ed070bc-10:08:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.8628543308553376
survival_time_median14.950000000000076
deviation-center-line_median0.38026001167224854
in-drivable-lane_median3.5500000000000504


other stats
agent_compute-ego_max0.018099677562713624
agent_compute-ego_mean0.01752783376975416
agent_compute-ego_median0.017365502119064333
agent_compute-ego_min0.017280653278174348
complete-iteration_max0.21285916328430177
complete-iteration_mean0.20142778949169077
complete-iteration_median0.19946717572664735
complete-iteration_min0.19391764322916663
deviation-center-line_max0.4243749399962627
deviation-center-line_mean0.3731096346821197
deviation-center-line_min0.30754357538771915
deviation-heading_max2.0228796574899586
deviation-heading_mean1.7702443430386383
deviation-heading_median1.7306529838314124
deviation-heading_min1.5967917470017687
driven_any_max2.5761522243532378
driven_any_mean2.4151286667896175
driven_any_median2.427088818874764
driven_any_min2.230184805055705
driven_lanedir_consec_max2.0993503343414757
driven_lanedir_consec_mean1.8367812319953285
driven_lanedir_consec_min1.5220659319291625
driven_lanedir_max2.0993503343414757
driven_lanedir_mean1.8367812319953285
driven_lanedir_median1.8628543308553376
driven_lanedir_min1.5220659319291625
get_duckie_state_max2.7379955685435663e-06
get_duckie_state_mean2.613702561366035e-06
get_duckie_state_median2.5999546051025393e-06
get_duckie_state_min2.5169054667154948e-06
get_robot_state_max0.01583110253016154
get_robot_state_mean0.015258512372506751
get_robot_state_median0.015314748038317947
get_robot_state_min0.014573450883229573
get_state_dump_max0.0136674960454305
get_state_dump_mean0.013527538414386102
get_state_dump_median0.01351643450178144
get_state_dump_min0.013409788608551026
get_ui_image_max0.0363680370648702
get_ui_image_mean0.03535990983070994
get_ui_image_median0.03525530099868775
get_ui_image_min0.034561000260594055
in-drivable-lane_max5.500000000000078
in-drivable-lane_mean3.8375000000000545
in-drivable-lane_min2.750000000000039
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.297395975550146, "get_ui_image": 0.034561000260594055, "step_physics": 0.07618773535053076, "survival_time": 14.050000000000065, "driven_lanedir": 1.5220659319291625, "get_state_dump": 0.01348690206045782, "sim_render-ego": 0.006001964582667232, "get_robot_state": 0.014918482600582029, "get_duckie_state": 2.7379955685435663e-06, "in-drivable-lane": 5.500000000000078, "agent_compute-ego": 0.017280653278174348, "deviation-heading": 1.5967917470017687, "complete-iteration": 0.19481493060699137, "set_robot_commands": 0.0046924305980315835, "deviation-center-line": 0.30754357538771915, "driven_lanedir_consec": 1.5220659319291625, "sim_compute_sim_state": 0.02353738635460253, "sim_compute_performance-ego": 0.004003867560009939}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.5761522243532378, "get_ui_image": 0.0350341796875, "step_physics": 0.07597347021102906, "survival_time": 14.950000000000076, "driven_lanedir": 2.0993503343414757, "get_state_dump": 0.013409788608551026, "sim_render-ego": 0.00591000239054362, "get_robot_state": 0.014573450883229573, "get_duckie_state": 2.5169054667154948e-06, "in-drivable-lane": 2.750000000000039, "agent_compute-ego": 0.017314844926198322, "deviation-heading": 1.705856524251432, "complete-iteration": 0.19391764322916663, "set_robot_commands": 0.004590032895406087, "deviation-center-line": 0.35967775588710227, "driven_lanedir_consec": 2.0993503343414757, "sim_compute_sim_state": 0.022975329558054605, "sim_compute_performance-ego": 0.003991282780965169}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.230184805055705, "get_ui_image": 0.03547642230987549, "step_physics": 0.08223007520039877, "survival_time": 14.950000000000076, "driven_lanedir": 1.6874342479168054, "get_state_dump": 0.013545966943105062, "sim_render-ego": 0.006210980415344239, "get_robot_state": 0.015711013476053873, "get_duckie_state": 2.5661786397298178e-06, "in-drivable-lane": 4.0500000000000576, "agent_compute-ego": 0.018099677562713624, "deviation-heading": 2.0228796574899586, "complete-iteration": 0.2041194208463033, "set_robot_commands": 0.004985849857330322, "deviation-center-line": 0.4243749399962627, "driven_lanedir_consec": 1.6874342479168054, "sim_compute_sim_state": 0.02358920494715373, "sim_compute_performance-ego": 0.004119393030802409}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.5567816621993815, "get_ui_image": 0.0363680370648702, "step_physics": 0.08653169473012289, "survival_time": 14.950000000000076, "driven_lanedir": 2.0382744137938698, "get_state_dump": 0.0136674960454305, "sim_render-ego": 0.0062479416529337565, "get_robot_state": 0.01583110253016154, "get_duckie_state": 2.6337305704752605e-06, "in-drivable-lane": 3.0500000000000433, "agent_compute-ego": 0.01741615931193034, "deviation-heading": 1.755449443411393, "complete-iteration": 0.21285916328430177, "set_robot_commands": 0.004878923098246257, "deviation-center-line": 0.4008422674573948, "driven_lanedir_consec": 2.0382744137938698, "sim_compute_sim_state": 0.02763418197631836, "sim_compute_performance-ego": 0.004129940668741862}}
set_robot_commands_max0.004985849857330322
set_robot_commands_mean0.004786809112253562
set_robot_commands_median0.00478567684813892
set_robot_commands_min0.004590032895406087
sim_compute_performance-ego_max0.004129940668741862
sim_compute_performance-ego_mean0.004061121010129845
sim_compute_performance-ego_median0.004061630295406174
sim_compute_performance-ego_min0.003991282780965169
sim_compute_sim_state_max0.02763418197631836
sim_compute_sim_state_mean0.024434025709032303
sim_compute_sim_state_median0.023563295650878127
sim_compute_sim_state_min0.022975329558054605
sim_render-ego_max0.0062479416529337565
sim_render-ego_mean0.006092722260372212
sim_render-ego_median0.006106472499005735
sim_render-ego_min0.00591000239054362
simulation-passed1
step_physics_max0.08653169473012289
step_physics_mean0.08023074387302037
step_physics_median0.07920890527546476
step_physics_min0.07597347021102906
survival_time_max14.950000000000076
survival_time_mean14.725000000000072
survival_time_min14.050000000000065
No reset possible
4087510837Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnomont02-5bd91ed070bc-10:07:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8386315137755154
survival_time_median10.22500000000001
deviation-center-line_median0.28026352189751086
in-drivable-lane_median6.525000000000015


other stats
agent_compute-ego_max0.01762529452641805
agent_compute-ego_mean0.01751346501431364
agent_compute-ego_median0.017516184301863354
agent_compute-ego_min0.017396196927109808
complete-iteration_max0.21126512815785964
complete-iteration_mean0.20445511717845288
complete-iteration_median0.20446142979094253
complete-iteration_min0.19763248097406672
deviation-center-line_max0.4269555154250936
deviation-center-line_mean0.27042634231963836
deviation-center-line_min0.09422281005843808
deviation-heading_max1.5908043340782778
deviation-heading_mean1.1953951625691168
deviation-heading_median1.3548063791675888
deviation-heading_min0.4811635578630124
driven_any_max4.084172319250181
driven_any_mean2.8858884684321637
driven_any_median2.7567785173331725
driven_any_min1.94582451981213
driven_lanedir_consec_max1.2928478886009294
driven_lanedir_consec_mean0.8380695326073487
driven_lanedir_consec_min0.3821672142774348
driven_lanedir_max1.2928478886009294
driven_lanedir_mean0.8380695326073487
driven_lanedir_median0.8386315137755154
driven_lanedir_min0.3821672142774348
get_duckie_state_max2.92247103661606e-06
get_duckie_state_mean2.799773033045212e-06
get_duckie_state_median2.796797789344492e-06
get_duckie_state_min2.683025516875802e-06
get_robot_state_max0.01622240813737063
get_robot_state_mean0.01603340144869046
get_robot_state_median0.01604689938153407
get_robot_state_min0.01581739889432306
get_state_dump_max0.013900208718997916
get_state_dump_mean0.013769813372248077
get_state_dump_median0.013741926869680714
get_state_dump_min0.01369519103063296
get_ui_image_max0.036203357362255606
get_ui_image_mean0.035999059021851326
get_ui_image_median0.03606189041237954
get_ui_image_min0.03566909790039063
in-drivable-lane_max9.650000000000084
in-drivable-lane_mean7.050000000000024
in-drivable-lane_min5.499999999999981
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.603375439435783, "get_ui_image": 0.036203357362255606, "step_physics": 0.08348286151885986, "survival_time": 9.700000000000005, "driven_lanedir": 0.8551023596499601, "get_state_dump": 0.013900208718997916, "sim_render-ego": 0.006507021864664923, "get_robot_state": 0.01622240813737063, "get_duckie_state": 2.92247103661606e-06, "in-drivable-lane": 5.950000000000008, "agent_compute-ego": 0.01757620413278796, "deviation-heading": 1.317340611889572, "complete-iteration": 0.20902593111254503, "set_robot_commands": 0.005061589565473734, "deviation-center-line": 0.25199273164262964, "driven_lanedir_consec": 0.8551023596499601, "sim_compute_sim_state": 0.025582513858362573, "sim_compute_performance-ego": 0.004334003654951902}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 4.084172319250181, "get_ui_image": 0.03566909790039063, "step_physics": 0.0775160829226176, "survival_time": 14.950000000000076, "driven_lanedir": 1.2928478886009294, "get_state_dump": 0.013711199760437012, "sim_render-ego": 0.006108124256134034, "get_robot_state": 0.016011358896891276, "get_duckie_state": 2.773602803548177e-06, "in-drivable-lane": 9.650000000000084, "agent_compute-ego": 0.01762529452641805, "deviation-heading": 1.5908043340782778, "complete-iteration": 0.19989692846934, "set_robot_commands": 0.004905776182810465, "deviation-center-line": 0.4269555154250936, "driven_lanedir_consec": 1.2928478886009294, "sim_compute_sim_state": 0.024100551605224608, "sim_compute_performance-ego": 0.004098278681437174}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 1.94582451981213, "get_ui_image": 0.03616190446566229, "step_physics": 0.07673990236569757, "survival_time": 7.299999999999982, "driven_lanedir": 0.3821672142774348, "get_state_dump": 0.01369519103063296, "sim_render-ego": 0.006210413697647721, "get_robot_state": 0.01581739889432306, "get_duckie_state": 2.683025516875802e-06, "in-drivable-lane": 5.499999999999981, "agent_compute-ego": 0.017396196927109808, "deviation-heading": 0.4811635578630124, "complete-iteration": 0.19763248097406672, "set_robot_commands": 0.004748468529688169, "deviation-center-line": 0.09422281005843808, "driven_lanedir_consec": 0.3821672142774348, "sim_compute_sim_state": 0.02247270982559413, "sim_compute_performance-ego": 0.0042399119024407374}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.9101815952305614, "get_ui_image": 0.03596187635909679, "step_physics": 0.08448923576709837, "survival_time": 10.750000000000018, "driven_lanedir": 0.8221606679010707, "get_state_dump": 0.013772653978924418, "sim_render-ego": 0.006221912073534589, "get_robot_state": 0.01608243986617687, "get_duckie_state": 2.8199927751408068e-06, "in-drivable-lane": 7.100000000000022, "agent_compute-ego": 0.01745616447093875, "deviation-heading": 1.3922721464456054, "complete-iteration": 0.21126512815785964, "set_robot_commands": 0.005003055306368096, "deviation-center-line": 0.3085343121523921, "driven_lanedir_consec": 0.8221606679010707, "sim_compute_sim_state": 0.027889093132906183, "sim_compute_performance-ego": 0.004235384076140648}}
set_robot_commands_max0.005061589565473734
set_robot_commands_mean0.004929722396085116
set_robot_commands_median0.004954415744589281
set_robot_commands_min0.004748468529688169
sim_compute_performance-ego_max0.004334003654951902
sim_compute_performance-ego_mean0.004226894578742615
sim_compute_performance-ego_median0.0042376479892906925
sim_compute_performance-ego_min0.004098278681437174
sim_compute_sim_state_max0.027889093132906183
sim_compute_sim_state_mean0.02501121710552187
sim_compute_sim_state_median0.02484153273179359
sim_compute_sim_state_min0.02247270982559413
sim_render-ego_max0.006507021864664923
sim_render-ego_mean0.006261867972995316
sim_render-ego_median0.006216162885591155
sim_render-ego_min0.006108124256134034
simulation-passed1
step_physics_max0.08448923576709837
step_physics_mean0.08055702064356834
step_physics_median0.08049947222073872
step_physics_min0.07673990236569757
survival_time_max14.950000000000076
survival_time_mean10.67500000000002
survival_time_min7.299999999999982
No reset possible
4086010843Fernanda Custodio Pereira do Carmo 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnomont02-5bd91ed070bc-10:09:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.6594095402928843
survival_time_median14.950000000000076
deviation-center-line_median0.714212753481138
in-drivable-lane_median0.350000000000005


other stats
agent_compute-ego_max0.06124839226404826
agent_compute-ego_mean0.03966636498769125
agent_compute-ego_median0.03990852952003479
agent_compute-ego_min0.017600008646647135
complete-iteration_max0.2467664361000061
complete-iteration_mean0.22854202012221017
complete-iteration_median0.2298268218835195
complete-iteration_min0.2077480006217957
deviation-center-line_max0.8370988520925234
deviation-center-line_mean0.6766422343779775
deviation-center-line_min0.4410445784571109
deviation-heading_max1.6651823623253386
deviation-heading_mean1.112440556276377
deviation-heading_median1.0540724630543623
deviation-heading_min0.6764349366714459
driven_any_max2.7568241917020515
driven_any_mean2.754564618380916
driven_any_median2.7568018022964527
driven_any_min2.747830677228707
driven_lanedir_consec_max2.7352879610438343
driven_lanedir_consec_mean2.395888990012845
driven_lanedir_consec_min1.5294489184217754
driven_lanedir_max2.7352879610438343
driven_lanedir_mean2.395888990012845
driven_lanedir_median2.6594095402928843
driven_lanedir_min1.5294489184217754
get_duckie_state_max2.6750564575195314e-06
get_duckie_state_mean2.5353829065958656e-06
get_duckie_state_median2.5681654612223307e-06
get_duckie_state_min2.330144246419271e-06
get_robot_state_max0.016026314894358316
get_robot_state_mean0.015773213505744933
get_robot_state_median0.015831985076268512
get_robot_state_min0.01540256897608439
get_state_dump_max0.013697694142659506
get_state_dump_mean0.013614198764165242
get_state_dump_median0.013620018561681112
get_state_dump_min0.01351906379063924
get_ui_image_max0.03723953088124593
get_ui_image_mean0.035802491903305055
get_ui_image_median0.035664885441462194
get_ui_image_min0.03464066584904989
in-drivable-lane_max6.450000000000092
in-drivable-lane_mean1.7875000000000254
in-drivable-lane_min0.0
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.747830677228707, "get_ui_image": 0.03635592699050903, "step_physics": 0.08182357470194498, "survival_time": 14.950000000000076, "driven_lanedir": 2.703493556602344, "get_state_dump": 0.01351906379063924, "sim_render-ego": 0.006208349068959554, "get_robot_state": 0.016026314894358316, "get_duckie_state": 2.6750564575195314e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.017600008646647135, "deviation-heading": 1.6651823623253386, "complete-iteration": 0.2077480006217957, "set_robot_commands": 0.005009223620096842, "deviation-center-line": 0.7778701174864772, "driven_lanedir_consec": 2.703493556602344, "sim_compute_sim_state": 0.02674283822377523, "sim_compute_performance-ego": 0.004300273259480794}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.7568150171314456, "get_ui_image": 0.03464066584904989, "step_physics": 0.07818817138671875, "survival_time": 14.950000000000076, "driven_lanedir": 2.7352879610438343, "get_state_dump": 0.013697261810302735, "sim_render-ego": 0.006100827852884928, "get_robot_state": 0.01540256897608439, "get_duckie_state": 2.540747324625651e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.033756717046101885, "deviation-heading": 1.265446875768505, "complete-iteration": 0.21498496135075887, "set_robot_commands": 0.004947168827056885, "deviation-center-line": 0.4410445784571109, "driven_lanedir_consec": 2.7352879610438343, "sim_compute_sim_state": 0.02395730177561442, "sim_compute_performance-ego": 0.004134988784790039}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.7567885874614597, "get_ui_image": 0.03497384389241536, "step_physics": 0.08092139720916748, "survival_time": 14.950000000000076, "driven_lanedir": 1.5294489184217754, "get_state_dump": 0.01354277531305949, "sim_render-ego": 0.006196225484212239, "get_robot_state": 0.015832126140594482, "get_duckie_state": 2.330144246419271e-06, "in-drivable-lane": 6.450000000000092, "agent_compute-ego": 0.06124839226404826, "deviation-heading": 0.6764349366714459, "complete-iteration": 0.2467664361000061, "set_robot_commands": 0.005056179364522298, "deviation-center-line": 0.6505553894757987, "driven_lanedir_consec": 1.5294489184217754, "sim_compute_sim_state": 0.02477228323618571, "sim_compute_performance-ego": 0.0040680885314941405}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.7568241917020515, "get_ui_image": 0.03723953088124593, "step_physics": 0.08768476645151774, "survival_time": 14.950000000000076, "driven_lanedir": 2.615325523983425, "get_state_dump": 0.013697694142659506, "sim_render-ego": 0.006275126934051514, "get_robot_state": 0.015831844011942545, "get_duckie_state": 2.5955835978190104e-06, "in-drivable-lane": 0.70000000000001, "agent_compute-ego": 0.04606034199396769, "deviation-heading": 0.8426980503402198, "complete-iteration": 0.24466868241628012, "set_robot_commands": 0.004989185333251953, "deviation-center-line": 0.8370988520925234, "driven_lanedir_consec": 2.615325523983425, "sim_compute_sim_state": 0.02854290644327799, "sim_compute_performance-ego": 0.004184813499450684}}
set_robot_commands_max0.005056179364522298
set_robot_commands_mean0.0050004392862319945
set_robot_commands_median0.004999204476674398
set_robot_commands_min0.004947168827056885
sim_compute_performance-ego_max0.004300273259480794
sim_compute_performance-ego_mean0.004172041018803915
sim_compute_performance-ego_median0.004159901142120361
sim_compute_performance-ego_min0.0040680885314941405
sim_compute_sim_state_max0.02854290644327799
sim_compute_sim_state_mean0.026003832419713335
sim_compute_sim_state_median0.02575756072998047
sim_compute_sim_state_min0.02395730177561442
sim_render-ego_max0.006275126934051514
sim_render-ego_mean0.006195132335027059
sim_render-ego_median0.006202287276585897
sim_render-ego_min0.006100827852884928
simulation-passed1
step_physics_max0.08768476645151774
step_physics_mean0.08215447743733724
step_physics_median0.08137248595555624
step_physics_min0.07818817138671875
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
4085410846Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailednomont02-5bd91ed070bc-10:04:34
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 263 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4084611275Vincenzo Polizzi 🇮🇹template-randomaido5-LF-sim-validationLFv-simsuccessnomont02-5bd91ed070bc-10:03:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6440373265804349
survival_time_median4.7749999999999915
deviation-center-line_median0.10361109150784235
in-drivable-lane_median2.5499999999999914


other stats
agent_compute-ego_max0.016531163248522527
agent_compute-ego_mean0.016247369760394842
agent_compute-ego_median0.01629271906872478
agent_compute-ego_min0.01587287765560728
complete-iteration_max0.2044227249694593
complete-iteration_mean0.1916565566945248
complete-iteration_median0.1880076909278392
complete-iteration_min0.18618811995296156
deviation-center-line_max0.15583868303837567
deviation-center-line_mean0.10879855071843549
deviation-center-line_min0.07213333681968147
deviation-heading_max0.976837469261332
deviation-heading_mean0.4922932418762878
deviation-heading_median0.3609623398756611
deviation-heading_min0.27041081849249704
driven_any_max2.8894611145876925
driven_any_mean1.8510551780477624
driven_any_median1.998666635086524
driven_any_min0.5174263274303096
driven_lanedir_consec_max0.8904446665936326
driven_lanedir_consec_mean0.6302364342579737
driven_lanedir_consec_min0.3424264172773923
driven_lanedir_max0.8904446665936326
driven_lanedir_mean0.6302364342579737
driven_lanedir_median0.6440373265804349
driven_lanedir_min0.3424264172773923
get_duckie_state_max2.774325284090909e-06
get_duckie_state_mean2.4698745685305924e-06
get_duckie_state_median2.4064112517793302e-06
get_duckie_state_min2.292350486472801e-06
get_robot_state_max0.01487204124187601
get_robot_state_mean0.014320307118766551
get_robot_state_median0.014357842419685338
get_robot_state_min0.013693502393819518
get_state_dump_max0.013673717325383966
get_state_dump_mean0.01346211935801816
get_state_dump_median0.013612153420864376
get_state_dump_min0.012950453264959928
get_ui_image_max0.03477602655237371
get_ui_image_mean0.03401457030445159
get_ui_image_median0.03400039952985781
get_ui_image_min0.03328145560571703
in-drivable-lane_max4.699999999999983
in-drivable-lane_mean2.5374999999999917
in-drivable-lane_min0.3500000000000003
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.1619321797959306, "get_ui_image": 0.03328145560571703, "step_physics": 0.07293402138402906, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6085881804686812, "get_state_dump": 0.013610847925735732, "sim_render-ego": 0.0056142241267834675, "get_robot_state": 0.013693502393819518, "get_duckie_state": 2.428636712543035e-06, "in-drivable-lane": 1.1999999999999966, "agent_compute-ego": 0.01644079159882109, "deviation-heading": 0.27041081849249704, "complete-iteration": 0.18618811995296156, "set_robot_commands": 0.004675768189511056, "deviation-center-line": 0.1246386243322534, "driven_lanedir_consec": 0.6085881804686812, "sim_compute_sim_state": 0.02190991175376763, "sim_compute_performance-ego": 0.0038814787137306343}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.5174263274303096, "get_ui_image": 0.033662672700553106, "step_physics": 0.07385728277009108, "survival_time": 1.4500000000000006, "driven_lanedir": 0.3424264172773923, "get_state_dump": 0.012950453264959928, "sim_render-ego": 0.006052839344945447, "get_robot_state": 0.01487204124187601, "get_duckie_state": 2.384185791015625e-06, "in-drivable-lane": 0.3500000000000003, "agent_compute-ego": 0.016531163248522527, "deviation-heading": 0.3106738984320317, "complete-iteration": 0.1884991711583631, "set_robot_commands": 0.004655648922098094, "deviation-center-line": 0.07213333681968147, "driven_lanedir_consec": 0.3424264172773923, "sim_compute_sim_state": 0.02182032322061473, "sim_compute_performance-ego": 0.003952832057558257}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.8354010903771174, "get_ui_image": 0.03433812635916251, "step_physics": 0.072097615842466, "survival_time": 6.749999999999984, "driven_lanedir": 0.6794864726921884, "get_state_dump": 0.01361345891599302, "sim_render-ego": 0.00587521305790654, "get_robot_state": 0.014611867622092918, "get_duckie_state": 2.292350486472801e-06, "in-drivable-lane": 4.699999999999983, "agent_compute-ego": 0.016144646538628474, "deviation-heading": 0.41125078131929055, "complete-iteration": 0.18751621069731536, "set_robot_commands": 0.004557812655413592, "deviation-center-line": 0.08258355868343131, "driven_lanedir_consec": 0.6794864726921884, "sim_compute_sim_state": 0.02224957324840404, "sim_compute_performance-ego": 0.003888847209789135}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.8894611145876925, "get_ui_image": 0.03477602655237371, "step_physics": 0.08516913471799908, "survival_time": 6.5999999999999845, "driven_lanedir": 0.8904446665936326, "get_state_dump": 0.013673717325383966, "sim_render-ego": 0.005616368669452089, "get_robot_state": 0.014103817217277758, "get_duckie_state": 2.774325284090909e-06, "in-drivable-lane": 3.899999999999986, "agent_compute-ego": 0.01587287765560728, "deviation-heading": 0.976837469261332, "complete-iteration": 0.2044227249694593, "set_robot_commands": 0.004604411847663649, "deviation-center-line": 0.15583868303837567, "driven_lanedir_consec": 0.8904446665936326, "sim_compute_sim_state": 0.026509225368499756, "sim_compute_performance-ego": 0.003949591607758493}}
set_robot_commands_max0.004675768189511056
set_robot_commands_mean0.0046234104036715976
set_robot_commands_median0.004630030384880871
set_robot_commands_min0.004557812655413592
sim_compute_performance-ego_max0.003952832057558257
sim_compute_performance-ego_mean0.00391818739720913
sim_compute_performance-ego_median0.003919219408773814
sim_compute_performance-ego_min0.0038814787137306343
sim_compute_sim_state_max0.026509225368499756
sim_compute_sim_state_mean0.02312225839782154
sim_compute_sim_state_median0.022079742501085835
sim_compute_sim_state_min0.02182032322061473
sim_render-ego_max0.006052839344945447
sim_render-ego_mean0.005789661299771886
sim_render-ego_median0.0057457908636793145
sim_render-ego_min0.0056142241267834675
simulation-passed1
step_physics_max0.08516913471799908
step_physics_mean0.07601451367864631
step_physics_median0.07339565207706007
step_physics_min0.072097615842466
survival_time_max6.749999999999984
survival_time_mean4.437499999999992
survival_time_min1.4500000000000006
No reset possible