Duckietown Challenges Home Challenges Submissions

Evaluator 2261

ID2261
evaluatorreg04-cb6499190cf5-1
ownerI don't have one 😀
machinearchimede
processreg04-cb6499190cf5-1
versiond-c:6.1.23;d-c-r:6.0.59
first heard
last heard
statusinactive
# evaluating
# success5 40839
# timeout
# failed2 40853
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
4090810815Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg04-cb6499190cf5-10:11:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9638564637565858
survival_time_median14.950000000000076
deviation-center-line_median0.5790900801339103
in-drivable-lane_median6.9250000000000655


other stats
agent_compute-ego_max0.027517536481221516
agent_compute-ego_mean0.02673795143763224
agent_compute-ego_median0.026856708923975625
agent_compute-ego_min0.025720851421356203
complete-iteration_max0.2920390876134237
complete-iteration_mean0.2880780496199925
complete-iteration_median0.287855220635732
complete-iteration_min0.2845626695950826
deviation-center-line_max0.8859230920283143
deviation-center-line_mean0.5511209580801452
deviation-center-line_min0.16038058002444577
deviation-heading_max1.5266222049967133
deviation-heading_mean1.363424343904691
deviation-heading_median1.3301222853704615
deviation-heading_min1.2668305998811265
driven_any_max2.0404067101030816
driven_any_mean1.8935905860381397
driven_any_median1.8823593935546623
driven_any_min1.769236846940151
driven_lanedir_consec_max1.6340362063918366
driven_lanedir_consec_mean0.9593764293571831
driven_lanedir_consec_min0.2757565835237248
driven_lanedir_max1.6340362063918366
driven_lanedir_mean0.9593764293571831
driven_lanedir_median0.9638564637565858
driven_lanedir_min0.2757565835237248
get_duckie_state_max3.966490427652995e-06
get_duckie_state_mean3.858606020609538e-06
get_duckie_state_median3.892580668131511e-06
get_duckie_state_min3.682772318522135e-06
get_robot_state_max0.024970816771189373
get_robot_state_mean0.02425820966561635
get_robot_state_median0.02413650631904602
get_robot_state_min0.023789009253184
get_state_dump_max0.020870975653330483
get_state_dump_mean0.020209426283836363
get_state_dump_median0.020111403067906698
get_state_dump_min0.019743923346201574
get_ui_image_max0.05012605905532837
get_ui_image_mean0.048815638422966
get_ui_image_median0.04907860676447551
get_ui_image_min0.04697928110758463
in-drivable-lane_max11.60000000000008
in-drivable-lane_mean7.025000000000063
in-drivable-lane_min2.6500000000000377
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.9364199530800537, "get_ui_image": 0.048851595719655354, "step_physics": 0.11156621138254802, "survival_time": 14.950000000000076, "driven_lanedir": 1.0841456098220386, "get_state_dump": 0.019743923346201574, "sim_render-ego": 0.009656736056009928, "get_robot_state": 0.023789009253184, "get_duckie_state": 3.682772318522135e-06, "in-drivable-lane": 6.050000000000036, "agent_compute-ego": 0.025720851421356203, "deviation-heading": 1.3210648444656796, "complete-iteration": 0.2845626695950826, "set_robot_commands": 0.006955049832661947, "deviation-center-line": 0.8859230920283143, "driven_lanedir_consec": 1.0841456098220386, "sim_compute_sim_state": 0.03156438112258911, "sim_compute_performance-ego": 0.006477777163187663}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 1.8282988340292712, "get_ui_image": 0.05012605905532837, "step_physics": 0.11123275995254515, "survival_time": 14.950000000000076, "driven_lanedir": 0.2757565835237248, "get_state_dump": 0.02027388572692871, "sim_render-ego": 0.010248843828837078, "get_robot_state": 0.024970816771189373, "get_duckie_state": 3.94900639851888e-06, "in-drivable-lane": 11.60000000000008, "agent_compute-ego": 0.027217888832092287, "deviation-heading": 1.2668305998811265, "complete-iteration": 0.291024223168691, "set_robot_commands": 0.007604955037434896, "deviation-center-line": 0.16038058002444577, "driven_lanedir_consec": 0.2757565835237248, "sim_compute_sim_state": 0.0322240948677063, "sim_compute_performance-ego": 0.006875906785329183}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 1.769236846940151, "get_ui_image": 0.049305617809295654, "step_physics": 0.10976276874542236, "survival_time": 14.950000000000076, "driven_lanedir": 0.843567317691133, "get_state_dump": 0.019948920408884685, "sim_render-ego": 0.00983063538869222, "get_robot_state": 0.024102470874786376, "get_duckie_state": 3.966490427652995e-06, "in-drivable-lane": 7.800000000000095, "agent_compute-ego": 0.026495529015858967, "deviation-heading": 1.5266222049967133, "complete-iteration": 0.284686218102773, "set_robot_commands": 0.007520839373270671, "deviation-center-line": 0.3016567048288905, "driven_lanedir_consec": 0.843567317691133, "sim_compute_sim_state": 0.030940210024515788, "sim_compute_performance-ego": 0.006534739335378011}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.0404067101030816, "get_ui_image": 0.04697928110758463, "step_physics": 0.1126816240946452, "survival_time": 14.950000000000076, "driven_lanedir": 1.6340362063918366, "get_state_dump": 0.020870975653330483, "sim_render-ego": 0.010010855992635091, "get_robot_state": 0.024170541763305665, "get_duckie_state": 3.836154937744141e-06, "in-drivable-lane": 2.6500000000000377, "agent_compute-ego": 0.027517536481221516, "deviation-heading": 1.3391797262752438, "complete-iteration": 0.2920390876134237, "set_robot_commands": 0.00768654982248942, "deviation-center-line": 0.8565234554389299, "driven_lanedir_consec": 1.6340362063918366, "sim_compute_sim_state": 0.03495595852533976, "sim_compute_performance-ego": 0.0069067621231079105}}
set_robot_commands_max0.00768654982248942
set_robot_commands_mean0.007441848516464234
set_robot_commands_median0.007562897205352783
set_robot_commands_min0.006955049832661947
sim_compute_performance-ego_max0.0069067621231079105
sim_compute_performance-ego_mean0.006698796351750692
sim_compute_performance-ego_median0.006705323060353597
sim_compute_performance-ego_min0.006477777163187663
sim_compute_sim_state_max0.03495595852533976
sim_compute_sim_state_mean0.03242116113503774
sim_compute_sim_state_median0.031894237995147706
sim_compute_sim_state_min0.030940210024515788
sim_render-ego_max0.010248843828837078
sim_render-ego_mean0.00993676781654358
sim_render-ego_median0.009920745690663656
sim_render-ego_min0.009656736056009928
simulation-passed1
step_physics_max0.1126816240946452
step_physics_mean0.1113108410437902
step_physics_median0.1113994856675466
step_physics_min0.10976276874542236
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
4089310827Daniil Lisussim-exercise-1aido5-LF-sim-validationLFv-simsuccessnoreg04-cb6499190cf5-10:11:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5897014380494225
survival_time_median14.950000000000076
deviation-center-line_median0.5152810033379416
in-drivable-lane_median1.525000000000007


other stats
agent_compute-ego_max0.06468896150588989
agent_compute-ego_mean0.05001998742421468
agent_compute-ego_median0.05482178449630737
agent_compute-ego_min0.025747419198354084
complete-iteration_max0.3300942325592041
complete-iteration_mean0.3127728164196014
complete-iteration_median0.31472773710886637
complete-iteration_min0.29154155890146893
deviation-center-line_max0.6962107907421589
deviation-center-line_mean0.47695260491505587
deviation-center-line_min0.18103762224218156
deviation-heading_max5.996471754793223
deviation-heading_mean4.049245555924051
deviation-heading_median4.6619110483213575
deviation-heading_min0.8766883722602664
driven_any_max1.956346585462504
driven_any_mean1.3702285856248075
driven_any_median1.294553257560679
driven_any_min0.9354612419153674
driven_lanedir_consec_max1.9515927474102224
driven_lanedir_consec_mean0.875919469035676
driven_lanedir_consec_min0.37268225263363686
driven_lanedir_max1.9515927474102224
driven_lanedir_mean1.0992040479878862
driven_lanedir_median0.898228778681053
driven_lanedir_min0.6487658871792169
get_duckie_state_max4.2883555094401045e-06
get_duckie_state_mean4.066030184427897e-06
get_duckie_state_median4.03443972269694e-06
get_duckie_state_min3.906885782877604e-06
get_robot_state_max0.0251069704691569
get_robot_state_mean0.024112237890561425
get_robot_state_median0.024132316907246907
get_robot_state_min0.02307734727859497
get_state_dump_max0.020013856887817386
get_state_dump_mean0.019662915666898092
get_state_dump_median0.01960590004920959
get_state_dump_min0.01942600568135579
get_ui_image_max0.05080204725265503
get_ui_image_mean0.049806564450263976
get_ui_image_median0.0497875185807546
get_ui_image_min0.04884917338689168
in-drivable-lane_max4.800000000000068
in-drivable-lane_mean1.9625000000000208
in-drivable-lane_min0.0
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.1608617466845592, "get_ui_image": 0.05080204725265503, "step_physics": 0.11810576597849529, "survival_time": 14.950000000000076, "driven_lanedir": 1.010466021055238, "get_state_dump": 0.01942600568135579, "sim_render-ego": 0.009679742654164632, "get_robot_state": 0.02387946844100952, "get_duckie_state": 4.2883555094401045e-06, "in-drivable-lane": 1.6499999999999997, "agent_compute-ego": 0.025747419198354084, "deviation-heading": 5.257979550378842, "complete-iteration": 0.29154155890146893, "set_robot_commands": 0.007313897609710693, "deviation-center-line": 0.6113472046566395, "driven_lanedir_consec": 0.5306369889196283, "sim_compute_sim_state": 0.02994063297907511, "sim_compute_performance-ego": 0.006433432102203369}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 0.9354612419153674, "get_ui_image": 0.04884917338689168, "step_physics": 0.11351107279459637, "survival_time": 14.950000000000076, "driven_lanedir": 0.7859915363068679, "get_state_dump": 0.019709653854370117, "sim_render-ego": 0.01014867385228475, "get_robot_state": 0.0251069704691569, "get_duckie_state": 4.119078318277995e-06, "in-drivable-lane": 1.4000000000000146, "agent_compute-ego": 0.047999069690704346, "deviation-heading": 5.996471754793223, "complete-iteration": 0.30869803349177044, "set_robot_commands": 0.0078494397799174, "deviation-center-line": 0.6962107907421589, "driven_lanedir_consec": 0.37268225263363686, "sim_compute_sim_state": 0.02850599606831869, "sim_compute_performance-ego": 0.006790975729624431}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 1.4282447684367992, "get_ui_image": 0.05072005192438762, "step_physics": 0.1099572491645813, "survival_time": 14.950000000000076, "driven_lanedir": 0.6487658871792169, "get_state_dump": 0.01950214624404907, "sim_render-ego": 0.009913592338562012, "get_robot_state": 0.024385165373484292, "get_duckie_state": 3.949801127115885e-06, "in-drivable-lane": 4.800000000000068, "agent_compute-ego": 0.0616444993019104, "deviation-heading": 4.065842546263872, "complete-iteration": 0.3207574407259623, "set_robot_commands": 0.007600229581197103, "deviation-center-line": 0.4192148020192436, "driven_lanedir_consec": 0.6487658871792169, "sim_compute_sim_state": 0.030260136127471925, "sim_compute_performance-ego": 0.006547728379567464}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 1.956346585462504, "get_ui_image": 0.04885498523712158, "step_physics": 0.11515825827916464, "survival_time": 14.950000000000076, "driven_lanedir": 1.9515927474102224, "get_state_dump": 0.020013856887817386, "sim_render-ego": 0.009470340410868326, "get_robot_state": 0.02307734727859497, "get_duckie_state": 3.906885782877604e-06, "in-drivable-lane": 0.0, "agent_compute-ego": 0.06468896150588989, "deviation-heading": 0.8766883722602664, "complete-iteration": 0.3300942325592041, "set_robot_commands": 0.007340595722198486, "deviation-center-line": 0.18103762224218156, "driven_lanedir_consec": 1.9515927474102224, "sim_compute_sim_state": 0.03481105168660482, "sim_compute_performance-ego": 0.00644463857014974}}
set_robot_commands_max0.0078494397799174
set_robot_commands_mean0.007526040673255921
set_robot_commands_median0.007470412651697794
set_robot_commands_min0.007313897609710693
sim_compute_performance-ego_max0.006790975729624431
sim_compute_performance-ego_mean0.006554193695386251
sim_compute_performance-ego_median0.006496183474858602
sim_compute_performance-ego_min0.006433432102203369
sim_compute_sim_state_max0.03481105168660482
sim_compute_sim_state_mean0.03087945421536763
sim_compute_sim_state_median0.03010038455327352
sim_compute_sim_state_min0.02850599606831869
sim_render-ego_max0.01014867385228475
sim_render-ego_mean0.00980308731396993
sim_render-ego_median0.009796667496363322
sim_render-ego_min0.009470340410868326
simulation-passed1
step_physics_max0.11810576597849529
step_physics_mean0.1141830865542094
step_physics_median0.11433466553688049
step_physics_min0.1099572491645813
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
4088811280Moustafa Elarabitemplate-pytorchaido5-LF-sim-validationLFv-simfailednoreg04-cb6499190cf5-10:01:09
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 327, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego" aborted:

error in ego |Exception while handling a message on topic "get_commands".
             |
             || Traceback (most recent call last):
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 331, in loop
             ||     handle_message_node(parsed, receiver0, context0)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 486, in handle_message_node
             ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
             ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
             ||     f(**kwargs)
             ||   File "solution.py", line 71, in on_received_get_commands
             ||     velocity, omega = self.compute_action(self.current_image)
             ||   File "solution.py", line 67, in compute_action
             ||     action = self.model.predict(observation)
             ||   File "/workspace/model.py", line 95, in predict
             ||     omega = output[:, 1].unsqueeze(1) * self.max_steering * self.gain
             ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 778, in __getattr__
             ||     raise ModuleAttributeError("'{}' object has no attribute '{}'".format(
             || torch.nn.modules.module.ModuleAttributeError: 'Squeezenet' object has no attribute 'gain'
             ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4087710837Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg04-cb6499190cf5-10:08:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8223572079573374
survival_time_median10.85000000000002
deviation-center-line_median0.2565984965267313
in-drivable-lane_median6.350000000000025


other stats
agent_compute-ego_max0.027778975908146347
agent_compute-ego_mean0.026752708204587087
agent_compute-ego_median0.026457139491016704
agent_compute-ego_min0.02631757792816859
complete-iteration_max0.29329930701756585
complete-iteration_mean0.28594350859052226
complete-iteration_median0.2881619117548351
complete-iteration_min0.2741509038348531
deviation-center-line_max0.49796669130513815
deviation-center-line_mean0.2756459683652126
deviation-center-line_min0.09142018910224964
deviation-heading_max1.6547637733969431
deviation-heading_mean1.2113059362833791
deviation-heading_median1.3474859502010297
deviation-heading_min0.4954880713345134
driven_any_max2.99402605871003
driven_any_mean2.8125109835526048
driven_any_median2.938131820811434
driven_any_min2.379754233877521
driven_lanedir_consec_max1.4866663543783722
driven_lanedir_consec_mean0.867213893055514
driven_lanedir_consec_min0.3374748019290089
driven_lanedir_max1.4866663543783722
driven_lanedir_mean0.867213893055514
driven_lanedir_median0.8223572079573374
driven_lanedir_min0.3374748019290089
get_duckie_state_max3.922262857126635e-06
get_duckie_state_mean3.766077624131763e-06
get_duckie_state_median3.756453457488317e-06
get_duckie_state_min3.629140724423784e-06
get_robot_state_max0.024030280916878345
get_robot_state_mean0.023238288952544105
get_robot_state_median0.02343278226139314
get_robot_state_min0.022057310370511788
get_state_dump_max0.021300372412038405
get_state_dump_mean0.020235404321265076
get_state_dump_median0.019958544396677276
get_state_dump_min0.01972415607966734
get_ui_image_max0.049627360333217664
get_ui_image_mean0.048398858319589934
get_ui_image_median0.04865083129384448
get_ui_image_min0.0466664103574531
in-drivable-lane_max9.100000000000016
in-drivable-lane_mean6.687500000000016
in-drivable-lane_min4.9499999999999975
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.379754233877521, "get_ui_image": 0.049627360333217664, "step_physics": 0.11326474286197276, "survival_time": 8.899999999999991, "driven_lanedir": 0.916128822664507, "get_state_dump": 0.01972415607966734, "sim_render-ego": 0.009575979093487345, "get_robot_state": 0.024030280916878345, "get_duckie_state": 3.814697265625e-06, "in-drivable-lane": 4.9499999999999975, "agent_compute-ego": 0.026553605379683248, "deviation-heading": 1.3319361831608645, "complete-iteration": 0.2881831198595883, "set_robot_commands": 0.007476517323697551, "deviation-center-line": 0.26796754645916127, "driven_lanedir_consec": 0.916128822664507, "sim_compute_sim_state": 0.031219209178110188, "sim_compute_performance-ego": 0.006489598349239049}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.99402605871003, "get_ui_image": 0.04870393049663008, "step_physics": 0.10830265148732456, "survival_time": 11.050000000000022, "driven_lanedir": 1.4866663543783722, "get_state_dump": 0.01975897102873789, "sim_render-ego": 0.009523461846744312, "get_robot_state": 0.02379488729243904, "get_duckie_state": 3.629140724423784e-06, "in-drivable-lane": 5.050000000000028, "agent_compute-ego": 0.026360673602350157, "deviation-heading": 1.6547637733969431, "complete-iteration": 0.2881407036500819, "set_robot_commands": 0.0076227835400611565, "deviation-center-line": 0.49796669130513815, "driven_lanedir_consec": 1.4866663543783722, "sim_compute_sim_state": 0.037044001920190875, "sim_compute_performance-ego": 0.006808611062856821}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.910178721458692, "get_ui_image": 0.0466664103574531, "step_physics": 0.1031757022059241, "survival_time": 10.750000000000018, "driven_lanedir": 0.3374748019290089, "get_state_dump": 0.021300372412038405, "sim_render-ego": 0.009092958583388218, "get_robot_state": 0.022057310370511788, "get_duckie_state": 3.922262857126635e-06, "in-drivable-lane": 9.100000000000016, "agent_compute-ego": 0.027778975908146347, "deviation-heading": 0.4954880713345134, "complete-iteration": 0.2741509038348531, "set_robot_commands": 0.006659884785496911, "deviation-center-line": 0.09142018910224964, "driven_lanedir_consec": 0.3374748019290089, "sim_compute_sim_state": 0.030310370201288267, "sim_compute_performance-ego": 0.006875485043193018}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.966084920164177, "get_ui_image": 0.04859773209105888, "step_physics": 0.11566070987753672, "survival_time": 10.95000000000002, "driven_lanedir": 0.7285855932501678, "get_state_dump": 0.020158117764616665, "sim_render-ego": 0.009264771796796963, "get_robot_state": 0.023070677230347236, "get_duckie_state": 3.698209649351634e-06, "in-drivable-lane": 7.650000000000023, "agent_compute-ego": 0.02631757792816859, "deviation-heading": 1.363035717241195, "complete-iteration": 0.29329930701756585, "set_robot_commands": 0.007138065007179295, "deviation-center-line": 0.2452294465943013, "driven_lanedir_consec": 0.7285855932501678, "sim_compute_sim_state": 0.03667186166597828, "sim_compute_performance-ego": 0.006198742618299511}}
set_robot_commands_max0.0076227835400611565
set_robot_commands_mean0.007224312664108728
set_robot_commands_median0.007307291165438423
set_robot_commands_min0.006659884785496911
sim_compute_performance-ego_max0.006875485043193018
sim_compute_performance-ego_mean0.0065931092683970995
sim_compute_performance-ego_median0.006649104706047935
sim_compute_performance-ego_min0.006198742618299511
sim_compute_sim_state_max0.037044001920190875
sim_compute_sim_state_mean0.033811360741391905
sim_compute_sim_state_median0.03394553542204423
sim_compute_sim_state_min0.030310370201288267
sim_render-ego_max0.009575979093487345
sim_render-ego_mean0.009364292830104207
sim_render-ego_median0.00939411682177064
sim_render-ego_min0.009092958583388218
simulation-passed1
step_physics_max0.11566070987753672
step_physics_mean0.11010095160818954
step_physics_median0.11078369717464868
step_physics_min0.1031757022059241
survival_time_max11.050000000000022
survival_time_mean10.412500000000012
survival_time_min8.899999999999991
No reset possible
4086110842Yishu Malhotra 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg04-cb6499190cf5-10:10:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.6510585203671315
survival_time_median14.950000000000076
deviation-center-line_median0.31737290024547554
in-drivable-lane_median3.0500000000000433


other stats
agent_compute-ego_max0.02687471866607666
agent_compute-ego_mean0.0256239932088808
agent_compute-ego_median0.025614040952673705
agent_compute-ego_min0.02439317226409912
complete-iteration_max0.2956333653132121
complete-iteration_mean0.28478268655763794
complete-iteration_median0.2828328425214038
complete-iteration_min0.27783169587453205
deviation-center-line_max0.45535622407282234
deviation-center-line_mean0.34132225473252253
deviation-center-line_min0.27518699436631683
deviation-heading_max1.49546521841615
deviation-heading_mean1.3280540735849409
deviation-heading_median1.390665425891186
deviation-heading_min1.035420224141242
driven_any_max2.3318709408541225
driven_any_mean1.9842300168925169
driven_any_median1.999748743087831
driven_any_min1.6055516405402832
driven_lanedir_consec_max1.984653949551055
driven_lanedir_consec_mean1.4905102676561086
driven_lanedir_consec_min0.6752700803391163
driven_lanedir_max1.984653949551055
driven_lanedir_mean1.4905102676561086
driven_lanedir_median1.6510585203671315
driven_lanedir_min0.6752700803391163
get_duckie_state_max3.566741943359375e-06
get_duckie_state_mean3.471092511248845e-06
get_duckie_state_median3.482731439734018e-06
get_duckie_state_min3.352165222167969e-06
get_robot_state_max0.024258159001668295
get_robot_state_mean0.02358189250558569
get_robot_state_median0.02364247054914542
get_robot_state_min0.022784469922383623
get_state_dump_max0.020334437688191733
get_state_dump_mean0.0194313979982048
get_state_dump_median0.01936815477186634
get_state_dump_min0.018654844760894775
get_ui_image_max0.050783971150716145
get_ui_image_mean0.049179587531749
get_ui_image_median0.04870233370594898
get_ui_image_min0.04852971156438192
in-drivable-lane_max9.700000000000088
in-drivable-lane_mean4.437500000000051
in-drivable-lane_min1.9500000000000275
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.6055516405402832, "get_ui_image": 0.048857849314465504, "step_physics": 0.1086234497035154, "survival_time": 10.85000000000002, "driven_lanedir": 1.3307024331869766, "get_state_dump": 0.019282879367951426, "sim_render-ego": 0.009454893077024115, "get_robot_state": 0.023339367132582424, "get_duckie_state": 3.5147513112714214e-06, "in-drivable-lane": 1.9500000000000275, "agent_compute-ego": 0.024821817600232665, "deviation-heading": 1.49546521841615, "complete-iteration": 0.2783709682077856, "set_robot_commands": 0.0072750506862517324, "deviation-center-line": 0.31234228935669395, "driven_lanedir_consec": 1.3307024331869766, "sim_compute_sim_state": 0.030525218506562545, "sim_compute_performance-ego": 0.0059913797861969416}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.2437441293126414, "get_ui_image": 0.04852971156438192, "step_physics": 0.10607162872950236, "survival_time": 14.950000000000076, "driven_lanedir": 1.984653949551055, "get_state_dump": 0.01945343017578125, "sim_render-ego": 0.009481579462687174, "get_robot_state": 0.023945573965708417, "get_duckie_state": 3.4507115681966144e-06, "in-drivable-lane": 2.6500000000000377, "agent_compute-ego": 0.026406264305114745, "deviation-heading": 1.463693717977684, "complete-iteration": 0.27783169587453205, "set_robot_commands": 0.007379531065622965, "deviation-center-line": 0.45535622407282234, "driven_lanedir_consec": 1.984653949551055, "sim_compute_sim_state": 0.029952290852864584, "sim_compute_performance-ego": 0.006405844688415528}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 1.75575335686302, "get_ui_image": 0.048546818097432456, "step_physics": 0.11238231976826986, "survival_time": 14.950000000000076, "driven_lanedir": 0.6752700803391163, "get_state_dump": 0.020334437688191733, "sim_render-ego": 0.009728394349416096, "get_robot_state": 0.024258159001668295, "get_duckie_state": 3.566741943359375e-06, "in-drivable-lane": 9.700000000000088, "agent_compute-ego": 0.02687471866607666, "deviation-heading": 1.3176371338046882, "complete-iteration": 0.28729471683502195, "set_robot_commands": 0.007318867842356364, "deviation-center-line": 0.27518699436631683, "driven_lanedir_consec": 0.6752700803391163, "sim_compute_sim_state": 0.030168045361836753, "sim_compute_performance-ego": 0.0074680900573730466}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.3318709408541225, "get_ui_image": 0.050783971150716145, "step_physics": 0.12195271492004396, "survival_time": 14.950000000000076, "driven_lanedir": 1.9714146075472867, "get_state_dump": 0.018654844760894775, "sim_render-ego": 0.00906409740447998, "get_robot_state": 0.022784469922383623, "get_duckie_state": 3.352165222167969e-06, "in-drivable-lane": 3.450000000000049, "agent_compute-ego": 0.02439317226409912, "deviation-heading": 1.035420224141242, "complete-iteration": 0.2956333653132121, "set_robot_commands": 0.006833051840464274, "deviation-center-line": 0.32240351113425714, "driven_lanedir_consec": 1.9714146075472867, "sim_compute_sim_state": 0.035034115314483645, "sim_compute_performance-ego": 0.0059421491622924805}}
set_robot_commands_max0.007379531065622965
set_robot_commands_mean0.007201625358673834
set_robot_commands_median0.007296959264304048
set_robot_commands_min0.006833051840464274
sim_compute_performance-ego_max0.0074680900573730466
sim_compute_performance-ego_mean0.0064518659235695
sim_compute_performance-ego_median0.006198612237306235
sim_compute_performance-ego_min0.0059421491622924805
sim_compute_sim_state_max0.035034115314483645
sim_compute_sim_state_mean0.03141991750893688
sim_compute_sim_state_median0.030346631934199647
sim_compute_sim_state_min0.029952290852864584
sim_render-ego_max0.009728394349416096
sim_render-ego_mean0.00943224107340184
sim_render-ego_median0.009468236269855646
sim_render-ego_min0.00906409740447998
simulation-passed1
step_physics_max0.12195271492004396
step_physics_mean0.11225752828033288
step_physics_median0.11050288473589262
step_physics_min0.10607162872950236
survival_time_max14.950000000000076
survival_time_mean13.92500000000006
survival_time_min10.85000000000002
No reset possible
4085310846Ayman Shams 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simfailednoreg04-cb6499190cf5-10:05:15
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 186 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4083910854Dishank Bansal 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg04-cb6499190cf5-10:11:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.016189449518156
survival_time_median14.950000000000076
deviation-center-line_median0.5308484188052294
in-drivable-lane_median2.7250000000000387


other stats
agent_compute-ego_max0.026464346249898275
agent_compute-ego_mean0.025916640162467953
agent_compute-ego_median0.02597287893295288
agent_compute-ego_min0.02525645653406779
complete-iteration_max0.2993050694465637
complete-iteration_mean0.2851833921670914
complete-iteration_median0.2810683921972911
complete-iteration_min0.27929171482721965
deviation-center-line_max0.6234585546740656
deviation-center-line_mean0.5229686650109799
deviation-center-line_min0.40671926775939554
deviation-heading_max2.600424406248711
deviation-heading_mean2.1274473485908993
deviation-heading_median2.073859867797127
deviation-heading_min1.7616452525206314
driven_any_max2.552596538760124
driven_any_mean2.550488261426705
driven_any_median2.552547178449971
driven_any_min2.5442621500467544
driven_lanedir_consec_max2.181444270807022
driven_lanedir_consec_mean1.9660962771952009
driven_lanedir_consec_min1.6505619389374688
driven_lanedir_max2.181444270807022
driven_lanedir_mean1.9660962771952009
driven_lanedir_median2.016189449518156
driven_lanedir_min1.6505619389374688
get_duckie_state_max3.685156504313151e-06
get_duckie_state_mean3.517071406046549e-06
get_duckie_state_median3.565549850463867e-06
get_duckie_state_min3.2520294189453126e-06
get_robot_state_max0.025071214040120443
get_robot_state_mean0.023546582857767746
get_robot_state_median0.023236831029256184
get_robot_state_min0.02264145533243815
get_state_dump_max0.020005388259887697
get_state_dump_mean0.019331572850545247
get_state_dump_median0.019257036844889323
get_state_dump_min0.018806829452514648
get_ui_image_max0.05247777541478475
get_ui_image_mean0.049529397288958235
get_ui_image_median0.04909998218218485
get_ui_image_min0.047439849376678465
in-drivable-lane_max4.850000000000069
in-drivable-lane_mean3.037500000000043
in-drivable-lane_min1.8500000000000263
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 2.5442621500467544, "get_ui_image": 0.04887131134668986, "step_physics": 0.10874218543370565, "survival_time": 14.950000000000076, "driven_lanedir": 1.6505619389374688, "get_state_dump": 0.019327576955159503, "sim_render-ego": 0.009332151412963868, "get_robot_state": 0.02323685884475708, "get_duckie_state": 3.674825032552083e-06, "in-drivable-lane": 4.850000000000069, "agent_compute-ego": 0.025566646258036296, "deviation-heading": 1.8378361578657567, "complete-iteration": 0.27929171482721965, "set_robot_commands": 0.006980963548024495, "deviation-center-line": 0.40671926775939554, "driven_lanedir_consec": 1.6505619389374688, "sim_compute_sim_state": 0.030970449447631835, "sim_compute_performance-ego": 0.006052698294321696}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 2.5525829788707943, "get_ui_image": 0.04932865301767985, "step_physics": 0.11283432245254515, "survival_time": 14.950000000000076, "driven_lanedir": 2.181444270807022, "get_state_dump": 0.018806829452514648, "sim_render-ego": 0.008960283597310385, "get_robot_state": 0.02264145533243815, "get_duckie_state": 3.2520294189453126e-06, "in-drivable-lane": 1.8500000000000263, "agent_compute-ego": 0.02525645653406779, "deviation-heading": 2.3098835777284976, "complete-iteration": 0.28075103839238486, "set_robot_commands": 0.00696569283803304, "deviation-center-line": 0.6234585546740656, "driven_lanedir_consec": 2.181444270807022, "sim_compute_sim_state": 0.02974982182184855, "sim_compute_performance-ego": 0.006001018683115642}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 2.5525113780291475, "get_ui_image": 0.047439849376678465, "step_physics": 0.11208733320236204, "survival_time": 14.950000000000076, "driven_lanedir": 1.894868524210931, "get_state_dump": 0.01918649673461914, "sim_render-ego": 0.009385228157043455, "get_robot_state": 0.023236803213755288, "get_duckie_state": 3.456274668375651e-06, "in-drivable-lane": 3.300000000000047, "agent_compute-ego": 0.026379111607869467, "deviation-heading": 2.600424406248711, "complete-iteration": 0.2813857460021973, "set_robot_commands": 0.007260259787241618, "deviation-center-line": 0.5336721670891911, "driven_lanedir_consec": 1.894868524210931, "sim_compute_sim_state": 0.03011905034383138, "sim_compute_performance-ego": 0.006080175240834554}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 2.552596538760124, "get_ui_image": 0.05247777541478475, "step_physics": 0.115440936088562, "survival_time": 14.950000000000076, "driven_lanedir": 2.1375103748253803, "get_state_dump": 0.020005388259887697, "sim_render-ego": 0.009984771410624186, "get_robot_state": 0.025071214040120443, "get_duckie_state": 3.685156504313151e-06, "in-drivable-lane": 2.1500000000000306, "agent_compute-ego": 0.026464346249898275, "deviation-heading": 1.7616452525206314, "complete-iteration": 0.2993050694465637, "set_robot_commands": 0.007862209479014079, "deviation-center-line": 0.5280246705212676, "driven_lanedir_consec": 2.1375103748253803, "sim_compute_sim_state": 0.035295608043670657, "sim_compute_performance-ego": 0.006483179728190104}}
set_robot_commands_max0.007862209479014079
set_robot_commands_mean0.007267281413078308
set_robot_commands_median0.007120611667633057
set_robot_commands_min0.00696569283803304
sim_compute_performance-ego_max0.006483179728190104
sim_compute_performance-ego_mean0.006154267986615499
sim_compute_performance-ego_median0.006066436767578125
sim_compute_performance-ego_min0.006001018683115642
sim_compute_sim_state_max0.035295608043670657
sim_compute_sim_state_mean0.031533732414245605
sim_compute_sim_state_median0.03054474989573161
sim_compute_sim_state_min0.02974982182184855
sim_render-ego_max0.009984771410624186
sim_render-ego_mean0.009415608644485473
sim_render-ego_median0.009358689785003664
sim_render-ego_min0.008960283597310385
simulation-passed1
step_physics_max0.115440936088562
step_physics_mean0.11227619429429372
step_physics_median0.11246082782745362
step_physics_min0.10874218543370565
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible