Duckietown Challenges Home Challenges Submissions

Evaluator 820

ID820
evaluatoridsc-rudolf-12355
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-12355
versiond-c:4.0.19;d-c-r:4.0.29;d-s:4.0.21
first heard
last heard
statusinactive
# evaluating
# success1 20718
# timeout1 20755
# failed
# error2 20751
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
207552985Artem Ioselevskiichallenge-aido_LF-template-tensorflowaido2-LF-sim-testingstep1-simulationtimeoutnoidsc-rudolf-123551:05:05
I can see how the jo [...]
I can see how the job 20755 is timeout because passed 3905 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
207542986Maxim Scherbakovchallenge-aido_LF-template-tensorflowaido2-LF-sim-testingstep1-simulationerrornoidsc-rudolf-123550:01:02
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 135, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 305, in run_episode
    sim_ci.write_topic_and_expect_zero('set_robot_commands', commands)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "set_robot_commands".
                   |
                   || Traceback (most recent call last):
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "gym_bridge.py", line 299, in on_received_set_robot_commands
                   ||     raise Exception(msg)
                   || Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}.
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 439, in <module>
    wrap(cie)
  File "experiment_manager.py", line 427, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 135, in main
 >     physics_dt=config.physics_dt)
 >   File "experiment_manager.py", line 305, in run_episode
 >     sim_ci.write_topic_and_expect_zero('set_robot_commands', commands)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
 >     others = read_until_over(fpout, timeout=timeout, nickname=nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
 >     raise RemoteNodeAborted(m)
 > zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
 > 
 > error in simulator |Exception while handling a message on topic "set_robot_commands".
 >                    |
 >                    || Traceback (most recent call last):
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
 >                    ||     handle_message_node(parsed, receiver0, context0)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
 >                    ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
 >                    ||     f(**kwargs)
 >                    ||   File "gym_bridge.py", line 299, in on_received_set_robot_commands
 >                    ||     raise Exception(msg)
 >                    || Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}.
 >                    ||
 > 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
207512989Maxim Scherbakovchallenge-aido_LF-template-tensorflowaido2-LF-sim-testingstep1-simulationerrornoidsc-rudolf-123550:01:32
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
207182828Liam Paull 🇨🇦challenge-aido_LF-template-randomaido2-LF-sim-testingstep1-simulationsuccessnoidsc-rudolf-123550:12:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.576224177547854
survival_time_median2.499999999999999
deviation-center-line_median0.10178109518352266
in-drivable-lane_median0.29999999999999893


other stats
agent_compute-ego_max0.15053701400756836
agent_compute-ego_mean0.13152439652106448
agent_compute-ego_median0.12890723286842817
agent_compute-ego_min0.12234530999110296
deviation-center-line_max0.2616907164891491
deviation-center-line_mean0.12846709462781825
deviation-center-line_min0.07606389208452573
deviation-heading_max0.5845520464047074
deviation-heading_mean0.3257509962889007
deviation-heading_median0.34180507322850073
deviation-heading_min0.19231746951364015
driven_any_max1.6077077287168993
driven_any_mean0.8895468525909374
driven_any_median1.00521020931812
driven_any_min0.25817502165929507
driven_lanedir_consec_max1.104102494174326
driven_lanedir_consec_mean0.6438530097285392
driven_lanedir_consec_min0.2479316890295844
driven_lanedir_max1.104102494174326
driven_lanedir_mean0.6438530097285392
driven_lanedir_median0.576224177547854
driven_lanedir_min0.2479316890295844
in-drivable-lane_max1.1499999999999972
in-drivable-lane_mean0.4999999999999985
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.006879208370262, "sim_physics": 0.050525917197173496, "survival_time": 2.6499999999999986, "driven_lanedir": 0.576224177547854, "sim_render-ego": 0.055981748508957194, "in-drivable-lane": 0.9499999999999976, "agent_compute-ego": 0.1275354466348324, "deviation-heading": 0.19231746951364015, "set_robot_commands": 0.07279626828319621, "deviation-center-line": 0.1591461472514269, "driven_lanedir_consec": 0.576224177547854, "sim_compute_sim_state": 0.034578093942606226, "sim_compute_performance-ego": 0.0586474616572542, "sim_compute_robot_state-ego": 0.05851712316836951}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.012120091757381, "sim_physics": 0.05301167414738582, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8136951506166006, "sim_render-ego": 0.05749694200662466, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.13969361782073977, "deviation-heading": 0.5845520464047074, "set_robot_commands": 0.07526899301088773, "deviation-center-line": 0.14883191153694192, "driven_lanedir_consec": 0.8136951506166006, "sim_compute_sim_state": 0.03921342354554396, "sim_compute_performance-ego": 0.0575936299103957, "sim_compute_robot_state-ego": 0.05871912149282602}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.25817502165929507, "sim_physics": 0.049055894215901695, "survival_time": 0.9000000000000002, "driven_lanedir": 0.2479316890295844, "sim_render-ego": 0.06283677948845758, "in-drivable-lane": 0, "agent_compute-ego": 0.15053701400756836, "deviation-heading": 0.2409964306091145, "set_robot_commands": 0.07756949795616998, "deviation-center-line": 0.08655487635315534, "driven_lanedir_consec": 0.2479316890295844, "sim_compute_sim_state": 0.043498224682278104, "sim_compute_performance-ego": 0.07031003634134929, "sim_compute_robot_state-ego": 0.08011878861321343}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.426480003506557, "sim_physics": 0.04565876960754395, "survival_time": 1.2500000000000004, "driven_lanedir": 0.4016627329012578, "sim_render-ego": 0.05881966590881347, "in-drivable-lane": 0, "agent_compute-ego": 0.141564302444458, "deviation-heading": 0.4164332504677331, "set_robot_commands": 0.07648547172546387, "deviation-center-line": 0.082953742710957, "driven_lanedir_consec": 0.4016627329012578, "sim_compute_sim_state": 0.03558939933776856, "sim_compute_performance-ego": 0.05930604934692383, "sim_compute_robot_state-ego": 0.059791545867919925}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.6077077287168993, "sim_physics": 0.04973724560859876, "survival_time": 3.899999999999994, "driven_lanedir": 1.104102494174326, "sim_render-ego": 0.054962097070156, "in-drivable-lane": 1.099999999999996, "agent_compute-ego": 0.1276065630790515, "deviation-heading": 0.2234630787028836, "set_robot_commands": 0.07440215196364965, "deviation-center-line": 0.2616907164891491, "driven_lanedir_consec": 1.104102494174326, "sim_compute_sim_state": 0.0356448430281419, "sim_compute_performance-ego": 0.05851866648747371, "sim_compute_robot_state-ego": 0.060772241690219976}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.00521020931812, "sim_physics": 0.05021999806773906, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7342544018264343, "sim_render-ego": 0.050879609828092615, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.12890723286842817, "deviation-heading": 0.4331502474042864, "set_robot_commands": 0.07388438983839385, "deviation-center-line": 0.08286543269327928, "driven_lanedir_consec": 0.7342544018264343, "sim_compute_sim_state": 0.035400230057385504, "sim_compute_performance-ego": 0.053657736097063334, "sim_compute_robot_state-ego": 0.05837294520164023}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.061139363596407, "sim_physics": 0.051213596548352926, "survival_time": 2.799999999999998, "driven_lanedir": 0.5407802163771538, "sim_render-ego": 0.05162621821675982, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.13089660661561148, "deviation-heading": 0.4173534905141365, "set_robot_commands": 0.07389558213097709, "deviation-center-line": 0.11044564318146607, "driven_lanedir_consec": 0.5407802163771538, "sim_compute_sim_state": 0.03827888199261257, "sim_compute_performance-ego": 0.05605188437870571, "sim_compute_robot_state-ego": 0.05750680821282523}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.31684307844903337, "sim_physics": 0.045315515427362354, "survival_time": 1.0500000000000005, "driven_lanedir": 0.31006397235722083, "sim_render-ego": 0.04891089030674526, "in-drivable-lane": 0, "agent_compute-ego": 0.12567029680524552, "deviation-heading": 0.22073408131453212, "set_robot_commands": 0.08077980223156157, "deviation-center-line": 0.09736553935832194, "driven_lanedir_consec": 0.31006397235722083, "sim_compute_sim_state": 0.03562669526963007, "sim_compute_performance-ego": 0.05561439196268717, "sim_compute_robot_state-ego": 0.05768442153930664}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2235421314565855, "sim_physics": 0.0554810365041097, "survival_time": 2.9999999999999973, "driven_lanedir": 0.7130802655897908, "sim_render-ego": 0.05581874052683512, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.12962918281555175, "deviation-heading": 0.3431054400943528, "set_robot_commands": 0.07521765232086182, "deviation-center-line": 0.07950641387442065, "driven_lanedir_consec": 0.7130802655897908, "sim_compute_sim_state": 0.03794248501459758, "sim_compute_performance-ego": 0.05733166138331095, "sim_compute_robot_state-ego": 0.061589376131693525}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.0237129117269934, "sim_physics": 0.052948090765211314, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5412689097463406, "sim_render-ego": 0.05150529631861934, "in-drivable-lane": 1.0499999999999976, "agent_compute-ego": 0.12357624813362404, "deviation-heading": 0.34180507322850073, "set_robot_commands": 0.07145137257046169, "deviation-center-line": 0.10178109518352266, "driven_lanedir_consec": 0.5412689097463406, "sim_compute_sim_state": 0.0359625948799981, "sim_compute_performance-ego": 0.05725235409206814, "sim_compute_robot_state-ego": 0.0610436686763057}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5963832318462032, "sim_physics": 0.04773323257248123, "survival_time": 3.8499999999999943, "driven_lanedir": 1.0570175213618045, "sim_render-ego": 0.05594062495541263, "in-drivable-lane": 1.149999999999996, "agent_compute-ego": 0.13078964221013056, "deviation-heading": 0.3640854335278077, "set_robot_commands": 0.07404049960049716, "deviation-center-line": 0.22559399326963192, "driven_lanedir_consec": 1.0570175213618045, "sim_compute_sim_state": 0.03628685567286107, "sim_compute_performance-ego": 0.058294819546984386, "sim_compute_robot_state-ego": 0.05817768480870631}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.4674041414069432, "sim_physics": 0.05453698723404496, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4592076134340175, "sim_render-ego": 0.058397010520652486, "in-drivable-lane": 0, "agent_compute-ego": 0.13942726453145346, "deviation-heading": 0.2341111777927785, "set_robot_commands": 0.07426125031930429, "deviation-center-line": 0.09605931838163916, "driven_lanedir_consec": 0.4592076134340175, "sim_compute_sim_state": 0.03730206136350279, "sim_compute_performance-ego": 0.05740827101248282, "sim_compute_robot_state-ego": 0.06206370283056189}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.9935811352973768, "sim_physics": 0.05250790596008301, "survival_time": 2.499999999999999, "driven_lanedir": 0.8323316597041028, "sim_render-ego": 0.05425585746765137, "in-drivable-lane": 0.2499999999999991, "agent_compute-ego": 0.12608039379119873, "deviation-heading": 0.3704409275201552, "set_robot_commands": 0.07137810230255127, "deviation-center-line": 0.12697477591295792, "driven_lanedir_consec": 0.8323316597041028, "sim_compute_sim_state": 0.03707084178924561, "sim_compute_performance-ego": 0.056147098541259766, "sim_compute_robot_state-ego": 0.05964721202850342}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.41227170739271374, "sim_physics": 0.04642183964069073, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4004707868233659, "sim_render-ego": 0.05133636181171124, "in-drivable-lane": 0, "agent_compute-ego": 0.12234530999110296, "deviation-heading": 0.3034330533203094, "set_robot_commands": 0.07372685579153207, "deviation-center-line": 0.07606389208452573, "driven_lanedir_consec": 0.4004707868233659, "sim_compute_sim_state": 0.03374822323138897, "sim_compute_performance-ego": 0.05496470744793232, "sim_compute_robot_state-ego": 0.05775417731358455}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.931752824363289, "sim_physics": 0.05091573794682821, "survival_time": 2.3999999999999995, "driven_lanedir": 0.9257035544382322, "sim_render-ego": 0.05050594607988993, "in-drivable-lane": 0, "agent_compute-ego": 0.12860682606697085, "deviation-heading": 0.20028374391857257, "set_robot_commands": 0.07443730533123016, "deviation-center-line": 0.19117292113587817, "driven_lanedir_consec": 0.9257035544382322, "sim_compute_sim_state": 0.03534020483493805, "sim_compute_performance-ego": 0.055787861347198486, "sim_compute_robot_state-ego": 0.05731812616189321}}
set_robot_commands_max0.08077980223156157
set_robot_commands_mean0.07463967969178258
set_robot_commands_median0.07426125031930429
set_robot_commands_min0.07137810230255127
sim_compute_performance-ego_max0.07031003634134929
sim_compute_performance-ego_mean0.057792441970205985
sim_compute_performance-ego_median0.05733166138331095
sim_compute_performance-ego_min0.053657736097063334
sim_compute_robot_state-ego_max0.08011878861321343
sim_compute_robot_state-ego_mean0.06060512958250464
sim_compute_robot_state-ego_median0.05871912149282602
sim_compute_robot_state-ego_min0.05731812616189321
sim_compute_sim_state_max0.043498224682278104
sim_compute_sim_state_mean0.03676553724283327
sim_compute_sim_state_median0.0359625948799981
sim_compute_sim_state_min0.03374822323138897
sim_physics_max0.0554810365041097
sim_physics_mean0.05035222942956715
sim_physics_median0.050525917197173496
sim_physics_min0.045315515427362354
sim_render-ego_max0.06283677948845758
sim_render-ego_mean0.05461825260102524
sim_render-ego_median0.054962097070156
sim_render-ego_min0.04891089030674526
simulation-passed1
survival_time_max3.899999999999994
survival_time_mean2.313333333333332
survival_time_min0.9000000000000002
No reset possible