Duckietown Challenges Home Challenges Submissions

Evaluator 784

ID784
evaluatorip-172-31-25-98-9205
ownerI don't have one 😀
machineip-172-31-25-98
processip-172-31-25-98-9205
versiond-c:4.0.18;d-c-r:4.0.28;d-s:4.0.18
first heard
last heard
statusinactive
# evaluating
# success8 19240
# timeout
# failed
# error11 19540
# aborted
# host-error3 19301
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
197033015Manfred Diazchallenge-aido_LF-template-pytorchaido2-LFVI-sim-validationstep1-simulationerrornoip-172-31-25-98-92050:01:30
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
196892999Manfred Diazchallenge-aido_LF-template-tensorflowaido2-LFV-sim-validationstep1-simulationerrornoip-172-31-25-98-92050:02:47
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
196832988Maxim Scherbakovchallenge-aido_LF-template-tensorflowaido2-LF-sim-testingstep1-simulationerrornoip-172-31-25-98-92050:00:27
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
196812986Maxim Scherbakovchallenge-aido_LF-template-tensorflowaido2-LF-sim-testingstep1-simulationerrornoip-172-31-25-98-92050:02:33
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    sim_ci.write_topic_and_expect_zero('set_robot_commands', commands)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "set_robot_commands".
                   |
                   || Traceback (most recent call last):
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "gym_bridge.py", line 261, in on_received_set_robot_commands
                   ||     raise Exception(msg)
                   || Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}.
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 435, in <module>
    wrap(cie)
  File "experiment_manager.py", line 423, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 136, in main
 >     physics_dt=config.physics_dt)
 >   File "experiment_manager.py", line 301, in run_episode
 >     sim_ci.write_topic_and_expect_zero('set_robot_commands', commands)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
 >     others = read_until_over(fpout, timeout=timeout, nickname=nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
 >     raise RemoteNodeAborted(m)
 > zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
 > 
 > error in simulator |Exception while handling a message on topic "set_robot_commands".
 >                    |
 >                    || Traceback (most recent call last):
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
 >                    ||     handle_message_node(parsed, receiver0, context0)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
 >                    ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
 >                    ||     f(**kwargs)
 >                    ||   File "gym_bridge.py", line 261, in on_received_set_robot_commands
 >                    ||     raise Exception(msg)
 >                    || Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}.
 >                    ||
 >
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
196802985Artem Ioselevskiichallenge-aido_LF-template-tensorflowaido2-LF-sim-testingstep1-simulationerrornoip-172-31-25-98-92050:02:47
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 301, in run_episode
    sim_ci.write_topic_and_expect_zero('set_robot_commands', commands)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "set_robot_commands".
                   |
                   || Traceback (most recent call last):
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "gym_bridge.py", line 261, in on_received_set_robot_commands
                   ||     raise Exception(msg)
                   || Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}.
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 435, in <module>
    wrap(cie)
  File "experiment_manager.py", line 423, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 136, in main
 >     physics_dt=config.physics_dt)
 >   File "experiment_manager.py", line 301, in run_episode
 >     sim_ci.write_topic_and_expect_zero('set_robot_commands', commands)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply
 >     others = read_until_over(fpout, timeout=timeout, nickname=nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over
 >     raise RemoteNodeAborted(m)
 > zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:
 > 
 > error in simulator |Exception while handling a message on topic "set_robot_commands".
 >                    |
 >                    || Traceback (most recent call last):
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop
 >                    ||     handle_message_node(parsed, receiver0, context0)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node
 >                    ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
 >                    ||   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists
 >                    ||     f(**kwargs)
 >                    ||   File "gym_bridge.py", line 261, in on_received_set_robot_commands
 >                    ||     raise Exception(msg)
 >                    || Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}.
 >                    ||
 >
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
196772362Andrea Censi 🇨🇭random_agentaido2-LF-sim-testingstep1-simulationsuccessnoip-172-31-25-98-92050:05:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.909759304183658
survival_time_median2.3999999999999995
deviation-center-line_median0.0985043663838993
in-drivable-lane_median0.3499999999999992


other stats
agent_compute-ego_max0.14101568707879983
agent_compute-ego_mean0.11375010244143964
agent_compute-ego_median0.1059987229459426
agent_compute-ego_min0.10480176860635931
deviation-center-line_max0.13882443387861454
deviation-center-line_mean0.10001813696907735
deviation-center-line_min0.06219077976537568
deviation-heading_max0.26115230540596424
deviation-heading_mean0.2318832073723368
deviation-heading_median0.23893895947579852
deviation-heading_min0.1984807025035912
driven_any_max1.4367438864906803
driven_any_mean1.030907091561231
driven_any_median0.9157141892899948
driven_any_min0.8462726558162896
driven_lanedir_consec_max0.9542197633771314
driven_lanedir_consec_mean0.795970343519684
driven_lanedir_consec_min0.5348503251906886
driven_lanedir_max0.9542197633771314
driven_lanedir_mean0.795970343519684
driven_lanedir_median0.909759304183658
driven_lanedir_min0.5348503251906886
in-drivable-lane_max1.099999999999998
in-drivable-lane_mean0.4899999999999987
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0405624261079018, "sim_physics": 0.028039531887702224, "survival_time": 2.6499999999999986, "driven_lanedir": 0.5348503251906886, "sim_render-ego": 0.06632853903860417, "in-drivable-lane": 1.099999999999998, "agent_compute-ego": 0.14101568707879983, "deviation-heading": 0.24579559783100224, "set_robot_commands": 0.10098746137798956, "deviation-center-line": 0.07952733817751276, "driven_lanedir_consec": 0.5348503251906886, "sim_compute_sim_state": 0.04057747013164016, "sim_compute_performance-ego": 0.07194779504020259, "sim_compute_robot_state-ego": 0.0751253254008743}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.8462726558162896, "sim_physics": 0.01641160249710083, "survival_time": 2.2, "driven_lanedir": 0.6707200449851557, "sim_render-ego": 0.046652474186637184, "in-drivable-lane": 0.3499999999999992, "agent_compute-ego": 0.10480176860635931, "deviation-heading": 0.23893895947579852, "set_robot_commands": 0.07024174386804755, "deviation-center-line": 0.06219077976537568, "driven_lanedir_consec": 0.6707200449851557, "sim_compute_sim_state": 0.02913047508759932, "sim_compute_performance-ego": 0.052028536796569824, "sim_compute_robot_state-ego": 0.05442715774882923}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.9152423001012878, "sim_physics": 0.020246987872653537, "survival_time": 2.25, "driven_lanedir": 0.909759304183658, "sim_render-ego": 0.05094322098626031, "in-drivable-lane": 0, "agent_compute-ego": 0.1113521311018202, "deviation-heading": 0.1984807025035912, "set_robot_commands": 0.07920975155300564, "deviation-center-line": 0.1210437666399845, "driven_lanedir_consec": 0.909759304183658, "sim_compute_sim_state": 0.03424611621432834, "sim_compute_performance-ego": 0.057874435848659936, "sim_compute_robot_state-ego": 0.059841346740722653}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.9157141892899948, "sim_physics": 0.019877662261327107, "survival_time": 2.3999999999999995, "driven_lanedir": 0.910302279861786, "sim_render-ego": 0.04740318655967712, "in-drivable-lane": 0, "agent_compute-ego": 0.10558220247427624, "deviation-heading": 0.2150484716453277, "set_robot_commands": 0.06907506783803304, "deviation-center-line": 0.13882443387861454, "driven_lanedir_consec": 0.910302279861786, "sim_compute_sim_state": 0.03071821729342143, "sim_compute_performance-ego": 0.05362041791280111, "sim_compute_robot_state-ego": 0.05372434854507446}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.4367438864906803, "sim_physics": 0.0194853333865895, "survival_time": 3.399999999999996, "driven_lanedir": 0.9542197633771314, "sim_render-ego": 0.049567022744347065, "in-drivable-lane": 0.9999999999999964, "agent_compute-ego": 0.1059987229459426, "deviation-heading": 0.26115230540596424, "set_robot_commands": 0.07398403742734123, "deviation-center-line": 0.0985043663838993, "driven_lanedir_consec": 0.9542197633771314, "sim_compute_sim_state": 0.03053857298458324, "sim_compute_performance-ego": 0.05357067374622121, "sim_compute_robot_state-ego": 0.055910657433902514}}
set_robot_commands_max0.10098746137798956
set_robot_commands_mean0.0786996124128834
set_robot_commands_median0.07398403742734123
set_robot_commands_min0.06907506783803304
sim_compute_performance-ego_max0.07194779504020259
sim_compute_performance-ego_mean0.05780837186889094
sim_compute_performance-ego_median0.05362041791280111
sim_compute_performance-ego_min0.052028536796569824
sim_compute_robot_state-ego_max0.0751253254008743
sim_compute_robot_state-ego_mean0.05980576717388063
sim_compute_robot_state-ego_median0.055910657433902514
sim_compute_robot_state-ego_min0.05372434854507446
sim_compute_sim_state_max0.04057747013164016
sim_compute_sim_state_mean0.0330421703423145
sim_compute_sim_state_median0.03071821729342143
sim_compute_sim_state_min0.02913047508759932
sim_physics_max0.028039531887702224
sim_physics_mean0.02081222358107464
sim_physics_median0.019877662261327107
sim_physics_min0.01641160249710083
sim_render-ego_max0.06632853903860417
sim_render-ego_mean0.05217888870310517
sim_render-ego_median0.049567022744347065
sim_render-ego_min0.046652474186637184
simulation-passed1
survival_time_max3.399999999999996
survival_time_mean2.579999999999999
survival_time_min2.2
No reset possible
196332413jiang pengBaseline solution using reinforcement learningaido2-LFVI-sim-validationstep1-simulationsuccessnoip-172-31-25-98-92050:35:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median-1.3614573264245535
survival_time_median14.950000000000076
deviation-center-line_median1.0119696714086834
in-drivable-lane_median4.050000000000022


other stats
agent_compute-ego_max0.1952957781155904
agent_compute-ego_mean0.1788155647913615
agent_compute-ego_median0.17361107985178628
agent_compute-ego_min0.17030028820037843
deviation-center-line_max1.0119699909612991
deviation-center-line_mean1.0119697191394268
deviation-center-line_min1.0119693200897382
deviation-heading_max6.81542792438532
deviation-heading_mean6.815427924385318
deviation-heading_median6.815427924385318
deviation-heading_min6.815427924385317
driven_any_max4.390824152784938
driven_any_mean4.3908241527849325
driven_any_median4.3908241527849325
driven_any_min4.390824152784927
driven_lanedir_consec_max-1.361452200425545
driven_lanedir_consec_mean-1.3614579701548792
driven_lanedir_consec_min-1.3614641213545289
driven_lanedir_max-2.315814608416598
driven_lanedir_mean-2.3158170164441083
driven_lanedir_median-2.3158169926021728
driven_lanedir_min-2.315819376788188
in-drivable-lane_max4.050000000000022
in-drivable-lane_mean4.050000000000022
in-drivable-lane_min4.050000000000022
per-episodes
details{"udem1-0-0": {"driven_any": 4.390824152784927, "sim_physics": 0.042914228439331056, "survival_time": 14.950000000000076, "driven_lanedir": -2.315816992602138, "sim_render-ego": 0.06738905191421508, "in-drivable-lane": 4.050000000000022, "agent_compute-ego": 0.17030028820037843, "deviation-heading": 6.815427924385318, "set_robot_commands": 0.09710419575373332, "deviation-center-line": 1.0119693200897382, "driven_lanedir_consec": -1.361457326424502, "sim_compute_sim_state": 0.04095041672388713, "sim_compute_performance-ego": 0.0708923045794169, "sim_compute_robot_state-ego": 0.0754766313234965, "sim_compute_robot_state-npc0": 0.07351468086242675, "sim_compute_robot_state-npc1": 0.07083796739578246, "sim_compute_robot_state-npc2": 0.07134899616241455, "sim_compute_robot_state-npc3": 0.0727089262008667}, "udem1-1-0": {"driven_any": 4.390824152784938, "sim_physics": 0.04497409343719482, "survival_time": 14.950000000000076, "driven_lanedir": -2.315814608416598, "sim_render-ego": 0.06777490615844727, "in-drivable-lane": 4.050000000000022, "agent_compute-ego": 0.17361107985178628, "deviation-heading": 6.815427924385319, "set_robot_commands": 0.0987984005610148, "deviation-center-line": 1.0119699909612991, "driven_lanedir_consec": -1.361452200425545, "sim_compute_sim_state": 0.04004632790883382, "sim_compute_performance-ego": 0.07245697657267253, "sim_compute_robot_state-ego": 0.07556244055430095, "sim_compute_robot_state-npc0": 0.07065730651219686, "sim_compute_robot_state-npc1": 0.07153990825017294, "sim_compute_robot_state-npc2": 0.07101067463556926, "sim_compute_robot_state-npc3": 0.07174695571263631}, "udem1-2-0": {"driven_any": 4.390824152784927, "sim_physics": 0.04389215548833211, "survival_time": 14.950000000000076, "driven_lanedir": -2.315819376788188, "sim_render-ego": 0.0636060365041097, "in-drivable-lane": 4.050000000000022, "agent_compute-ego": 0.17185094912846882, "deviation-heading": 6.815427924385317, "set_robot_commands": 0.09529425700505574, "deviation-center-line": 1.0119699909612945, "driven_lanedir_consec": -1.3614641213545289, "sim_compute_sim_state": 0.03982341527938843, "sim_compute_performance-ego": 0.06953674793243408, "sim_compute_robot_state-ego": 0.07201866149902343, "sim_compute_robot_state-npc0": 0.07014225880304972, "sim_compute_robot_state-npc1": 0.06965395371119182, "sim_compute_robot_state-npc2": 0.06959777116775513, "sim_compute_robot_state-npc3": 0.07234389781951904}, "udem1-3-0": {"driven_any": 4.3908241527849325, "sim_physics": 0.04630169709523519, "survival_time": 14.950000000000076, "driven_lanedir": -2.3158169926021728, "sim_render-ego": 0.06615400473276774, "in-drivable-lane": 4.050000000000022, "agent_compute-ego": 0.1830197286605835, "deviation-heading": 6.81542792438532, "set_robot_commands": 0.1006187653541565, "deviation-center-line": 1.0119696714086834, "driven_lanedir_consec": -1.3614573264245535, "sim_compute_sim_state": 0.042271084785461426, "sim_compute_performance-ego": 0.07254142522811889, "sim_compute_robot_state-ego": 0.07720977942148845, "sim_compute_robot_state-npc0": 0.07294538577397665, "sim_compute_robot_state-npc1": 0.07339427550633748, "sim_compute_robot_state-npc2": 0.0737052607536316, "sim_compute_robot_state-npc3": 0.07448353211085001}, "udem1-4-0": {"driven_any": 4.390824152784936, "sim_physics": 0.05072779099146525, "survival_time": 14.950000000000076, "driven_lanedir": -2.3158171118114455, "sim_render-ego": 0.07017317136128744, "in-drivable-lane": 4.050000000000022, "agent_compute-ego": 0.1952957781155904, "deviation-heading": 6.815427924385317, "set_robot_commands": 0.10674546559651692, "deviation-center-line": 1.0119696222761183, "driven_lanedir_consec": -1.3614588761452664, "sim_compute_sim_state": 0.04514465014139811, "sim_compute_performance-ego": 0.0771464729309082, "sim_compute_robot_state-ego": 0.0833860723177592, "sim_compute_robot_state-npc0": 0.0771009341875712, "sim_compute_robot_state-npc1": 0.0777640430132548, "sim_compute_robot_state-npc2": 0.07755679289499919, "sim_compute_robot_state-npc3": 0.07814574162165323}}
set_robot_commands_max0.10674546559651692
set_robot_commands_mean0.09971221685409544
set_robot_commands_median0.0987984005610148
set_robot_commands_min0.09529425700505574
sim_compute_performance-ego_max0.0771464729309082
sim_compute_performance-ego_mean0.07251478544871012
sim_compute_performance-ego_median0.07245697657267253
sim_compute_performance-ego_min0.06953674793243408
sim_compute_robot_state-ego_max0.0833860723177592
sim_compute_robot_state-ego_mean0.0767307170232137
sim_compute_robot_state-ego_median0.07556244055430095
sim_compute_robot_state-ego_min0.07201866149902343
sim_compute_robot_state-npc0_max0.0771009341875712
sim_compute_robot_state-npc0_mean0.07287211322784423
sim_compute_robot_state-npc0_median0.07294538577397665
sim_compute_robot_state-npc0_min0.07014225880304972
sim_compute_robot_state-npc1_max0.0777640430132548
sim_compute_robot_state-npc1_mean0.0726380295753479
sim_compute_robot_state-npc1_median0.07153990825017294
sim_compute_robot_state-npc1_min0.06965395371119182
sim_compute_robot_state-npc2_max0.07755679289499919
sim_compute_robot_state-npc2_mean0.07264389912287395
sim_compute_robot_state-npc2_median0.07134899616241455
sim_compute_robot_state-npc2_min0.06959777116775513
sim_compute_robot_state-npc3_max0.07814574162165323
sim_compute_robot_state-npc3_mean0.07388581069310506
sim_compute_robot_state-npc3_median0.0727089262008667
sim_compute_robot_state-npc3_min0.07174695571263631
sim_compute_sim_state_max0.04514465014139811
sim_compute_sim_state_mean0.04164717896779378
sim_compute_sim_state_median0.04095041672388713
sim_compute_sim_state_min0.03982341527938843
sim_physics_max0.05072779099146525
sim_physics_mean0.04576199309031168
sim_physics_median0.04497409343719482
sim_physics_min0.042914228439331056
sim_render-ego_max0.07017317136128744
sim_render-ego_mean0.06701943413416545
sim_render-ego_median0.06738905191421508
sim_render-ego_min0.0636060365041097
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible
196212447Nicky EichmannBaseline solution using reinforcement learningaido2-LF-sim-validationstep1-simulationerrornoip-172-31-25-98-92050:01:41
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 88, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 435, in <module>
    wrap(cie)
  File "experiment_manager.py", line 423, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 88, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 >
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
195912449Nicky EichmannBaseline solution using reinforcement learningaido2-LFV-sim-validationstep1-simulationhost-errornoip-172-31-25-98-92050:10:02
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job19591-361995', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job19591-361995', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job19591-361995', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
195882459Liam Paull 🇨🇦challenge-aido_LF-template-rosaido2-LFV-sim-testingstep1-simulationerrornoip-172-31-25-98-92050:00:35
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
195832464Liam Paull 🇨🇦challenge-aido_LF-template-rosaido2-LFVI-sim-validationstep1-simulationerrornoip-172-31-25-98-92050:00:34
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
195732487Liam Paull 🇨🇦challenge-aido_LF-baseline-duckietownaido2-LFVI-sim-testingstep1-simulationerrornoip-172-31-25-98-92050:00:49
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
195472512Rami Al-Naim 🇷🇺challenge-aido_LF-template-rosaido2-LFVI-sim-testingstep1-simulationerrornoip-172-31-25-98-92050:02:13
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
195402533Liam Paull 🇨🇦baseline-IL-sim-tensorflowaido2-LFV-sim-validationstep1-simulationerrornoip-172-31-25-98-92050:01:04
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
195162564Alexander Karavaevchallenge-aido_LF-template-ros - Template solution using ROSaido2-LFV-sim-testingstep1-simulationsuccessnoip-172-31-25-98-92050:08:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5487132829419987
survival_time_median2.3499999999999996
deviation-center-line_median0.11041400208354252
in-drivable-lane_median0


other stats
agent_compute-ego_max0.07873538293336567
agent_compute-ego_mean0.07403490069940676
agent_compute-ego_median0.07477673379386343
agent_compute-ego_min0.06892049312591553
deviation-center-line_max0.1279102488429998
deviation-center-line_mean0.10518151035257096
deviation-center-line_min0.06917270915638028
deviation-heading_max0.4712184191829178
deviation-heading_mean0.3828337640550057
deviation-heading_median0.36368201628851926
deviation-heading_min0.3257229985354382
driven_any_max1.259664216011687
driven_any_mean0.7702827269903464
driven_any_median0.6642785413159032
driven_any_min0.3059207793076613
driven_lanedir_consec_max0.9402632878459154
driven_lanedir_consec_mean0.588039443394122
driven_lanedir_consec_min0.28022225990343186
driven_lanedir_max0.9402632878459154
driven_lanedir_mean0.588039443394122
driven_lanedir_median0.5487132829419987
driven_lanedir_min0.28022225990343186
in-drivable-lane_max1.8499999999999936
in-drivable-lane_mean0.559999999999998
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0590446751809168, "sim_physics": 0.04235625848537538, "survival_time": 4.099999999999993, "driven_lanedir": 0.5205812780744896, "sim_render-ego": 0.0680422521219021, "in-drivable-lane": 1.8499999999999936, "agent_compute-ego": 0.07477673379386343, "deviation-heading": 0.4069710981449338, "set_robot_commands": 0.10761471783242577, "deviation-center-line": 0.105230017637756, "driven_lanedir_consec": 0.5205812780744896, "sim_compute_sim_state": 0.041477598795076696, "sim_compute_performance-ego": 0.07590043544769287, "sim_compute_robot_state-ego": 0.07895291723856111, "sim_compute_robot_state-npc0": 0.07245760429196241, "sim_compute_robot_state-npc1": 0.07247288924891775, "sim_compute_robot_state-npc2": 0.07372355461120605, "sim_compute_robot_state-npc3": 0.07336156542708235}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.3059207793076613, "sim_physics": 0.045116607959453874, "survival_time": 1.3000000000000005, "driven_lanedir": 0.28022225990343186, "sim_render-ego": 0.06184698985173152, "in-drivable-lane": 0, "agent_compute-ego": 0.06892049312591553, "deviation-heading": 0.36368201628851926, "set_robot_commands": 0.10482395612276516, "deviation-center-line": 0.06917270915638028, "driven_lanedir_consec": 0.28022225990343186, "sim_compute_sim_state": 0.04127195248237023, "sim_compute_performance-ego": 0.07828194361466628, "sim_compute_robot_state-ego": 0.08091880724980281, "sim_compute_robot_state-npc0": 0.07081993726583627, "sim_compute_robot_state-npc1": 0.07166121556208684, "sim_compute_robot_state-npc2": 0.07379209078275241, "sim_compute_robot_state-npc3": 0.07040890363546518}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.6642785413159032, "sim_physics": 0.0410768579929433, "survival_time": 2.3499999999999996, "driven_lanedir": 0.6504171082047745, "sim_render-ego": 0.06236492319309965, "in-drivable-lane": 0, "agent_compute-ego": 0.07135750385040933, "deviation-heading": 0.34657428812321933, "set_robot_commands": 0.0948933794143352, "deviation-center-line": 0.1279102488429998, "driven_lanedir_consec": 0.6504171082047745, "sim_compute_sim_state": 0.03889192418849215, "sim_compute_performance-ego": 0.06854949098952273, "sim_compute_robot_state-ego": 0.06927751480264867, "sim_compute_robot_state-npc0": 0.06828035192286715, "sim_compute_robot_state-npc1": 0.0666019155624065, "sim_compute_robot_state-npc2": 0.06845215026368486, "sim_compute_robot_state-npc3": 0.06851057803377192}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.259664216011687, "sim_physics": 0.046177216938563755, "survival_time": 4.549999999999992, "driven_lanedir": 0.9402632878459154, "sim_render-ego": 0.06554188571133457, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.07638438979347983, "deviation-heading": 0.4712184191829178, "set_robot_commands": 0.10632532769507104, "deviation-center-line": 0.1131805740421762, "driven_lanedir_consec": 0.9402632878459154, "sim_compute_sim_state": 0.04239003736894209, "sim_compute_performance-ego": 0.07302587634914524, "sim_compute_robot_state-ego": 0.07877262084038703, "sim_compute_robot_state-npc0": 0.07375858380244328, "sim_compute_robot_state-npc1": 0.0752702833531977, "sim_compute_robot_state-npc2": 0.07359990968809023, "sim_compute_robot_state-npc3": 0.07292077305552723}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5625054231355635, "sim_physics": 0.03176815258829217, "survival_time": 1.900000000000001, "driven_lanedir": 0.5487132829419987, "sim_render-ego": 0.05861048949392218, "in-drivable-lane": 0, "agent_compute-ego": 0.07873538293336567, "deviation-heading": 0.3257229985354382, "set_robot_commands": 0.0904537188379388, "deviation-center-line": 0.11041400208354252, "driven_lanedir_consec": 0.5487132829419987, "sim_compute_sim_state": 0.034558848330849094, "sim_compute_performance-ego": 0.06078318545692846, "sim_compute_robot_state-ego": 0.06489775055333187, "sim_compute_robot_state-npc0": 0.06320709303805702, "sim_compute_robot_state-npc1": 0.0630900357898913, "sim_compute_robot_state-npc2": 0.06097976784957083, "sim_compute_robot_state-npc3": 0.06153353891874615}}
set_robot_commands_max0.10761471783242577
set_robot_commands_mean0.1008222199805072
set_robot_commands_median0.10482395612276516
set_robot_commands_min0.0904537188379388
sim_compute_performance-ego_max0.07828194361466628
sim_compute_performance-ego_mean0.07130818637159111
sim_compute_performance-ego_median0.07302587634914524
sim_compute_performance-ego_min0.06078318545692846
sim_compute_robot_state-ego_max0.08091880724980281
sim_compute_robot_state-ego_mean0.0745639221369463
sim_compute_robot_state-ego_median0.07877262084038703
sim_compute_robot_state-ego_min0.06489775055333187
sim_compute_robot_state-npc0_max0.07375858380244328
sim_compute_robot_state-npc0_mean0.06970471406423322
sim_compute_robot_state-npc0_median0.07081993726583627
sim_compute_robot_state-npc0_min0.06320709303805702
sim_compute_robot_state-npc1_max0.0752702833531977
sim_compute_robot_state-npc1_mean0.06981926790330002
sim_compute_robot_state-npc1_median0.07166121556208684
sim_compute_robot_state-npc1_min0.0630900357898913
sim_compute_robot_state-npc2_max0.07379209078275241
sim_compute_robot_state-npc2_mean0.07010949463906087
sim_compute_robot_state-npc2_median0.07359990968809023
sim_compute_robot_state-npc2_min0.06097976784957083
sim_compute_robot_state-npc3_max0.07336156542708235
sim_compute_robot_state-npc3_mean0.06934707181411856
sim_compute_robot_state-npc3_median0.07040890363546518
sim_compute_robot_state-npc3_min0.06153353891874615
sim_compute_sim_state_max0.04239003736894209
sim_compute_sim_state_mean0.03971807223314605
sim_compute_sim_state_median0.04127195248237023
sim_compute_sim_state_min0.034558848330849094
sim_physics_max0.046177216938563755
sim_physics_mean0.04129901879292569
sim_physics_median0.04235625848537538
sim_physics_min0.03176815258829217
sim_render-ego_max0.0680422521219021
sim_render-ego_mean0.063281308074398
sim_render-ego_median0.06236492319309965
sim_render-ego_min0.05861048949392218
simulation-passed1
survival_time_max4.549999999999992
survival_time_mean2.839999999999997
survival_time_min1.3000000000000005
No reset possible
194632694Konstantin Chaikachallenge-aido_LF-baseline-duckietownaido2-LFV-sim-testingstep1-simulationsuccessnoip-172-31-25-98-92050:26:55
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.1232019533516335
survival_time_median13.70000000000006
deviation-center-line_median0.6704175238781824
in-drivable-lane_median0.6000000000000085


other stats
agent_compute-ego_max0.10323999722798666
agent_compute-ego_mean0.08696449191982994
agent_compute-ego_median0.08392731638720435
agent_compute-ego_min0.08042627222397748
deviation-center-line_max1.297349214302428
deviation-center-line_mean0.7636946188463909
deviation-center-line_min0.3230224981957092
deviation-heading_max6.204489815659745
deviation-heading_mean3.0150893077917384
deviation-heading_median2.7298763046355363
deviation-heading_min1.174497012585199
driven_any_max2.355672537011756
driven_any_mean1.8214908698235797
driven_any_median2.143843313302499
driven_any_min1.1101379372338978
driven_lanedir_consec_max2.1956133957340747
driven_lanedir_consec_mean1.320642366625673
driven_lanedir_consec_min0.9359784522620932
driven_lanedir_max2.1956133957340747
driven_lanedir_mean1.4569241886651911
driven_lanedir_median1.2603554463280324
driven_lanedir_min1.0880625854525316
in-drivable-lane_max5.700000000000081
in-drivable-lane_mean1.460000000000019
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.143843313302499, "sim_physics": 0.0425170277156969, "survival_time": 13.70000000000006, "driven_lanedir": 1.6173875624596834, "sim_render-ego": 0.0676516764355402, "in-drivable-lane": 1.0000000000000062, "agent_compute-ego": 0.08392731638720435, "deviation-heading": 6.204489815659745, "set_robot_commands": 0.10721796968557538, "deviation-center-line": 1.0250886386550633, "driven_lanedir_consec": 0.9359784522620932, "sim_compute_sim_state": 0.04261986182553925, "sim_compute_performance-ego": 0.07385903727399172, "sim_compute_robot_state-ego": 0.07981437836250249, "sim_compute_robot_state-npc0": 0.0747959361459217, "sim_compute_robot_state-npc1": 0.07566485178731654, "sim_compute_robot_state-npc2": 0.07489410543093716, "sim_compute_robot_state-npc3": 0.07500577582059985}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.1101379372338978, "sim_physics": 0.04733443922466702, "survival_time": 7.199999999999982, "driven_lanedir": 1.0880625854525316, "sim_render-ego": 0.06841799285676745, "in-drivable-lane": 0, "agent_compute-ego": 0.08288303348753187, "deviation-heading": 1.174497012585199, "set_robot_commands": 0.10882998175091212, "deviation-center-line": 0.6704175238781824, "driven_lanedir_consec": 1.0880625854525316, "sim_compute_sim_state": 0.04181605080763499, "sim_compute_performance-ego": 0.07617582380771637, "sim_compute_robot_state-ego": 0.08186744484636518, "sim_compute_robot_state-npc0": 0.07629621360037062, "sim_compute_robot_state-npc1": 0.0767356124189165, "sim_compute_robot_state-npc2": 0.07687219480673473, "sim_compute_robot_state-npc3": 0.07700502541330126}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.1824491074220642, "sim_physics": 0.04310445068708432, "survival_time": 7.649999999999981, "driven_lanedir": 1.1232019533516335, "sim_render-ego": 0.06443221896302466, "in-drivable-lane": 0, "agent_compute-ego": 0.08042627222397748, "deviation-heading": 2.0771724805279264, "set_robot_commands": 0.10136186063679214, "deviation-center-line": 0.5025952192005714, "driven_lanedir_consec": 1.1232019533516335, "sim_compute_sim_state": 0.040577028311935125, "sim_compute_performance-ego": 0.07131902220981573, "sim_compute_robot_state-ego": 0.07482745912339953, "sim_compute_robot_state-npc0": 0.06950127221400441, "sim_compute_robot_state-npc1": 0.06962327863655839, "sim_compute_robot_state-npc2": 0.06951773712058472, "sim_compute_robot_state-npc3": 0.07027455560522142}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.31535145414768, "sim_physics": 0.04439557004137104, "survival_time": 14.700000000000074, "driven_lanedir": 1.2603554463280324, "sim_render-ego": 0.06700794875216322, "in-drivable-lane": 5.700000000000081, "agent_compute-ego": 0.08434584027244932, "deviation-heading": 2.7298763046355363, "set_robot_commands": 0.10920665377662296, "deviation-center-line": 0.3230224981957092, "driven_lanedir_consec": 1.2603554463280324, "sim_compute_sim_state": 0.04176543845611365, "sim_compute_performance-ego": 0.0738660259311702, "sim_compute_robot_state-ego": 0.07881604570920775, "sim_compute_robot_state-npc0": 0.07503774052574522, "sim_compute_robot_state-npc1": 0.07607827786685659, "sim_compute_robot_state-npc2": 0.07590355273006724, "sim_compute_robot_state-npc3": 0.07496087810620158}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.355672537011756, "sim_physics": 0.035943799018859864, "survival_time": 14.950000000000076, "driven_lanedir": 2.1956133957340747, "sim_render-ego": 0.0593560520807902, "in-drivable-lane": 0.6000000000000085, "agent_compute-ego": 0.10323999722798666, "deviation-heading": 2.8894109255502856, "set_robot_commands": 0.10283668359120687, "deviation-center-line": 1.297349214302428, "driven_lanedir_consec": 2.1956133957340747, "sim_compute_sim_state": 0.03810878117879232, "sim_compute_performance-ego": 0.06722407182057699, "sim_compute_robot_state-ego": 0.07295995076497395, "sim_compute_robot_state-npc0": 0.06799406051635742, "sim_compute_robot_state-npc1": 0.068286878267924, "sim_compute_robot_state-npc2": 0.06933971881866455, "sim_compute_robot_state-npc3": 0.06407937208811441}}
set_robot_commands_max0.10920665377662296
set_robot_commands_mean0.10589062988822193
set_robot_commands_median0.10721796968557538
set_robot_commands_min0.10136186063679214
sim_compute_performance-ego_max0.07617582380771637
sim_compute_performance-ego_mean0.0724887962086542
sim_compute_performance-ego_median0.07385903727399172
sim_compute_performance-ego_min0.06722407182057699
sim_compute_robot_state-ego_max0.08186744484636518
sim_compute_robot_state-ego_mean0.07765705576128978
sim_compute_robot_state-ego_median0.07881604570920775
sim_compute_robot_state-ego_min0.07295995076497395
sim_compute_robot_state-npc0_max0.07629621360037062
sim_compute_robot_state-npc0_mean0.07272504460047988
sim_compute_robot_state-npc0_median0.0747959361459217
sim_compute_robot_state-npc0_min0.06799406051635742
sim_compute_robot_state-npc1_max0.0767356124189165
sim_compute_robot_state-npc1_mean0.07327777979551439
sim_compute_robot_state-npc1_median0.07566485178731654
sim_compute_robot_state-npc1_min0.068286878267924
sim_compute_robot_state-npc2_max0.07687219480673473
sim_compute_robot_state-npc2_mean0.07330546178139767
sim_compute_robot_state-npc2_median0.07489410543093716
sim_compute_robot_state-npc2_min0.06933971881866455
sim_compute_robot_state-npc3_max0.07700502541330126
sim_compute_robot_state-npc3_mean0.0722651214066877
sim_compute_robot_state-npc3_median0.07496087810620158
sim_compute_robot_state-npc3_min0.06407937208811441
sim_compute_sim_state_max0.04261986182553925
sim_compute_sim_state_mean0.04097743211600307
sim_compute_sim_state_median0.04176543845611365
sim_compute_sim_state_min0.03810878117879232
sim_physics_max0.04733443922466702
sim_physics_mean0.042659057337535826
sim_physics_median0.04310445068708432
sim_physics_min0.035943799018859864
sim_render-ego_max0.06841799285676745
sim_render-ego_mean0.06537317781765714
sim_render-ego_median0.06700794875216322
sim_render-ego_min0.0593560520807902
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean11.640000000000034
survival_time_min7.199999999999982
No reset possible
194492640Egor Zamotaevchallenge-aido_LF-template-ros - Template solution using ROSaido2-LF-sim-testingstep1-simulationsuccessnoip-172-31-25-98-92050:06:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5487126868955534
survival_time_median2.3499999999999996
deviation-center-line_median0.11041377901227004
in-drivable-lane_median0


other stats
agent_compute-ego_max0.07529729985176249
agent_compute-ego_mean0.07064818910010603
agent_compute-ego_median0.07039280344800251
agent_compute-ego_min0.06499813153193547
deviation-center-line_max0.12791029264553877
deviation-center-line_mean0.10518143804729678
deviation-center-line_min0.06917252689874298
deviation-heading_max0.4712184191829178
deviation-heading_mean0.3828337640550056
deviation-heading_median0.3636820162885191
deviation-heading_min0.3257229985354382
driven_any_max1.259664216011687
driven_any_mean0.7702827269903463
driven_any_median0.6642785413158997
driven_any_min0.3059207793076624
driven_lanedir_consec_max0.9402632878459154
driven_lanedir_consec_mean0.5880385612453785
driven_lanedir_consec_min0.2802213062291129
driven_lanedir_max0.9402632878459154
driven_lanedir_mean0.5880385612453785
driven_lanedir_median0.5487126868955534
driven_lanedir_min0.2802213062291129
in-drivable-lane_max1.8499999999999936
in-drivable-lane_mean0.559999999999998
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0590446751809168, "sim_physics": 0.02685137202100056, "survival_time": 4.099999999999993, "driven_lanedir": 0.5205812780744896, "sim_render-ego": 0.06293605304345852, "in-drivable-lane": 1.8499999999999936, "agent_compute-ego": 0.07039280344800251, "deviation-heading": 0.4069710981449338, "set_robot_commands": 0.09895099372398562, "deviation-center-line": 0.105230017637756, "driven_lanedir_consec": 0.5205812780744896, "sim_compute_sim_state": 0.03939242770032185, "sim_compute_performance-ego": 0.06730954240008098, "sim_compute_robot_state-ego": 0.07130969152217959}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.3059207793076624, "sim_physics": 0.023305223538325384, "survival_time": 1.3000000000000005, "driven_lanedir": 0.2802213062291129, "sim_render-ego": 0.05684177692119892, "in-drivable-lane": 0, "agent_compute-ego": 0.06499813153193547, "deviation-heading": 0.3636820162885191, "set_robot_commands": 0.09071938808147724, "deviation-center-line": 0.06917252689874298, "driven_lanedir_consec": 0.2802213062291129, "sim_compute_sim_state": 0.03762274522047777, "sim_compute_performance-ego": 0.06789666872758132, "sim_compute_robot_state-ego": 0.071053514113793}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.6642785413158997, "sim_physics": 0.02483008770232505, "survival_time": 2.3499999999999996, "driven_lanedir": 0.6504142471818213, "sim_render-ego": 0.06458403201813394, "in-drivable-lane": 0, "agent_compute-ego": 0.07529729985176249, "deviation-heading": 0.34657428812321933, "set_robot_commands": 0.10127755936155929, "deviation-center-line": 0.12791029264553877, "driven_lanedir_consec": 0.6504142471818213, "sim_compute_sim_state": 0.04016671789453385, "sim_compute_performance-ego": 0.07198919641210678, "sim_compute_robot_state-ego": 0.07598550776217847}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.259664216011687, "sim_physics": 0.0302948296725095, "survival_time": 4.549999999999992, "driven_lanedir": 0.9402632878459154, "sim_render-ego": 0.06574122460333856, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.07303082549964988, "deviation-heading": 0.4712184191829178, "set_robot_commands": 0.10210090679126782, "deviation-center-line": 0.1131805740421762, "driven_lanedir_consec": 0.9402632878459154, "sim_compute_sim_state": 0.041204098816756365, "sim_compute_performance-ego": 0.07234522274562291, "sim_compute_robot_state-ego": 0.07661407596462376}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5625054231355648, "sim_physics": 0.02523935468573319, "survival_time": 1.900000000000001, "driven_lanedir": 0.5487126868955534, "sim_render-ego": 0.060059459585892525, "in-drivable-lane": 0, "agent_compute-ego": 0.06952188516917981, "deviation-heading": 0.3257229985354382, "set_robot_commands": 0.09482224991447048, "deviation-center-line": 0.11041377901227004, "driven_lanedir_consec": 0.5487126868955534, "sim_compute_sim_state": 0.038515329360961914, "sim_compute_performance-ego": 0.06854822761134098, "sim_compute_robot_state-ego": 0.07485125566783704}}
set_robot_commands_max0.10210090679126782
set_robot_commands_mean0.09757421957455208
set_robot_commands_median0.09895099372398562
set_robot_commands_min0.09071938808147724
sim_compute_performance-ego_max0.07234522274562291
sim_compute_performance-ego_mean0.0696177715793466
sim_compute_performance-ego_median0.06854822761134098
sim_compute_performance-ego_min0.06730954240008098
sim_compute_robot_state-ego_max0.07661407596462376
sim_compute_robot_state-ego_mean0.07396280900612237
sim_compute_robot_state-ego_median0.07485125566783704
sim_compute_robot_state-ego_min0.071053514113793
sim_compute_sim_state_max0.041204098816756365
sim_compute_sim_state_mean0.03938026379861035
sim_compute_sim_state_median0.03939242770032185
sim_compute_sim_state_min0.03762274522047777
sim_physics_max0.0302948296725095
sim_physics_mean0.026104173523978737
sim_physics_median0.02523935468573319
sim_physics_min0.023305223538325384
sim_render-ego_max0.06574122460333856
sim_render-ego_mean0.06203250923440449
sim_render-ego_median0.06293605304345852
sim_render-ego_min0.05684177692119892
simulation-passed1
survival_time_max4.549999999999992
survival_time_mean2.839999999999997
survival_time_min1.3000000000000005
No reset possible
194462645Egor Zamotaevchallenge-aido_LF-template-ros - Template solution using ROSaido2-LFV-sim-validationstep1-simulationhost-errornoip-172-31-25-98-92050:02:26
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job19446-730830', 'ps', '-q', 'solution']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job19446-730830', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job19446-730830', 'ps', '-q', 'solution']' returned non-zero exit status 1.

stdout | 

stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
194122699Konstantin Chaikachallenge-aido_LF-baseline-duckietownaido2-LFVI-sim-validationstep1-simulationsuccessnoip-172-31-25-98-92050:26:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.4575065735206416
survival_time_median3.2499999999999964
deviation-center-line_median0.23022199665281676
in-drivable-lane_median0


other stats
agent_compute-ego_max0.10910807956348766
agent_compute-ego_mean0.10028566392903308
agent_compute-ego_median0.09964600709768442
agent_compute-ego_min0.09552093823750814
deviation-center-line_max1.4336432087553297
deviation-center-line_mean0.6301027813199432
deviation-center-line_min0.07666525860598492
deviation-heading_max3.7109366351875015
deviation-heading_mean1.7612426807883026
deviation-heading_median0.7502839600189561
deviation-heading_min0.6841880973221255
driven_any_max2.3556641052152285
driven_any_mean1.164213484754307
driven_any_median0.4752923056793824
driven_any_min0.21812077720243775
driven_lanedir_consec_max2.235409200150684
driven_lanedir_consec_mean1.077542821734485
driven_lanedir_consec_min0.1747988423031619
driven_lanedir_max2.23726002153108
driven_lanedir_mean1.0782052202839671
driven_lanedir_median0.4575065735206416
driven_lanedir_min0.1747988423031619
in-drivable-lane_max0.8500000000000121
in-drivable-lane_mean0.17000000000000243
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 2.3449296955498475, "sim_physics": 0.05385844310124715, "survival_time": 14.950000000000076, "driven_lanedir": 2.116684797338103, "sim_render-ego": 0.07603572765986125, "in-drivable-lane": 0.8500000000000121, "agent_compute-ego": 0.10074636618296304, "deviation-heading": 2.943803713166762, "set_robot_commands": 0.1301601568857829, "deviation-center-line": 1.4336432087553297, "driven_lanedir_consec": 2.115223625971087, "sim_compute_sim_state": 0.04754151821136474, "sim_compute_performance-ego": 0.08477632999420166, "sim_compute_robot_state-ego": 0.0930199106534322, "sim_compute_robot_state-npc0": 0.08508650859196981, "sim_compute_robot_state-npc1": 0.08579297463099161, "sim_compute_robot_state-npc2": 0.08550958156585693, "sim_compute_robot_state-npc3": 0.08596503734588623}, "udem1-1-0": {"driven_any": 0.21812077720243775, "sim_physics": 0.05582082632816199, "survival_time": 1.6500000000000008, "driven_lanedir": 0.1747988423031619, "sim_render-ego": 0.08216583367550012, "in-drivable-lane": 0, "agent_compute-ego": 0.10910807956348766, "deviation-heading": 0.7170009982461676, "set_robot_commands": 0.13198560656923236, "deviation-center-line": 0.07666525860598492, "driven_lanedir_consec": 0.1747988423031619, "sim_compute_sim_state": 0.04913722385059704, "sim_compute_performance-ego": 0.09385259946187335, "sim_compute_robot_state-ego": 0.09468537388425884, "sim_compute_robot_state-npc0": 0.08669434894214977, "sim_compute_robot_state-npc1": 0.08747716383500533, "sim_compute_robot_state-npc2": 0.09132135998119008, "sim_compute_robot_state-npc3": 0.08748483657836914}, "udem1-2-0": {"driven_any": 0.4752923056793824, "sim_physics": 0.0685802386357234, "survival_time": 3.2499999999999964, "driven_lanedir": 0.4575065735206416, "sim_render-ego": 0.07421175883366511, "in-drivable-lane": 0, "agent_compute-ego": 0.09964600709768442, "deviation-heading": 0.6841880973221255, "set_robot_commands": 0.12249726882347692, "deviation-center-line": 0.23022199665281676, "driven_lanedir_consec": 0.4575065735206416, "sim_compute_sim_state": 0.045558966123140775, "sim_compute_performance-ego": 0.08038598207327036, "sim_compute_robot_state-ego": 0.09462339694683368, "sim_compute_robot_state-npc0": 0.0792160731095534, "sim_compute_robot_state-npc1": 0.08015786317678598, "sim_compute_robot_state-npc2": 0.07892928857069749, "sim_compute_robot_state-npc3": 0.08290820488562951}, "udem1-3-0": {"driven_any": 0.42706054012463945, "sim_physics": 0.062054254240908865, "survival_time": 2.9499999999999975, "driven_lanedir": 0.4047758667268495, "sim_render-ego": 0.07313567905102746, "in-drivable-lane": 0, "agent_compute-ego": 0.09640692856352208, "deviation-heading": 0.7502839600189561, "set_robot_commands": 0.12099213115239547, "deviation-center-line": 0.21636801081226129, "driven_lanedir_consec": 0.4047758667268495, "sim_compute_sim_state": 0.04507081791505975, "sim_compute_performance-ego": 0.08145683094606561, "sim_compute_robot_state-ego": 0.09315770359362588, "sim_compute_robot_state-npc0": 0.08017863661555921, "sim_compute_robot_state-npc1": 0.07758906332112975, "sim_compute_robot_state-npc2": 0.08016461032932087, "sim_compute_robot_state-npc3": 0.07904569981461865}, "udem1-4-0": {"driven_any": 2.3556641052152285, "sim_physics": 0.05326221386591593, "survival_time": 14.950000000000076, "driven_lanedir": 2.23726002153108, "sim_render-ego": 0.0758126982053121, "in-drivable-lane": 0, "agent_compute-ego": 0.09552093823750814, "deviation-heading": 3.7109366351875015, "set_robot_commands": 0.1243363642692566, "deviation-center-line": 1.1936154317733236, "driven_lanedir_consec": 2.235409200150684, "sim_compute_sim_state": 0.04594416697820028, "sim_compute_performance-ego": 0.08168052911758422, "sim_compute_robot_state-ego": 0.09026271184285482, "sim_compute_robot_state-npc0": 0.08311652421951293, "sim_compute_robot_state-npc1": 0.08258322318394978, "sim_compute_robot_state-npc2": 0.08081874052683512, "sim_compute_robot_state-npc3": 0.08203700065612793}}
set_robot_commands_max0.13198560656923236
set_robot_commands_mean0.12599430554002886
set_robot_commands_median0.1243363642692566
set_robot_commands_min0.12099213115239547
sim_compute_performance-ego_max0.09385259946187335
sim_compute_performance-ego_mean0.08443045431859905
sim_compute_performance-ego_median0.08168052911758422
sim_compute_performance-ego_min0.08038598207327036
sim_compute_robot_state-ego_max0.09468537388425884
sim_compute_robot_state-ego_mean0.09314981938420108
sim_compute_robot_state-ego_median0.09315770359362588
sim_compute_robot_state-ego_min0.09026271184285482
sim_compute_robot_state-npc0_max0.08669434894214977
sim_compute_robot_state-npc0_mean0.08285841829574904
sim_compute_robot_state-npc0_median0.08311652421951293
sim_compute_robot_state-npc0_min0.0792160731095534
sim_compute_robot_state-npc1_max0.08747716383500533
sim_compute_robot_state-npc1_mean0.08272005762957249
sim_compute_robot_state-npc1_median0.08258322318394978
sim_compute_robot_state-npc1_min0.07758906332112975
sim_compute_robot_state-npc2_max0.09132135998119008
sim_compute_robot_state-npc2_mean0.0833487161947801
sim_compute_robot_state-npc2_median0.08081874052683512
sim_compute_robot_state-npc2_min0.07892928857069749
sim_compute_robot_state-npc3_max0.08748483657836914
sim_compute_robot_state-npc3_mean0.0834881558561263
sim_compute_robot_state-npc3_median0.08290820488562951
sim_compute_robot_state-npc3_min0.07904569981461865
sim_compute_sim_state_max0.04913722385059704
sim_compute_sim_state_mean0.04665053861567252
sim_compute_sim_state_median0.04594416697820028
sim_compute_sim_state_min0.04507081791505975
sim_physics_max0.0685802386357234
sim_physics_mean0.05871519523439147
sim_physics_median0.05582082632816199
sim_physics_min0.05326221386591593
sim_render-ego_max0.08216583367550012
sim_render-ego_mean0.07627233948507321
sim_render-ego_median0.0758126982053121
sim_render-ego_min0.07313567905102746
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean7.550000000000031
survival_time_min1.6500000000000008
No reset possible
193422885Andrea Daniele 🇮🇹minimal_agent (Python 3)aido2-LFVI-sim-validationstep1-simulationsuccessnoip-172-31-25-98-92050:29:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.6900582707828695
survival_time_median14.700000000000074
deviation-center-line_median0.3591683262706219
in-drivable-lane_median1.200000000000017


other stats
agent_compute-ego_max0.17640529719876571
agent_compute-ego_mean0.1599382952125453
agent_compute-ego_median0.15471284025806492
agent_compute-ego_min0.1439375912186003
deviation-center-line_max0.5528244200733968
deviation-center-line_mean0.3250388716749043
deviation-center-line_min0.12474392863155234
deviation-heading_max1.7841667326335335
deviation-heading_mean1.0687591038626745
deviation-heading_median0.9805073681793997
deviation-heading_min0.6869591234399757
driven_any_max2.196039556148948
driven_any_mean1.706254237612001
driven_any_median2.1660738591449116
driven_any_min0.9910472848979056
driven_lanedir_consec_max1.8205635516580176
driven_lanedir_consec_mean1.3480125540484669
driven_lanedir_consec_min0.616683423521228
driven_lanedir_max2.1929637182965305
driven_lanedir_mean1.482287538796306
driven_lanedir_median1.8205635516580176
driven_lanedir_min0.616683423521228
in-drivable-lane_max2.3999999999999915
in-drivable-lane_mean1.250000000000006
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 2.1660738591449116, "sim_physics": 0.046286760544290345, "survival_time": 14.700000000000074, "driven_lanedir": 1.961201560444252, "sim_render-ego": 0.07037173890743126, "in-drivable-lane": 1.200000000000017, "agent_compute-ego": 0.1757820106688, "deviation-heading": 0.9805073681793997, "set_robot_commands": 0.11112620960287496, "deviation-center-line": 0.3591683262706219, "driven_lanedir_consec": 1.6900582707828695, "sim_compute_sim_state": 0.04374785244870348, "sim_compute_performance-ego": 0.07741398876216136, "sim_compute_robot_state-ego": 0.08248710956703238, "sim_compute_robot_state-npc0": 0.07743302981058757, "sim_compute_robot_state-npc1": 0.0772391533365055, "sim_compute_robot_state-npc2": 0.07681902817317418, "sim_compute_robot_state-npc3": 0.07732661808429121}, "udem1-1-0": {"driven_any": 1.0094322716915194, "sim_physics": 0.04983289308950935, "survival_time": 7.099999999999983, "driven_lanedir": 0.616683423521228, "sim_render-ego": 0.06931325079689563, "in-drivable-lane": 2.3999999999999915, "agent_compute-ego": 0.17640529719876571, "deviation-heading": 0.7757170340260829, "set_robot_commands": 0.11111604327886876, "deviation-center-line": 0.12474392863155234, "driven_lanedir_consec": 0.616683423521228, "sim_compute_sim_state": 0.043680039929672024, "sim_compute_performance-ego": 0.07588456550114592, "sim_compute_robot_state-ego": 0.08178797909911249, "sim_compute_robot_state-npc0": 0.07666364018346222, "sim_compute_robot_state-npc1": 0.07657709423924836, "sim_compute_robot_state-npc2": 0.07555840048991458, "sim_compute_robot_state-npc3": 0.07485881489767156}, "udem1-2-0": {"driven_any": 2.168678216176718, "sim_physics": 0.041716625730870135, "survival_time": 14.750000000000076, "driven_lanedir": 1.8205635516580176, "sim_render-ego": 0.06461297051381257, "in-drivable-lane": 1.7500000000000249, "agent_compute-ego": 0.15471284025806492, "deviation-heading": 1.7841667326335335, "set_robot_commands": 0.09794627124980344, "deviation-center-line": 0.38105433778148035, "driven_lanedir_consec": 1.8205635516580176, "sim_compute_sim_state": 0.03878180536173158, "sim_compute_performance-ego": 0.06815962791442871, "sim_compute_robot_state-ego": 0.073812380483595, "sim_compute_robot_state-npc0": 0.06944710925474006, "sim_compute_robot_state-npc1": 0.06898266501345877, "sim_compute_robot_state-npc2": 0.06786873865935762, "sim_compute_robot_state-npc3": 0.06779884645494363}, "udem1-3-0": {"driven_any": 2.196039556148948, "sim_physics": 0.04060430924097697, "survival_time": 14.950000000000076, "driven_lanedir": 2.1929637182965305, "sim_render-ego": 0.06297622362772624, "in-drivable-lane": 0, "agent_compute-ego": 0.1488537367184957, "deviation-heading": 0.6869591234399757, "set_robot_commands": 0.09336665471394856, "deviation-center-line": 0.5528244200733968, "driven_lanedir_consec": 1.792732084218717, "sim_compute_sim_state": 0.03740458011627197, "sim_compute_performance-ego": 0.06664966344833374, "sim_compute_robot_state-ego": 0.07131583611170451, "sim_compute_robot_state-npc0": 0.06646754582722982, "sim_compute_robot_state-npc1": 0.06795763810475668, "sim_compute_robot_state-npc2": 0.06643844842910766, "sim_compute_robot_state-npc3": 0.06695991357167562}, "udem1-4-0": {"driven_any": 0.9910472848979056, "sim_physics": 0.04036577078547791, "survival_time": 6.849999999999984, "driven_lanedir": 0.8200254400615028, "sim_render-ego": 0.05522838648218308, "in-drivable-lane": 0.8999999999999968, "agent_compute-ego": 0.1439375912186003, "deviation-heading": 1.116445261034381, "set_robot_commands": 0.08900124661243745, "deviation-center-line": 0.20740334561747012, "driven_lanedir_consec": 0.8200254400615028, "sim_compute_sim_state": 0.03451063163089056, "sim_compute_performance-ego": 0.061367583100813146, "sim_compute_robot_state-ego": 0.06531394833195818, "sim_compute_robot_state-npc0": 0.061529500641091896, "sim_compute_robot_state-npc1": 0.06073185823259562, "sim_compute_robot_state-npc2": 0.0614565950240532, "sim_compute_robot_state-npc3": 0.06197603427580673}}
set_robot_commands_max0.11112620960287496
set_robot_commands_mean0.10051128509158665
set_robot_commands_median0.09794627124980344
set_robot_commands_min0.08900124661243745
sim_compute_performance-ego_max0.07741398876216136
sim_compute_performance-ego_mean0.06989508574537658
sim_compute_performance-ego_median0.06815962791442871
sim_compute_performance-ego_min0.061367583100813146
sim_compute_robot_state-ego_max0.08248710956703238
sim_compute_robot_state-ego_mean0.07494345071868051
sim_compute_robot_state-ego_median0.073812380483595
sim_compute_robot_state-ego_min0.06531394833195818
sim_compute_robot_state-npc0_max0.07743302981058757
sim_compute_robot_state-npc0_mean0.07030816514342232
sim_compute_robot_state-npc0_median0.06944710925474006
sim_compute_robot_state-npc0_min0.061529500641091896
sim_compute_robot_state-npc1_max0.0772391533365055
sim_compute_robot_state-npc1_mean0.070297681785313
sim_compute_robot_state-npc1_median0.06898266501345877
sim_compute_robot_state-npc1_min0.06073185823259562
sim_compute_robot_state-npc2_max0.07681902817317418
sim_compute_robot_state-npc2_mean0.06962824215512145
sim_compute_robot_state-npc2_median0.06786873865935762
sim_compute_robot_state-npc2_min0.0614565950240532
sim_compute_robot_state-npc3_max0.07732661808429121
sim_compute_robot_state-npc3_mean0.06978404545687775
sim_compute_robot_state-npc3_median0.06779884645494363
sim_compute_robot_state-npc3_min0.06197603427580673
sim_compute_sim_state_max0.04374785244870348
sim_compute_sim_state_mean0.03962498189745393
sim_compute_sim_state_median0.03878180536173158
sim_compute_sim_state_min0.03451063163089056
sim_physics_max0.04983289308950935
sim_physics_mean0.04376127187822494
sim_physics_median0.041716625730870135
sim_physics_min0.04036577078547791
sim_render-ego_max0.07037173890743126
sim_render-ego_mean0.06450051406560976
sim_render-ego_median0.06461297051381257
sim_render-ego_min0.05522838648218308
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean11.67000000000004
survival_time_min6.849999999999984
No reset possible
193012866Konstantin Chaikachallenge-aido_LF-baseline-duckietownaido2-LF-sim-testingstep1-simulationhost-errornoip-172-31-25-98-92050:08:33
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 990, in run_docker
    return subprocess.check_output(cmd0, cwd=cwd, stderr=sys.stderr)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job19301-84122', 'ps', '-q', 'solution']' returned non-zero exit status 255.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 478, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/project/src/duckietown_challenges_runner/runner.py", line 573, in run_single
    write_logs(wd, project, services=config['services'])
  File "/project/src/duckietown_challenges_runner/runner.py", line 899, in write_logs
    container_id = get_id_for_service(wd, project, service)
  File "/project/src/duckietown_challenges_runner/runner.py", line 585, in get_id_for_service
    o = run_docker(wd, project, cmd, get_output=True)
  File "/project/src/duckietown_challenges_runner/runner.py", line 998, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job19301-84122', 'ps', '-q', 'solution']:

   >  Command '['docker-compose', '-p', 'job19301-84122', 'ps', '-q', 'solution']' returned non-zero exit status 255.

stdout | 

stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
192402952Bhairav MehtaBaseline solution using reinforcement learningaido2-LFVI-sim-testingstep1-simulationsuccessnoip-172-31-25-98-92050:34:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.31014014568290404
survival_time_median14.950000000000076
deviation-center-line_median0.1346279912392476
in-drivable-lane_median7.500000000000043


other stats
agent_compute-ego_max0.2326900974909465
agent_compute-ego_mean0.1854853350321452
agent_compute-ego_median0.17903335332870485
agent_compute-ego_min0.15975534439086914
deviation-center-line_max0.2438363867952777
deviation-center-line_mean0.1462369250959627
deviation-center-line_min0.0944885159547387
deviation-heading_max5.805694788929202
deviation-heading_mean5.707077365020204
deviation-heading_median5.673820644905057
deviation-heading_min5.609164881618511
driven_any_max1.0735552381895146
driven_any_mean1.005350607971314
driven_any_median0.989363307948958
driven_any_min0.9719715863633704
driven_lanedir_consec_max0.3528368176118426
driven_lanedir_consec_mean0.30966617330294
driven_lanedir_consec_min0.28457097882095717
driven_lanedir_max0.3528368176118426
driven_lanedir_mean0.30966617330294
driven_lanedir_median0.31014014568290404
driven_lanedir_min0.28457097882095717
in-drivable-lane_max7.60000000000004
in-drivable-lane_mean7.490000000000042
in-drivable-lane_min7.350000000000041
per-episodes
details{"udem1-0-0": {"driven_any": 0.989363307948958, "sim_physics": 0.05322553157806397, "survival_time": 14.950000000000076, "driven_lanedir": 0.28457097882095717, "sim_render-ego": 0.07450211366017659, "in-drivable-lane": 7.550000000000042, "agent_compute-ego": 0.2326900974909465, "deviation-heading": 5.671256891563117, "set_robot_commands": 0.1200515389442444, "deviation-center-line": 0.0944885159547387, "driven_lanedir_consec": 0.28457097882095717, "sim_compute_sim_state": 0.04527525027592977, "sim_compute_performance-ego": 0.07958590745925903, "sim_compute_robot_state-ego": 0.08909347693125406, "sim_compute_robot_state-npc0": 0.08050858497619628, "sim_compute_robot_state-npc1": 0.08009713729222616, "sim_compute_robot_state-npc2": 0.08013178745905558, "sim_compute_robot_state-npc3": 0.08085083723068237}, "udem1-1-0": {"driven_any": 1.0735552381895146, "sim_physics": 0.040902761618296306, "survival_time": 14.950000000000076, "driven_lanedir": 0.3528368176118426, "sim_render-ego": 0.060097620487213135, "in-drivable-lane": 7.350000000000041, "agent_compute-ego": 0.15975534439086914, "deviation-heading": 5.673820644905057, "set_robot_commands": 0.09293169418970744, "deviation-center-line": 0.2438363867952777, "driven_lanedir_consec": 0.3528368176118426, "sim_compute_sim_state": 0.03638797521591187, "sim_compute_performance-ego": 0.06301568746566773, "sim_compute_robot_state-ego": 0.06715715169906616, "sim_compute_robot_state-npc0": 0.06438514312108358, "sim_compute_robot_state-npc1": 0.06531092802683512, "sim_compute_robot_state-npc2": 0.06331029415130615, "sim_compute_robot_state-npc3": 0.06396291573842366}, "udem1-2-0": {"driven_any": 1.0165457278795544, "sim_physics": 0.04185333569844564, "survival_time": 14.950000000000076, "driven_lanedir": 0.31014014568290404, "sim_render-ego": 0.06148410876592, "in-drivable-lane": 7.60000000000004, "agent_compute-ego": 0.17352457364400228, "deviation-heading": 5.609164881618511, "set_robot_commands": 0.09458048582077026, "deviation-center-line": 0.15411210136738243, "driven_lanedir_consec": 0.31014014568290404, "sim_compute_sim_state": 0.03841041167577107, "sim_compute_performance-ego": 0.06705973386764526, "sim_compute_robot_state-ego": 0.07137993892033895, "sim_compute_robot_state-npc0": 0.06798120180765788, "sim_compute_robot_state-npc1": 0.06792013724644978, "sim_compute_robot_state-npc2": 0.06897556145985921, "sim_compute_robot_state-npc3": 0.0691277543703715}, "udem1-3-0": {"driven_any": 0.9719715863633704, "sim_physics": 0.046111419200897216, "survival_time": 14.950000000000076, "driven_lanedir": 0.2874831955787671, "sim_render-ego": 0.0657302451133728, "in-drivable-lane": 7.500000000000043, "agent_compute-ego": 0.17903335332870485, "deviation-heading": 5.805694788929202, "set_robot_commands": 0.09956874608993532, "deviation-center-line": 0.1346279912392476, "driven_lanedir_consec": 0.2874831955787671, "sim_compute_sim_state": 0.03953250010808309, "sim_compute_performance-ego": 0.06995598236719767, "sim_compute_robot_state-ego": 0.0749052341779073, "sim_compute_robot_state-npc0": 0.07265694936116536, "sim_compute_robot_state-npc1": 0.06985303163528442, "sim_compute_robot_state-npc2": 0.07080966949462891, "sim_compute_robot_state-npc3": 0.06981817960739135}, "udem1-4-0": {"driven_any": 0.975317179475172, "sim_physics": 0.04480419238408406, "survival_time": 14.950000000000076, "driven_lanedir": 0.31329972882022905, "sim_render-ego": 0.06602842410405477, "in-drivable-lane": 7.45000000000004, "agent_compute-ego": 0.1824233063062032, "deviation-heading": 5.775449618085134, "set_robot_commands": 0.10098349491755168, "deviation-center-line": 0.10411963012316702, "driven_lanedir_consec": 0.31329972882022905, "sim_compute_sim_state": 0.04233459949493408, "sim_compute_performance-ego": 0.0723104190826416, "sim_compute_robot_state-ego": 0.07613937457402548, "sim_compute_robot_state-npc0": 0.07095357577006021, "sim_compute_robot_state-npc1": 0.07134546518325806, "sim_compute_robot_state-npc2": 0.07098812500635783, "sim_compute_robot_state-npc3": 0.07210702578226726}}
set_robot_commands_max0.1200515389442444
set_robot_commands_mean0.10162319199244182
set_robot_commands_median0.09956874608993532
set_robot_commands_min0.09293169418970744
sim_compute_performance-ego_max0.07958590745925903
sim_compute_performance-ego_mean0.07038554604848227
sim_compute_performance-ego_median0.06995598236719767
sim_compute_performance-ego_min0.06301568746566773
sim_compute_robot_state-ego_max0.08909347693125406
sim_compute_robot_state-ego_mean0.07573503526051839
sim_compute_robot_state-ego_median0.0749052341779073
sim_compute_robot_state-ego_min0.06715715169906616
sim_compute_robot_state-npc0_max0.08050858497619628
sim_compute_robot_state-npc0_mean0.07129709100723267
sim_compute_robot_state-npc0_median0.07095357577006021
sim_compute_robot_state-npc0_min0.06438514312108358
sim_compute_robot_state-npc1_max0.08009713729222616
sim_compute_robot_state-npc1_mean0.0709053398768107
sim_compute_robot_state-npc1_median0.06985303163528442
sim_compute_robot_state-npc1_min0.06531092802683512
sim_compute_robot_state-npc2_max0.08013178745905558
sim_compute_robot_state-npc2_mean0.07084308751424154
sim_compute_robot_state-npc2_median0.07080966949462891
sim_compute_robot_state-npc2_min0.06331029415130615
sim_compute_robot_state-npc3_max0.08085083723068237
sim_compute_robot_state-npc3_mean0.07117334254582722
sim_compute_robot_state-npc3_median0.06981817960739135
sim_compute_robot_state-npc3_min0.06396291573842366
sim_compute_sim_state_max0.04527525027592977
sim_compute_sim_state_mean0.04038814735412598
sim_compute_sim_state_median0.03953250010808309
sim_compute_sim_state_min0.03638797521591187
sim_physics_max0.05322553157806397
sim_physics_mean0.04537944809595744
sim_physics_median0.04480419238408406
sim_physics_min0.040902761618296306
sim_render-ego_max0.07450211366017659
sim_render-ego_mean0.06556850242614747
sim_render-ego_median0.0657302451133728
sim_render-ego_min0.060097620487213135
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible